{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T17:13:04Z","timestamp":1776100384923,"version":"3.50.1"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T00:00:00Z","timestamp":1746144000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2025,5,2]]},"abstract":"<jats:p>Software engineers have historically relied on human-powered Q&amp;A platforms like Stack Overflow (SO) as coding aids. With the rise of generative AI, developers have started to adopt AI chatbots, such as ChatGPT, in their software development process. Recognizing the potential parallels between human-powered Q&amp;A platforms and AI-powered question-based chatbots, we investigate and compare how developers integrate this assistance into their real-world coding experiences by conducting a thematic analysis of 1700+ Reddit posts. Through a comparative study of SO and ChatGPT, we identified each platform's strengths, use cases, and barriers. Our findings suggest that ChatGPT offers fast, clear, comprehensive responses and fosters a more respectful environment than SO. However, concerns about ChatGPT's reliability stem from its overly confident tone and the absence of validation mechanisms like SO's voting system. Based on these findings, we synthesized the design implications for future GenAI code assistants and recommend a workflow leveraging each platform's unique features to improve developer experiences.<\/jats:p>","DOI":"10.1145\/3710927","type":"journal-article","created":{"date-parts":[[2025,5,20]],"date-time":"2025-05-20T11:36:19Z","timestamp":1747740979000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["'Always Nice and Confident, Sometimes Wrong': Developer's Experiences Engaging Generative AI Chatbots Versus Human-Powered Q&amp;A Platforms"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6084-5131","authenticated-orcid":false,"given":"Jiachen","family":"Li","sequence":"first","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8486-9384","authenticated-orcid":false,"given":"Elizabeth D.","family":"Mynatt","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3891-5460","authenticated-orcid":false,"given":"Varun","family":"Mishra","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1187-9298","authenticated-orcid":false,"given":"Jonathan","family":"Bell","sequence":"additional","affiliation":[{"name":"Northeastern University, Boston, MA, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2012. Dark Matter Developers: The Unseen 99%. https:\/\/www.hanselman.com\/blog\/dark-matter-developers-the-unseen-99"},{"key":"e_1_2_1_2_1","unstructured":"2018. Stack Overflow isn't very welcoming. It's time for that to change. https:\/\/medium.com\/@jayhanlon\/welcomewagon-dd57cbdd54d9"},{"key":"e_1_2_1_3_1","unstructured":"2022. Why posting GPT and ChatGPT generated answers is not currently acceptable. https:\/\/stackoverflow.com\/help\/gptpolicy"},{"key":"e_1_2_1_4_1","unstructured":"2023. Code of conduct Stack Overflow. https:\/\/stackoverflow.com\/conduct"},{"key":"e_1_2_1_5_1","unstructured":"2023. Copilot by Github. https:\/\/github.com\/features\/copilot\/"},{"key":"e_1_2_1_6_1","unstructured":"2023. Developer sentiment around AI\/ML. https:\/\/stackoverflow.co\/labs\/developer-sentiment-ai-ml\/"},{"key":"e_1_2_1_7_1","unstructured":"2023. Home page of subreddit: r\/ChatGPTCoding. https:\/\/www.reddit.com\/r\/ChatGPTCoding\/"},{"key":"e_1_2_1_8_1","unstructured":"2023. PRAW: The Python Reddit API Wrapper. https:\/\/praw.readthedocs.io\/en\/stable\/"},{"key":"e_1_2_1_9_1","unstructured":"2023. Reddit posts: What are some good subreddits for Programming\/CS\/Technology? https:\/\/www.reddit.com\/r\/ learnprogramming\/comments\/3e0bu9\/what_are_some_good_subreddits_for\/"},{"key":"e_1_2_1_10_1","unstructured":"2023. Stack Exchange all Q&A sites home page. https:\/\/stackexchange.com\/sites?view=list#users"},{"key":"e_1_2_1_11_1","volume-title":"Stack Overflow Annual Developer Survey","year":"2023","unstructured":"2023. Stack Overflow Annual Developer Survey 2023. https:\/\/survey.stackoverflow.co\/2023\/"},{"key":"e_1_2_1_12_1","unstructured":"2023. Temporary policy: Generative AI (e.g. ChatGPT) is banned. https:\/\/meta.stackoverflow.com\/questions\/421831\/temporary-policy-generative-ai-e-g-chatgpt-is-banned"},{"key":"e_1_2_1_13_1","unstructured":"2023. Temporary policy: Generative AI (e.g. ChatGPT) is banned. https:\/\/meta.stackoverflow.com\/questions\/421831\/ temporary-policy-generative-ai-e-g-chatgpt-is-banned"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/91478.91485"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1108\/DTA-07-2017-0054"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.2466\/03.CP.3.4"},{"key":"e_1_2_1_17_1","volume-title":"The Age of Snippet Programming: Toward Understanding Developer Communities in Stack Overflow and Reddit. In Companion Proceedings of the ACM Web Conference","author":"Antelmi Alessia","year":"2023","unstructured":"Alessia Antelmi, Gennaro Cordasco, Daniele De Vinco, and Carmine Spagnuolo. 2023. The Age of Snippet Programming: Toward Understanding Developer Communities in Stack Overflow and Reddit. In Companion Proceedings of the ACM Web Conference 2023. 1218--1224."},{"key":"e_1_2_1_18_1","volume-title":"Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garc\u00eda, Sergio Gil-L\u00f3pez, Daniel Molina, Richard Benjamins, et al.","author":"Arrieta Alejandro Barredo","year":"2020","unstructured":"Alejandro Barredo Arrieta, Natalia D\u00edaz-Rodr\u00edguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador Garc\u00eda, Sergio Gil-L\u00f3pez, Daniel Molina, Richard Benjamins, et al. 2020. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information fusion 58 (2020), 82--115."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196430"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2023.12.014"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196321.3196333"},{"key":"e_1_2_1_22_1","volume-title":"Unraveling the Impact: An Empirical Investigation of ChatGPT's Exclusion from Stack Overflow. Available at SSRN 4481959","author":"Borwankar Sameer","year":"2023","unstructured":"Sameer Borwankar and Warut Khern-am nuai. 2023. Unraveling the Impact: An Empirical Investigation of ChatGPT's Exclusion from Stack Overflow. Available at SSRN 4481959 (2023)."},{"key":"e_1_2_1_23_1","volume-title":"Using thematic analysis in psychology. Qualitative research in psychology 3, 2","author":"Braun Virginia","year":"2006","unstructured":"Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110454"},{"key":"e_1_2_1_25_1","volume-title":"How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools. arXiv preprint arXiv:2212.03491","author":"Cheng Ruijia","year":"2022","unstructured":"Ruijia Cheng, Ruotong Wang, Thomas Zimmermann, and Denae Ford. 2022. \" It would work for me too\": How Online Communities Shape Software Developers' Trust in AI-Powered Code Generation Tools. arXiv preprint arXiv:2212.03491 (2022)."},{"key":"e_1_2_1_26_1","volume-title":"Bastin Tony Roy Savarimuthu, and Stephen Cranefield","author":"Cheriyan Jithin","year":"2021","unstructured":"Jithin Cheriyan, Bastin Tony Roy Savarimuthu, and Stephen Cranefield. 2021. Norm violation in online communities--A study of Stack Overflow comments. In Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XIII: International Workshops COIN 2017 and COINE 2020, Sao Paulo, Brazil, May 8--9, 2017 and Virtual Event, May 9, 2020, Revised Selected Papers. Springer, 20--34."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639201"},{"key":"e_1_2_1_28_1","volume-title":"Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1","author":"Corbin Juliet M","year":"1990","unstructured":"Juliet M Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1 (1990), 3--21."},{"key":"e_1_2_1_29_1","volume-title":"Towards Explainability in NLP: Analyzing and Calculating Word Saliency through Word Properties. arXiv preprint arXiv:2207.08083","author":"Dong Jialiang","year":"2022","unstructured":"Jialiang Dong, Zhitao Guan, Longfei Wu, Zijian Zhang, and Xiaojiang Du. 2022. Towards Explainability in NLP: Analyzing and Calculating Word Saliency through Word Properties. arXiv preprint arXiv:2207.08083 (2022)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0031360"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3502076"},{"key":"e_1_2_1_32_1","volume-title":"Saliency learning: Teaching the model where to pay attention. arXiv preprint arXiv:1902.08649","author":"Ghaeini Reza","year":"2019","unstructured":"Reza Ghaeini, Xiaoli Z Fern, Hamed Shahbazi, and Prasad Tadepalli. 2019. Saliency learning: Teaching the model where to pay attention. arXiv preprint arXiv:1902.08649 (2019)."},{"key":"e_1_2_1_33_1","volume-title":"The discovery of grounded theory","author":"Glaser Barney G","year":"1968","unstructured":"Barney G Glaser, Anselm L Strauss, and Elizabeth Strutzel. 1968. The discovery of grounded theory; strategies for qualitative research. Nursing research 17, 4 (1968), 364."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v8i1.14550"},{"key":"e_1_2_1_35_1","volume-title":"Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions. arXiv preprint arXiv:2308.02312","author":"Kabir Samia","year":"2023","unstructured":"Samia Kabir, David N Udo-Imeh, Bonan Kou, and Tianyi Zhang. 2023. Who Answers It Better? An In-Depth Analysis of ChatGPT and Stack Overflow Answers to Software Engineering Questions. arXiv preprint arXiv:2308.02312 (2023)."},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Sam Lau and Philip J Guo. 2023. From\" Ban It Till We Understand It\" to\" Resistance is Futile\": How University Programming Instructors Plan to Adapt as More Students Use AI Code Generation and Explanation Tools such as ChatGPT and GitHub Copilot. (2023).","DOI":"10.1145\/3568813.3600138"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979366"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979366"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2012.6405249"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1057\/s41275-017-0075-5"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.150"},{"key":"e_1_2_1_42_1","volume-title":"Fernando Figueira Filho, and Alexey Zagalsky","author":"Storey Margaret-Anne","year":"2014","unstructured":"Margaret-Anne Storey, Leif Singer, Brendan Cleary, Fernando Figueira Filho, and Alexey Zagalsky. 2014. The (r) evolution of social media in software engineering. Future of software engineering proceedings (2014), 100--116."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985907"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519665"},{"key":"e_1_2_1_45_1","first-page":"1024","article-title":"How do users revise answers on technical Q&A websites? A case study on Stack Overflow","volume":"46","author":"Chen Tse-Hsun","year":"2018","unstructured":"ShaoweiWang, Tse-Hsun Chen, and Ahmed E Hassan. 2018. How do users revise answers on technical Q&A websites? A case study on Stack Overflow. IEEE Transactions on Software Engineering 46, 9 (2018), 1024--1038.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_2_1_46_1","volume-title":"Can ChatGPT Kill User-Generated Q&A Platforms? Available at SSRN 4448938","author":"Xue Junzhi","year":"2023","unstructured":"Junzhi Xue, Lizheng Wang, Jinyang Zheng, Yongjun Li, and Yong Tan. 2023. Can ChatGPT Kill User-Generated Q&A Platforms? Available at SSRN 4448938 (2023)."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2631775.2631809"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/RSSE.2012.6233407"},{"key":"e_1_2_1_49_1","volume-title":"5 Models on Political Issues in the US and China. arXiv preprint arXiv:2312.09917","author":"Zhou Di","year":"2023","unstructured":"Di Zhou and Yinxian Zhang. 2023. Red AI? Inconsistent Responses from GPT3. 5 Models on Political Issues in the US and China. arXiv preprint arXiv:2312.09917 (2023)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10180-z"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710927","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3710927","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T09:20:31Z","timestamp":1755768031000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3710927"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,5,2]]}},"alternative-id":["10.1145\/3710927"],"URL":"https:\/\/doi.org\/10.1145\/3710927","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"assertion":[{"value":"2025-05-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}