{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T17:52:29Z","timestamp":1776275549866,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":40,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T00:00:00Z","timestamp":1715385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,5,11]]},"DOI":"10.1145\/3613904.3641936","type":"proceedings-article","created":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T08:39:12Z","timestamp":1715416752000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":70,"title":["Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-9660-9878","authenticated-orcid":false,"given":"Hussein","family":"Mozannar","sequence":"first","affiliation":[{"name":"IDSS and CSAIL, MIT, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7741-3861","authenticated-orcid":false,"given":"Gagan","family":"Bansal","sequence":"additional","affiliation":[{"name":"Microsoft Research, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4986-7794","authenticated-orcid":false,"given":"Adam","family":"Fourney","sequence":"additional","affiliation":[{"name":"Microsoft Research, Microsoft Research, United States"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8823-0614","authenticated-orcid":false,"given":"Eric","family":"Horvitz","sequence":"additional","affiliation":[{"name":"Microsoft, United States"}]}],"member":"320","published-online":{"date-parts":[[2024,5,11]]},"reference":[{"key":"e_1_3_3_3_1_1","unstructured":"Amazon. 2022. ML-powered coding companion \u2013 Amazon CodeWhisperer. https:\/\/aws.amazon.com\/codewhisperer\/"},{"key":"e_1_3_3_3_2_1","volume-title":"Is github\u2019s copilot as bad as humans at introducing vulnerabilities in code?Empirical Software Engineering 28, 6","author":"Asare Owura","year":"2023","unstructured":"Owura Asare, Meiyappan Nagappan, and N Asokan. 2023. Is github\u2019s copilot as bad as humans at introducing vulnerabilities in code?Empirical Software Engineering 28, 6 (2023), 1\u201324."},{"key":"e_1_3_3_3_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586030"},{"key":"e_1_3_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(77)80039-4"},{"key":"e_1_3_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/358841.358847"},{"key":"e_1_3_3_3_6_1","volume-title":"Language models are few-shot learners. Advances in neural information processing systems 33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared\u00a0D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877\u20131901."},{"key":"e_1_3_3_3_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/358886.358895"},{"key":"e_1_3_3_3_8_1","volume-title":"Jared Kaplan, Harri Edwards, Yuri Burda","author":"Chen Mark","year":"2021","unstructured":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de\u00a0Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 1, 1 (2021), 1\u20132."},{"key":"e_1_3_3_3_9_1","volume-title":"Github copilot ai pair programmer: Asset or liability?Journal of Systems and Software 203","author":"Dakhel Arghavan\u00a0Moradi","year":"2023","unstructured":"Arghavan\u00a0Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel\u00a0C Desmarais, and Zhen Ming\u00a0Jack Jiang. 2023. Github copilot ai pair programmer: Asset or liability?Journal of Systems and Software 203 (2023), 111734."},{"key":"e_1_3_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/18.243461"},{"key":"e_1_3_3_3_11_1","volume-title":"Out of the bleu: how should we assess quality of the code generation models?Journal of Systems and Software 203","author":"Evtikhiev Mikhail","year":"2023","unstructured":"Mikhail Evtikhiev, Egor Bogomolov, Yaroslav Sokolov, and Timofey Bryksin. 2023. Out of the bleu: how should we assess quality of the code generation models?Journal of Systems and Software 203 (2023), 111741."},{"key":"e_1_3_3_3_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3453928"},{"key":"e_1_3_3_3_13_1","volume-title":"The SPACE of Developer Productivity: There\u2019s more to it than you think.Queue 19, 1","author":"Forsgren Nicole","year":"2021","unstructured":"Nicole Forsgren, Margaret-Anne Storey, Chandra Maddila, Thomas Zimmermann, Brian Houck, and Jenna Butler. 2021. The SPACE of Developer Productivity: There\u2019s more to it than you think.Queue 19, 1 (2021), 20\u201348."},{"key":"e_1_3_3_3_14_1","unstructured":"Github. 2022. GitHub copilot - your AI pair programmer. https:\/\/github.com\/features\/copilot"},{"key":"e_1_3_3_3_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00101"},{"key":"e_1_3_3_3_16_1","volume-title":"Measuring Coding Challenge Competence With APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). ., ., 1\u20132.","author":"Hendrycks Dan","year":"2021","unstructured":"Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, 2021. Measuring Coding Challenge Competence With APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). ., ., 1\u20132."},{"key":"e_1_3_3_3_17_1","volume-title":"Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. ., ., 1\u201319","author":"Jiang Ellen","year":"2022","unstructured":"Ellen Jiang, Edwin Toh, Alejandra Molina, Kristen Olson, Claire Kayacik, Aaron Donsbach, Carrie\u00a0J Cai, and Michael Terry. 2022. Discovering the Syntax and Strategies of Natural Language Programming with Generative Language Models. In CHI Conference on Human Factors in Computing Systems. ., ., 1\u201319."},{"key":"e_1_3_3_3_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/235833.236054"},{"key":"e_1_3_3_3_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197091.3197107"},{"key":"e_1_3_3_3_20_1","unstructured":"Eirini Kalliamvakou. 2022. Research: Quantifying github copilot\u2019s impact on developer productivity and happiness. https:\/\/github.blog\/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness\/"},{"key":"e_1_3_3_3_21_1","volume-title":"Competition-level code generation with alphacode. arXiv preprint arXiv:2203.07814 1, 1","author":"Li Yujia","year":"2022","unstructured":"Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R\u00e9mi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin\u00a0Dal Lago, 2022. Competition-level code generation with alphacode. arXiv preprint arXiv:2203.07814 1, 1 (2022), 1\u20132."},{"key":"e_1_3_3_3_22_1","volume-title":"Understanding the Usability of AI Programming Assistants. arXiv preprint arXiv:2303.17125 1, 1","author":"Liang T","year":"2023","unstructured":"Jenny\u00a0T Liang, Chenyang Yang, and Brad\u00a0A Myers. 2023. Understanding the Usability of AI Programming Assistants. arXiv preprint arXiv:2303.17125 1, 1 (2023), 1\u20132."},{"key":"e_1_3_3_3_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/223904.223969"},{"key":"e_1_3_3_3_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3145904"},{"key":"e_1_3_3_3_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"e_1_3_3_3_26_1","volume-title":"Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs?arXiv preprint arXiv:2112.02125 1, 1","author":"Pearce Hammond","year":"2021","unstructured":"Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2021. Can OpenAI Codex and Other Large Language Models Help Us Fix Security Bugs?arXiv preprint arXiv:2112.02125 1, 1 (2021), 1\u20132."},{"key":"e_1_3_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389279"},{"key":"e_1_3_3_3_28_1","volume-title":"The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590 1, 1","author":"Peng Sida","year":"2023","unstructured":"Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590 1, 1 (2023), 1\u20132."},{"key":"e_1_3_3_3_29_1","volume-title":"Usability and Interactions with Copilot for Novice Programmers. arXiv preprint arXiv:2304.02491 1, 1","author":"Prather James","year":"2023","unstructured":"James Prather, Brent\u00a0N Reeves, Paul Denny, Brett\u00a0A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie\u00a0Antonio Santos. 2023. \" It\u2019s Weird That it Knows What I Want\": Usability and Interactions with Copilot for Novice Programmers. arXiv preprint arXiv:2304.02491 1, 1 (2023), 1\u20132."},{"key":"e_1_3_3_3_30_1","volume-title":"What is it like to program with artificial intelligence?arXiv preprint arXiv:2208.06213 1, 1","author":"Sarkar Advait","year":"2022","unstructured":"Advait Sarkar, Andrew\u00a0D Gordon, Carina Negreanu, Christian Poelitz, Sruti\u00a0Srinivasa Ragavan, and Ben Zorn. 2022. What is it like to program with artificial intelligence?arXiv preprint arXiv:2208.06213 1, 1 (2022), 1\u20132."},{"key":"e_1_3_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/356835.356840"},{"key":"e_1_3_3_3_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00089"},{"key":"e_1_3_3_3_33_1","unstructured":"Maxim\u00a0Tabachnyk Tabachnyk and Stoyan Nikolov. 2022. ML-enhanced code completion improves developer productivity. https:\/\/ai.googleblog.com\/2022\/07\/ml-enhanced-code-completion-improves"},{"key":"e_1_3_3_3_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519665"},{"key":"e_1_3_3_3_35_1","volume-title":"A survey on semi-supervised learning. Machine learning 109, 2","author":"Van\u00a0Engelen E","year":"2020","unstructured":"Jesper\u00a0E Van\u00a0Engelen and Holger\u00a0H Hoos. 2020. A survey on semi-supervised learning. Machine learning 109, 2 (2020), 373\u2013440."},{"key":"e_1_3_3_3_36_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan\u00a0N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017), 1\u20132."},{"key":"e_1_3_3_3_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1149933.1149940"},{"key":"e_1_3_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397481.3450656"},{"key":"e_1_3_3_3_39_1","volume-title":"Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programming. arXiv preprint arXiv:2306.05153 1, 1","author":"Wu Tongshuang","year":"2023","unstructured":"Tongshuang Wu, Kenneth Koedinger, 2023. Is AI the better programming partner? Human-Human Pair Programming vs. Human-AI pAIr Programming. arXiv preprint arXiv:2306.05153 1, 1 (2023), 1\u20132."},{"key":"e_1_3_3_3_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534864"}],"event":{"name":"CHI '24: CHI Conference on Human Factors in Computing Systems","location":"Honolulu HI USA","acronym":"CHI '24","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGACCESS ACM Special Interest Group on Accessible Computing"]},"container-title":["Proceedings of the CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3613904.3641936","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3613904.3641936","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:28Z","timestamp":1750291048000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3613904.3641936"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,11]]},"references-count":40,"alternative-id":["10.1145\/3613904.3641936","10.1145\/3613904"],"URL":"https:\/\/doi.org\/10.1145\/3613904.3641936","relation":{},"subject":[],"published":{"date-parts":[[2024,5,11]]},"assertion":[{"value":"2024-05-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}