{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T19:06:42Z","timestamp":1776107202202,"version":"3.50.1"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>\n                    Many IDEs support an autocomplete feature, which may increase developer productivity by reducing typing requirements and by providing convenient access to relevant information. However, to date, there has been no evaluation of the actual benefit of autocomplete to programmers. We conducted a between-subjects experiment\n                    <jats:inline-formula>\n                      <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" display=\"inline\">\n                        <mml:mo>(<\/mml:mo>\n                        <mml:mtext>N<\/mml:mtext>\n                        <mml:mo>=<\/mml:mo>\n                        <mml:mn>32<\/mml:mn>\n                        <mml:mo>)<\/mml:mo>\n                      <\/mml:math>\n                    <\/jats:inline-formula>\n                    using an eye tracker to evaluate the costs and benefits of IDE-based autocomplete features to programmers who use an unfamiliar API. Participants who used autocomplete spent significantly less time reading documentation and got significantly higher scores on our post-study API knowledge test, indicating that it helped them learn more about the API. However, autocomplete did not significantly reduce the number of keystrokes required to finish tasks. We conclude that the primary benefit of autocomplete is in providing information, not in reducing time spent typing.\n                  <\/jats:p>","DOI":"10.1145\/3660765","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"1284-1306","source":"Crossref","is-referenced-by-count":5,"title":["An Analysis of the Costs and Benefits of Autocomplete in IDEs"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-7198-0786","authenticated-orcid":false,"given":"Shaokang","family":"Jiang","sequence":"first","affiliation":[{"name":"University of California San Diego, La Jolla, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9369-4069","authenticated-orcid":false,"given":"Michael","family":"Coblenz","sequence":"additional","affiliation":[{"name":"University of California San Diego, La Jolla, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","unstructured":"Maike Ahrens. 2020. Towards Automatic Capturing of Traceability Links by Combining Eye Tracking and Interaction Data. In 2020 IEEE 28th International Requirements Engineering Conference (RE). 434\u2013439. https:\/\/doi.org\/10.1109\/RE48521.2020.00064 10.1109\/RE48521.2020.00064","DOI":"10.1109\/RE48521.2020.00064"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.39"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196471"},{"key":"e_1_3_1_5_2","article-title":"On the opportunities and risks of foundation models","author":"Bommasani Rishi","year":"2021","unstructured":"Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. 2021. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021).","journal-title":"arXiv preprint arXiv:2108.07258"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-021-01554-0"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","unstructured":"Matteo Ciniselli Nathan Cooper Luca Pascarella Denys Poshyvanyk Massimiliano Di Penta and Gabriele Bavota. 2021. An Empirical Study on the Usage of BERT Models for Code Completion. In 2021 IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR). 108\u2013119. https:\/\/doi.org\/10.1109\/MSR52588.2021.00024 10.1109\/MSR52588.2021.00024","DOI":"10.1109\/MSR52588.2021.00024"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbr.2022.100213"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568299"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","unstructured":"Sarah Fakhoury Devjeet Roy Harry Pines Tyler Cleveland Cole S. Peterson Venera Arnaoudova Bonita Sharif and Jonathan Maletic. 2021. gazel: Supporting Source Code Edits in Eye-Tracking Studies. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). 69\u201372. https:\/\/doi.org\/10.1109\/ICSE-Companion52605.2021.00038 10.1109\/ICSE-Companion52605.2021.00038","DOI":"10.1109\/ICSE-Companion52605.2021.00038"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2110192.2110203"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3204493.3208343"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","unstructured":"Vincent J. Hellendoorn Sebastian Proksch Harald C. Gall and Alberto Bacchelli. 2019. When Code Completion Fails: A Case Study on Real-World Completions. In 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE). 960\u2013970. https:\/\/doi.org\/10.1109\/ICSE.2019.00101 10.1109\/ICSE.2019.00101","DOI":"10.1109\/ICSE.2019.00101"},{"key":"e_1_3_1_14_2","volume-title":"Eye tracking: A comprehensive guide to methods and measures","author":"Holmqvist Kenneth","year":"2011","unstructured":"Kenneth Holmqvist, Marcus Nystr\u00f6m, Richard Andersson, Richard Dewhurst, Halszka Jarodzka, and Joost Van de Weijer. 2011. Eye tracking: A comprehensive guide to methods and measures. OUP Oxford."},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1007\/978-3-030-98404-5_36","volume-title":"Intelligent Human Computer Interaction","author":"Housholder Andrew","year":"2022","unstructured":"Andrew Housholder, Jonathan Reaban, Aira Peregrino, Georgia Votta, and Tauheed Khan Mohd. 2022. Evaluating Accuracy of the Tobii Eye Tracker 5. In Intelligent Human Computer Interaction, Jong-Hoon Kim, Madhusudan Singh, Javed Khan, Uma Shanker Tiwary, Marigankar Sur, and Dhananjay Singh (Eds.). Springer International Publishing, Cham, 379\u2013390."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510454.3522684"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-2852-3_9"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","unstructured":"Shaokang Jiang and Michael Coblenz. 2024. Artifact for \u201cAn Analysis of the Costs and Benefits of Autocomplete in IDEs\u201d. https:\/\/doi.org\/10.1145\/3580435 10.1145\/3580435","DOI":"10.1145\/3580435"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.3390\/s22030912"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379503.3403553"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","unstructured":"Jingxuan Li Rui Huang Wei Li Kai Yao and Weiguo Tan. 2021. Toward Less Hidden Cost of Code Completion with Acceptance and Ranking Models. In 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME). 195\u2013205. https:\/\/doi.org\/10.1109\/ICSME52107.2021.00024 10.1109\/ICSME52107.2021.00024","DOI":"10.1109\/ICSME52107.2021.00024"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3608128"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.15346\/hc.v6i1.3"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","unstructured":"Roberto Minelli and Michele Lanza. 2013. Visualizing the workflow of developers. In 2013 First IEEE Working Conference on Software Visualization (VISSOFT). 1\u20134. https:\/\/doi.org\/10.1109\/VISSOFT.2013.6650531 10.1109\/VISSOFT.2013.6650531","DOI":"10.1109\/VISSOFT.2013.6650531"},{"key":"e_1_3_1_25_2","article-title":"The Impact of AI on Developer Productivity: Evidence from GitHub Copilot","author":"Peng Sida","year":"2023","unstructured":"Sida Peng, Eirini Kalliamvakou, Peter Cihon, and Mert Demirer. 2023. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv:2302.06590 [cs.SE]","journal-title":"arXiv:2302.06590 [cs.SE]"},{"key":"e_1_3_1_26_2","article-title":"\" It\u2019s Weird That it Knows What I Want\": Usability and Interactions with Copilot for Novice Programmers","author":"Prather James","year":"2023","unstructured":"James Prather, Brent N Reeves, Paul Denny, Brett A Becker, Juho Leinonen, Andrew Luxton-Reilly, Garrett Powell, James Finnie-Ansley, and Eddie Antonio Santos. 2023. \" It\u2019s Weird That it Knows What I Want\": Usability and Interactions with Copilot for Novice Programmers. arXiv preprint arXiv:2304.02491 (2023).","journal-title":"arXiv preprint arXiv:2304.02491"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","unstructured":"Sebastian Proksch Sarah Nadi Sven Amann and Mira Mezini. 2017. Enriching in-IDE process information with finegrained source code history. In 2017 IEEE 24th International Conference on Software Analysis Evolution and Reengineering (SANER). 250\u2013260. https:\/\/doi.org\/10.1109\/SANER.2017.7884626 10.1109\/SANER.2017.7884626","DOI":"10.1109\/SANER.2017.7884626"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2666356.2594321"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1080\/17470210902816461"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379337.3415813"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","unstructured":"Bonita Sharif Cole Peterson Drew Guarnera Corey Bryant Zachary Buchanan Vlas Zyrianov and Jonathan Maletic. 2019. Practical Eye Tracking with iTrace. In 2019 IEEE\/ACM 6th International Workshop on Eye Movements in Programming (EMIP). 41\u201342. https:\/\/doi.org\/10.1109\/EMIP.2019.00015 10.1109\/EMIP.2019.00015","DOI":"10.1109\/EMIP.2019.00015"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/1321631.1321663"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519665"},{"key":"e_1_3_1_34_2","unstructured":"Boris Velichkovsky Markus Joos Jens Helmert and Sebastian Pannasch. 2005. Two visual systems and their eye movements: Evidence from static and dynamic scene perception. Proceedings of the XXVII Conference of the Cognitive Science Society (01 2005)."},{"key":"e_1_3_1_35_2","unstructured":"WakaTime. 2023. WakaTime 2022 Programming Stats \u2014 wakatime.com. https:\/\/wakatime.com\/blog\/57-wakatime-2022-programming-stats. [Accessed 05-09-2023]."},{"key":"e_1_3_1_36_2","unstructured":"Chaozheng Wang Junhao Hu Cuiyun Gao Yu Jin Tao Xie Hailiang Huang Zhenyu Lei and Yuetang Deng. 2023. Practitioners\u2019 Expectations on Code Completion. arXiv:2301.03846 [cs.SE]"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510457.3513061"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534864"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660765","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3660765","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T08:06:29Z","timestamp":1770192389000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3660765"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":37,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3660765"],"URL":"https:\/\/doi.org\/10.1145\/3660765","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}