{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T22:40:22Z","timestamp":1759012822907,"version":"3.44.0"},"publisher-location":"New York, NY, USA","reference-count":53,"publisher":"ACM","funder":[{"name":"Defense Advanced Research Projects Agency?s (DARPA) SciFy program","award":["HR00112520300"],"award-info":[{"award-number":["HR00112520300"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,9,28]]},"DOI":"10.1145\/3746059.3747652","type":"proceedings-article","created":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T07:44:49Z","timestamp":1758959089000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Answering Developer Questions with Annotated Agent-Discovered Program Traces"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5077-354X","authenticated-orcid":false,"given":"Litao","family":"Yan","sequence":"first","affiliation":[{"name":"University of Pennsylvania, Philadelphia, PA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6407-1316","authenticated-orcid":false,"given":"Jeffrey","family":"Tao","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, PA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1737-1276","authenticated-orcid":false,"given":"Lydia B","family":"Chilton","sequence":"additional","affiliation":[{"name":"Columbia University, New York, NY, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1523-3347","authenticated-orcid":false,"given":"Andrew","family":"Head","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, PA, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,9,27]]},"reference":[{"key":"e_1_3_3_3_2_2","doi-asserted-by":"crossref","unstructured":"F.\u00a0E. Allen and J. Cocke. 1976. A program data flow analysis procedure. Commun. ACM (1976) 137.","DOI":"10.1145\/360018.360025"},{"key":"e_1_3_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC60511.2024.00011"},{"key":"e_1_3_3_3_4_2","unstructured":"Mike Barnett Robert DeLine Akash Lal and Shaz Qadeer. 2014. Get me here: Using verification tools to answer developer questions. Microsoft Research Tech. Rep. MSR-TR-2014-10 (2014) 10\u00a0pages."},{"key":"e_1_3_3_3_5_2","doi-asserted-by":"crossref","unstructured":"David Binkley and Mark Harman. 2004. A survey of empirical results on program slicing. Advances in Computers 62 105178 (2004) 105\u2013178.","DOI":"10.1016\/S0065-2458(03)62003-6"},{"key":"e_1_3_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02217-3"},{"key":"e_1_3_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753706"},{"key":"e_1_3_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/365024.365325"},{"key":"e_1_3_3_3_9_2","first-page":"505","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","author":"Chi Ed\u00a0H.","year":"2003","unstructured":"Ed\u00a0H. Chi, Adam Rosien, Gesara Supattanasiri, Amanda Williams, Christiaan Royer, Celia Chow, Erica Robles, Brinda Dalal, Julie Chen, and Steve Cousins. 2003. The bloodhound project: automating discovery of web usability issues using the InfoScent\u03c0 simulator. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 505\u2013512."},{"key":"e_1_3_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523445"},{"key":"e_1_3_3_3_11_2","unstructured":"Cursor. 2024. Cursor: The AI-Powered Code Editor. https:\/\/www.cursor.so\/ Accessed April 1 2025."},{"key":"e_1_3_3_3_12_2","unstructured":"DEDALe. 2024. DEDALe: Understand Your Code Like Your Best Engineers. https:\/\/dedale.dev Accessed April 1 2025."},{"key":"e_1_3_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810331"},{"key":"e_1_3_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713715"},{"key":"e_1_3_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713581"},{"key":"e_1_3_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714141"},{"key":"e_1_3_3_3_17_2","doi-asserted-by":"crossref","unstructured":"Scott\u00a0D. Fleming Chris Scaffidi David Piorkowski Margaret Burnett Rachel Bellamy Joseph Lawrance and Irwin Kwan. 2013. An Information Foraging Theory Perspective on Tools for Debugging Refactoring and Reuse Tasks. ACM Trans. Softw. Eng. Methodol. Article 14 (2013) 41\u00a0pages.","DOI":"10.1145\/2430545.2430551"},{"key":"e_1_3_3_3_18_2","unstructured":"GitHub. 2024. GitHub Copilot. https:\/\/github.com\/features\/copilot Accessed April 1 2025."},{"key":"e_1_3_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/1321631.1321637"},{"key":"e_1_3_3_3_20_2","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian journal of statistics (1979) 65\u201370."},{"key":"e_1_3_3_3_21_2","unstructured":"Dong Huang Jie\u00a0M Zhang Michael Luck Qingwen Bu Yuhao Qing and Heming Cui. 2023. Agentcoder: Multi-agent-based code generation with iterative testing and optimisation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2312.13010 (2023)."},{"key":"e_1_3_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/985692.985712"},{"key":"e_1_3_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/1368088.1368130"},{"key":"e_1_3_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518942"},{"key":"e_1_3_3_3_25_2","doi-asserted-by":"crossref","unstructured":"Amy\u00a0J. Ko Brad\u00a0A. Myers Michael\u00a0J. Coblenz and Htet\u00a0Htet Aung. 2006. An Exploratory Study of How Developers Seek Relate and Collect Relevant Information during Software Maintenance Tasks. IEEE Transactions on Software Engineering 32 (2006) 971\u2013987.","DOI":"10.1109\/TSE.2006.116"},{"key":"e_1_3_3_3_26_2","unstructured":"Jing\u00a0Yu Koh Stephen McAleer Daniel Fried and Ruslan Salakhutdinov. 2024. Tree Search for Language Model Agents. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.01476 (2024)."},{"key":"e_1_3_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806829"},{"key":"e_1_3_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/1937117.1937125"},{"key":"e_1_3_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2011.6070388"},{"key":"e_1_3_3_3_30_2","doi-asserted-by":"crossref","unstructured":"Joseph Lawrance Christopher Bogart Margaret Burnett Rachel Bellamy Kyle Rector and Scott\u00a0D. Fleming. 2013. How Programmers Debug Revisited: An Information Foraging Theory Perspective. IEEE Transactions on Software Engineering 39 (2013) 197\u2013215.","DOI":"10.1109\/TSE.2010.111"},{"key":"e_1_3_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714104"},{"key":"e_1_3_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713748"},{"key":"e_1_3_3_3_33_2","unstructured":"Sanwal Manish. 2024. An autonomous multi-agent llm framework for agile software development. International Journal of Trend in Scientific Research and Development 8 5 (2024) 892\u2013898."},{"key":"e_1_3_3_3_34_2","unstructured":"NASA Ames Research Center. 2023. TLX @ NASA Ames - Home. https:\/\/humansystems.arc.nasa.gov\/groups\/TLX\/ Accessed September 15 2023."},{"key":"e_1_3_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985911"},{"key":"e_1_3_3_3_36_2","unstructured":"OpenAI. 2024. OpenAI o3-mini. https:\/\/openai.com\/index\/openai-o3-mini. Accessed: 2025-06-16."},{"key":"e_1_3_3_3_37_2","first-page":"157","volume-title":"Proceedings of the 8th Australasian Conference on Computing Education-Volume 52","author":"Parsons Dale","year":"2006","unstructured":"Dale Parsons and Patricia Haden. 2006. Parson\u2019s programming puzzles: a fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education-Volume 52. 157\u2013163."},{"key":"e_1_3_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/223904.223911"},{"key":"e_1_3_3_3_39_2","doi-asserted-by":"crossref","unstructured":"Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological review 106 4 (1999) 643.","DOI":"10.1037\/\/0033-295X.106.4.643"},{"key":"e_1_3_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713357"},{"key":"e_1_3_3_3_41_2","doi-asserted-by":"crossref","unstructured":"Franklin\u00a0E. Satterthwaite. 1946. An approximate distribution of estimates of variance components. Biometrics bulletin 2 6 (1946) 110\u2013114.","DOI":"10.2307\/3002019"},{"key":"e_1_3_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/1181775.1181779"},{"key":"e_1_3_3_3_43_2","doi-asserted-by":"crossref","unstructured":"Jonathan Sillito Gail\u00a0C. Murphy and Kris De\u00a0Volder. 2008. Asking and Answering Questions during a Programming Change Task. IEEE Transactions on Software Engineering 34 (2008) 434\u2013451.","DOI":"10.1109\/TSE.2008.26"},{"key":"e_1_3_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1117696.1117701"},{"key":"e_1_3_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2017.8103445"},{"key":"e_1_3_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.findings-naacl.176"},{"key":"e_1_3_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4380-9_16"},{"key":"e_1_3_3_3_48_2","unstructured":"Workorb. 2024. Comparing Latency of GPT-4o vs. GPT-4o Mini. https:\/\/www.workorb.com\/blog\/comparing-latency-of-gpt-4o-vs-gpt-4o-mini. Accessed: 2025-07-13."},{"key":"e_1_3_3_3_49_2","unstructured":"Chunqiu\u00a0Steven Xia Yinlin Deng Soren Dunn and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.01489 (2024)."},{"key":"e_1_3_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3654777.3676374"},{"key":"e_1_3_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445654"},{"key":"e_1_3_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642239"},{"key":"e_1_3_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME58846.2023.00038"},{"key":"e_1_3_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714154"}],"event":{"name":"UIST '25: The 38th Annual ACM Symposium on User Interface Software and Technology","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGGRAPH ACM Special Interest Group on Computer Graphics and Interactive Techniques"],"location":"Busan Republic of Korea","acronym":"UIST '25"},"container-title":["Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3746059.3747652","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,27]],"date-time":"2025-09-27T22:07:44Z","timestamp":1759010864000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3746059.3747652"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,27]]},"references-count":53,"alternative-id":["10.1145\/3746059.3747652","10.1145\/3746059"],"URL":"https:\/\/doi.org\/10.1145\/3746059.3747652","relation":{},"subject":[],"published":{"date-parts":[[2025,9,27]]},"assertion":[{"value":"2025-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}