{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:33:41Z","timestamp":1791459221704,"version":"4.3.4"},"reference-count":75,"publisher":"Association for Computing Machinery (ACM)","issue":"ISSTA","license":[{"start":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T00:00:00Z","timestamp":1750550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2025,6,22]]},"abstract":"<jats:p>Large language models (LLMs) have demonstrated remarkable performance in code generation, significantly enhancing the coding efficiency of developers. Recent advancements in LLM-based agents have led to significant progress in end-to-end automatic software engineering (ASE), particularly in software maintenance (e.g., fixing software issues) and evolution (e.g., adding new features). Despite these encouraging advances, current research faces two major challenges. First, state-of-the-art performance primarily depends on closed-source models like GPT-4, which significantly limits the technology\u2019s accessibility, and potential for customization in diverse software engineering tasks. This dependence also raises concerns about data privacy, particularly when handling sensitive codebases. Second, these models are predominantly trained on static code data, lacking a deep understanding of the dynamic interactions, iterative problem-solving processes, and evolutionary characteristics inherent in software development. Consequently, they may face challenges in navigating complex project structures and generating contextually relevant solutions, which can affect their practical utility in real-world scenarios.<\/jats:p>\n                  <jats:p>\n                    To address these challenges, our study adopts a software engineering perspective. We recognize that real-world software maintenance and evolution processes encompass not only static code data but also developers\u2019 thought processes, utilization of external tools, and the interaction between different functional personnel. Our objective is to develop an open-source large language model specifically optimized for software improvement, aiming to match the performance of closed-source alternatives while offering greater accessibility and customization potential. Consequently, we introduce the\n                    <jats:bold>Lingma SWE-GPT<\/jats:bold>\n                    series, comprising Lingma SWE-GPT 7B and Lingma SWE-GPT 72B. By learning from and simulating real-world code submission activities, Lingma SWE-GPT systematically incorporates the dynamic interactions and iterative problemsolving inherent in software development process\u2014such as repository understanding, fault localization, and patch generation\u2014thereby achieving a more comprehensive understanding of software improvement processes. We conducted experimental evaluations using SWE-bench-Verified benchmark (comprising 500 real GitHub issues), recently proposed by OpenAI. The results demonstrate that\n                    <jats:bold>Lingma SWE-GPT 72B successfully resolves 30.20% of the GitHub issues<\/jats:bold>\n                    , marking a significant improvement in automatic issue resolution (22.76% relative improvement compared to Llama 3.1 405B), approaching the performance of closed-source models (31.80% issues of GPT-4o resolved). Notably, Lingma SWE-GPT 7B resolves 18.20% of the issues, surpassing the 17.20% resolution rate of Llama 3.1 70B, highlighting the potential for applying smaller models to ASE tasks.\n                  <\/jats:p>","DOI":"10.1145\/3728981","type":"journal-article","created":{"date-parts":[[2025,6,22]],"date-time":"2025-06-22T10:52:56Z","timestamp":1750589576000},"page":"2362-2383","source":"Crossref","is-referenced-by-count":9,"title":["SWE-GPT: A Process-Centric Language Model for Automated Software Improvement"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7630-4113","authenticated-orcid":false,"given":"Yingwei","family":"Ma","sequence":"first","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3779-5885","authenticated-orcid":false,"given":"Rongyu","family":"Cao","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0311-5474","authenticated-orcid":false,"given":"Yongchang","family":"Cao","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5068-2260","authenticated-orcid":false,"given":"Yue","family":"Zhang","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2075-4283","authenticated-orcid":false,"given":"Jue","family":"Chen","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7060-9482","authenticated-orcid":false,"given":"Yibo","family":"Liu","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0034-5853","authenticated-orcid":false,"given":"Yuchen","family":"Liu","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4179-6979","authenticated-orcid":false,"given":"Binhua","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2795-3114","authenticated-orcid":false,"given":"Fei","family":"Huang","sequence":"additional","affiliation":[{"name":"Alibaba Group, 0000-0002-0686-0883, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0686-0883","authenticated-orcid":false,"given":"Yongbin","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,6,22]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)."},{"key":"e_1_3_1_3_2","doi-asserted-by":"crossref","unstructured":"Eman Abdullah AlOmar Anushkrishna Venkatakrishnan Mohamed Wiem Mkaouer Christian Newman and Ali Ouni. 2024. How to refactor this code? An exploratory study on developer-ChatGPT refactoring conversations. In Proceedings of the 21st International Conference on Mining Software Repositories. 202\u2013206.","DOI":"10.1145\/3643991.3645081"},{"key":"e_1_3_1_4_2","unstructured":"Anthropic. 2024. Introducing Claude 3.5 Sonnet. https:\/\/www.anthropic.com\/news\/claude-3-5-sonnet"},{"key":"e_1_3_1_5_2","unstructured":"Anthropic. 2024. Introducing the next generation of Claude. https:\/\/www.anthropic.com\/news\/claude-3-family"},{"key":"e_1_3_1_6_2","unstructured":"Carlos E. Jimenez John Yang Jiayi Geng. 2024. SWE-bench Lite: A Canonical Subset for Efficient Evaluation of Language Models as Software Engineers. https:\/\/www.swebench.com\/lite.html"},{"key":"e_1_3_1_7_2","first-page":"14","volume-title":"Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval","author":"Cavnar William B","year":"1994","unstructured":"William B Cavnar, John M Trenkle, et al. 1994. N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval, Vol. 161175. Ann Arbor, Michigan, 14."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678559"},{"key":"e_1_3_1_9_2","unstructured":"Dong Chen Shaoxin Lin Muhan Zeng Daoguang Zan Jian-Gang Wang Anton Cheshkov Jun Sun Hao Yu Guoliang Dong Artem Aliev et al. 2024. CodeR: Issue Resolving with Multi-Agent and Task Graphs. arXiv preprint arXiv:2406.01304 (2024)."},{"key":"e_1_3_1_10_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Josh Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. (2021). arXiv:2107.03374 [cs.LG]"},{"key":"e_1_3_1_11_2","unstructured":"Xinyun Chen Maxwell Lin Nathanael Sch\u00e4rli and Denny Zhou. 2023. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128 (2023)."},{"key":"e_1_3_1_12_2","unstructured":"Cognition. 2023. Introducing Devin. https:\/\/www.cognition.ai\/introducing-devin"},{"key":"e_1_3_1_13_2","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1145\/3649825","article-title":"Cycle: Learning to self-refine the code generation","author":"Ding Yangruibo","year":"2024","unstructured":"Yangruibo Ding, Marcus J Min, Gail Kaiser, and Baishakhi Ray. 2024. Cycle: Learning to self-refine the code generation. Proceedings of the ACM on Programming Languages 8, OOPSLA1 (2024), 392\u2013418.","journal-title":"Proceedings of the ACM on Programming Languages 8, OOPSLA1"},{"key":"e_1_3_1_14_2","unstructured":"Linyuan Gong Mostafa Elhoushi and Alvin Cheung. 2024. AST-T5: Structure-Aware Pretraining for Code Generation and Understanding. arXiv preprint arXiv:2401.03003 (2024)."},{"key":"e_1_3_1_15_2","unstructured":"Sirui Hong Xiawu Zheng Jonathan Chen Yuheng Cheng Jinlin Wang Ceyao Zhang Zili Wang Steven Ka Shing Yau Zijuan Lin Liyang Zhou et al. 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352 (2023)."},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Xiangbing Huang Yingwei Ma Haifang Zhou Zhijie Jiang Yuanliang Zhang Teng Wang and Shanshan Li. 2023. Towards Better Multilingual Code Search through Cross-Lingual Contrastive Learning. In Proceedings of the 14th Asia-Pacific Symposium on Internetware. 22\u201332.","DOI":"10.1145\/3609437.3609439"},{"key":"e_1_3_1_17_2","unstructured":"Huggingface Open LLM Leaderboard. 2024. Dataset Card for Evaluation run of Qwen. https:\/\/huggingface.co\/datasets\/open-llmleaderboard\/QwenQwen2-72B-details"},{"key":"e_1_3_1_18_2","unstructured":"Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu Lei Zhang Tianyu Liu Jiajun Zhang Bowen Yu Kai Dang et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186 (2024)."},{"key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/SANER56733.2023.00053","volume-title":"2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Jiang Zhijie","year":"2023","unstructured":"Zhijie Jiang, Haixu Xiong, Yingwei Ma, Yao Zhang, Yan Ding, Yun Xiong, and Shanshan Li. 2023. Automatic Code Annotation Generation Based on Heterogeneous Graph Structure. In 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 497\u2013508."},{"key":"e_1_3_1_20_2","unstructured":"Carlos E Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770 (2023)."},{"key":"e_1_3_1_21_2","unstructured":"Jiaolong Kong Mingfei Cheng Xiaofei Xie Shangqing Liu Xiaoning Du and Qi Guo. 2024. ContrastRepair: Enhancing Conversation-Based Automated Program Repair via Contrastive Test Case Pairs. arXiv preprint arXiv:2403.01971 (2024)."},{"key":"e_1_3_1_22_2","unstructured":"Cheryl Lee Chunqiu Steven Xia Jen-tse Huang Zhouruixin Zhu Lingming Zhang and Michael R Lyu. 2024. A Unified Debugging Approach via LLM-Based Multi-Agent Synergy. arXiv preprint arXiv:2404.17153 (2024)."},{"key":"e_1_3_1_23_2","first-page":"275","volume-title":"2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE)","author":"Li Jiaying","year":"2023","unstructured":"Jiaying Li, Yan Lei, Shanshan Li, Haifang Zhou, Yue Yu, Zhouyang Jia, Yingwei Ma, and Teng Wang. 2023. A two-stage framework for ambiguous classification in software engineering. In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 275\u2013286."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597207"},{"key":"e_1_3_1_25_2","unstructured":"Meiziniu Li Dongze Li Jianmeng Liu Jialun Cao Yongqiang Tian and Shing-Chi Cheung. 2024. DLLens: Testing Deep Learning Libraries via LLM-aided Synthesis. arXiv preprint arXiv:2406.07944 (2024)."},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Bingchang Liu Chaoyu Chen Zi Gong Cong Liao Huan Wang Zhichao Lei Ming Liang Dajun Chen Min Shen Hailian Zhou et al. 2024. Mftcoder: Boosting code llms with multitask fine-tuning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 5430\u20135441.","DOI":"10.1145\/3637528.3671609"},{"key":"e_1_3_1_27_2","unstructured":"Junwei Liu Kaixin Wang Yixuan Chen Xin Peng Zhenpeng Chen Lingming Zhang and Yiling Lou. 2024. Large Language Model-Based Agents for Software Engineering: A Survey. arXiv preprint arXiv:2409.02977 (2024)."},{"key":"e_1_3_1_28_2","unstructured":"Jiawei Liu Chunqiu Steven Xia Yuyao Wang and Lingming Zhang. 2024. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_3_1_29_2","unstructured":"Xiangyan Liu Bo Lan Zhiyuan Hu Yang Liu Zhicheng Zhang Wenmeng Zhou Fei Wang and Michael Shieh. 2024. CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databases. arXiv preprint arXiv:2408.03910 (2024)."},{"key":"e_1_3_1_30_2","unstructured":"Yizhou Liu Pengfei Gao Xinchen Wang Chao Peng and Zhao Zhang. 2024. MarsCode Agent: AI-native Automated Bug Fixing. arXiv preprint arXiv:2409.00899 (2024)."},{"key":"e_1_3_1_31_2","unstructured":"Anton Lozhkov Raymond Li Loubna Ben Allal Federico Cassano Joel Lamy-Poirier Nouamane Tazi Ao Tang Dmytro Pykhtar Jiawei Liu Yuxiang Wei et al. 2024. StarCoder 2 and The Stack v2: The Next Generation. arXiv preprint arXiv:2402.19173 (2024)."},{"key":"e_1_3_1_32_2","doi-asserted-by":"crossref","unstructured":"Qinyu Luo Yining Ye Shihao Liang Zhong Zhang Yujia Qin Yaxi Lu Yesai Wu Xin Cong Yankai Lin Yingli Zhang et al. 2024. RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation. arXiv preprint arXiv:2402.16667 (2024).","DOI":"10.18653\/v1\/2024.emnlp-demo.46"},{"key":"e_1_3_1_33_2","unstructured":"Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568 (2023)."},{"key":"e_1_3_1_34_2","unstructured":"Yingwei Ma Yue Liu Yue Yu Yuanliang Zhang Yu Jiang Changjian Wang and Shanshan Li. 2023. At Which Training Stage Does Code Data Help LLMs Reasoning? arXiv preprint arXiv:2309.16298 (2023)."},{"key":"e_1_3_1_35_2","unstructured":"Yingwei Ma Qingping Yang Rongyu Cao Binhua Li Fei Huang and Yongbin Li. 2024. How to Understand Whole Software Repository? arXiv preprint arXiv:2406.01422 (2024)."},{"key":"e_1_3_1_36_2","first-page":"120","volume-title":"2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Ma Yingwei","year":"2023","unstructured":"Yingwei Ma, Yue Yu, Shanshan Li, Zhouyang Jia, Jun Ma, Rulin Xu, Wei Dong, and Xiangke Liao. 2023. Mulcs: Towards a unified deep representation for multilingual code search. In 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 120\u2013131."},{"key":"e_1_3_1_37_2","unstructured":"Meta. 2024. Introducing Llama 3.1. https:\/\/ai.meta.com\/blog\/meta-llama-3-1\/"},{"key":"e_1_3_1_38_2","unstructured":"Niklas Muennighoff Qian Liu Armel Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro Von Werra and Shayne Longpre. 2023. Octopack: Instruction tuning code large language models. arXiv preprint arXiv:2308.07124 (2023)."},{"key":"e_1_3_1_39_2","unstructured":"Ansong Ni Miltiadis Allamanis Arman Cohan Yinlin Deng Kensen Shi Charles Sutton and Pengcheng Yin. 2024. NExT: Teaching Large Language Models to Reason about Code Execution. arXiv preprint arXiv:2404.14662 (2024)."},{"key":"e_1_3_1_40_2","unstructured":"OpenAI. 2024. Introducing GPT-4o. https:\/\/openai.com\/index\/hello-gpt-4o\/"},{"key":"e_1_3_1_41_2","unstructured":"OpenAI. 2024. Introducing SWE-bench Verified. https:\/\/openai.com\/index\/introducing-swe-bench-verified\/"},{"key":"e_1_3_1_42_2","unstructured":"Rangeet Pan Ali Reza Ibrahimzada Rahul Krishna Divya Sankar Lambert Pouguem Wassi Michele Merler Boris Sobolev Raju Pavuluri Saurabh Sinha and Reyhaneh Jabbarvand. 2023. Understanding the effectiveness of large language models in code translation. arXiv preprint arXiv:2308.03109 (2023)."},{"key":"e_1_3_1_43_2","unstructured":"Zhenyu Pan Rongyu Cao Yongchang Cao Yingwei Ma Binhua Li Fei Huang Han Liu and Yongbin Li. 2024. Codev-Bench: How Do LLMs Understand Developer-Centric Code Completion? arXiv preprint arXiv:2410.01353 (2024)."},{"key":"e_1_3_1_44_2","unstructured":"Paul Gauthier. 2024. Aider is ai pair programming in your terminal. https:\/\/aider.chat\/2024"},{"key":"e_1_3_1_45_2","unstructured":"Shuo Ren Daya Guo Shuai Lu Long Zhou Shujie Liu Duyu Tang Neel Sundaresan Ming Zhou Ambrosio Blanco and Shuai Ma. 2020. Codebleu: a method for automatic evaluation of code synthesis. arXiv preprint arXiv:2009.10297 (2020)."},{"key":"e_1_3_1_46_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre Tal Remez et al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950 (2023)."},{"key":"e_1_3_1_47_2","unstructured":"Bo Shen Jiaxin Zhang Taihong Chen Daoguang Zan Bing Geng An Fu Muhan Zeng Ailun Yu Jichuan Ji Jingyang Zhao et al. 2023. Pangu-coder2: Boosting large language models for code with ranking feedback. arXiv preprint arXiv:2307.14936 (2023)."},{"key":"e_1_3_1_48_2","unstructured":"Yuling Shi Songsong Wang Chengcheng Wan and Xiaodong Gu. 2024. From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging. arXiv preprint arXiv:2410.01215 (2024)."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC60848.2023.00025"},{"key":"e_1_3_1_50_2","unstructured":"CodeGemma Team. 2024. Codegemma: Open code models based on gemma. arXiv preprint arXiv:2406.11409 (2024)."},{"key":"e_1_3_1_51_2","unstructured":"Gemini Team Rohan Anil Sebastian Borgeaud Yonghui Wu Jean-Baptiste Alayrac Jiahui Yu Radu Soricut Johan Schalkwyk Andrew M Dai Anja Hauth et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)."},{"issue":"4","key":"e_1_3_1_52_2","first-page":"202","article-title":"Comparison of jaccard, dice, cosine similarity coefficient to find best fitness value for web retrieved documents using genetic algorithm","volume":"2","author":"Thada Vikas","year":"2013","unstructured":"Vikas Thada and Vivek Jaglan. 2013. Comparison of jaccard, dice, cosine similarity coefficient to find best fitness value for web retrieved documents using genetic algorithm. International Journal of Innovations in Engineering and Technology 2, 4 (2013), 202\u2013205.","journal-title":"International Journal of Innovations in Engineering and Technology"},{"key":"e_1_3_1_53_2","unstructured":"Yuchen Tian Weixiang Yan Qian Yang Qian Chen Wen Wang Ziyang Luo and Lei Ma. 2024. CodeHalu: Code Hallucinations in LLMs Driven by Execution-based Verification. arXiv preprint arXiv:2405.00253 (2024)."},{"key":"e_1_3_1_54_2","unstructured":"Xingyao Wang Yangyi Chen Lifan Yuan Yizhe Zhang Yunzhu Li Hao Peng and Heng Ji. 2024. Executable Code Actions Elicit Better LLM Agents. arXiv:2402.01030 [cs.CL]"},{"key":"e_1_3_1_55_2","unstructured":"Yuxiang Wei Zhe Wang Jiawei Liu Yifeng Ding and Lingming Zhang. 2024. Magicoder: Empowering code generation with oss-instruct. In Forty-first International Conference on Machine Learning."},{"key":"e_1_3_1_56_2","unstructured":"Chunqiu Steven Xia Yinlin Deng Soren Dunn and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489 (2024)."},{"key":"e_1_3_1_57_2","doi-asserted-by":"crossref","unstructured":"Chunqiu Steven Xia Matteo Paltenghi Jia Le Tian Michael Pradel and Lingming Zhang. 2024. Fuzz4all: Universal fuzzing with large language models. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering. 1\u201313.","DOI":"10.1145\/3597503.3639121"},{"key":"e_1_3_1_58_2","doi-asserted-by":"crossref","unstructured":"Yifan Xie Zhouyang Jia Shanshan Li Ying Wang Jun Ma Xiaoling Li Haoran Liu Ying Fu and Xiangke Liao. 2024. How to Pet a Two-Headed Snake? Solving Cross-Repository Compatibility Issues with Hera. In Proceedings of the 39th IEEE\/ACM International Conference on Automated Software Engineering. 694\u2013705.","DOI":"10.1145\/3691620.3695064"},{"key":"e_1_3_1_59_2","unstructured":"Ruiyang Xu Jialun Cao Yaojie Lu Hongyu Lin Xianpei Han Ben He Shing-Chi Cheung and Le Sun. 2024. CRUXEval-X: A Benchmark for Multilingual Code Reasoning Understanding and Execution. arXiv preprint arXiv:2408.13001 (2024)."},{"key":"e_1_3_1_60_2","first-page":"1876","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Xue Zhipeng","year":"2023","unstructured":"Zhipeng Xue, Zhipeng Gao, Xing Hu, and Shanping Li. 2023. ACWRecommender: A Tool for Validating Actionable Warnings with Weak Supervision. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1876\u20131880."},{"key":"e_1_3_1_61_2","doi-asserted-by":"crossref","unstructured":"Zhipeng Xue Zhipeng Gao Shaohua Wang Xing Hu Xin Xia and Shanping Li. 2024. SelfPiCo: Self-Guided Partial Code Execution with LLMs. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1389\u20131401.","DOI":"10.1145\/3650212.3680368"},{"key":"e_1_3_1_62_2","unstructured":"Jiwei Yan Jinhao Huang Chunrong Fang Jun Yan and Jian Zhang. 2024. Better Debugging: Combining Static Analysis and LLMs for Explainable Crashing Fault Localization. arXiv preprint arXiv:2408.12070 (2024)."},{"key":"e_1_3_1_63_2","unstructured":"An Yang Baosong Yang Binyuan Hui Bo Zheng Bowen Yu Chang Zhou Chengpeng Li Chengyuan Li Dayiheng Liu Fei Huang et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671 (2024)."},{"key":"e_1_3_1_64_2","doi-asserted-by":"crossref","unstructured":"John Yang Carlos E Jimenez Alexander Wettig Kilian Lieret Shunyu Yao Karthik Narasimhan and Ofir Press. 2024. Swe-agent: Agent-computer interfaces enable automated software engineering. arXiv preprint arXiv:2405.15793 (2024).","DOI":"10.52202\/079017-1601"},{"key":"e_1_3_1_65_2","unstructured":"Zhaojian Yu Xin Zhang Ning Shang Yangyu Huang Can Xu Yishujie Zhao Wenxiang Hu and Qiufeng Yin. 2023. Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation. arXiv preprint arXiv:2312.14187 (2023)."},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.52202\/068431-1126"},{"key":"e_1_3_1_67_2","doi-asserted-by":"crossref","unstructured":"Kechi Zhang Jia Li Ge Li Xianjie Shi and Zhi Jin. 2024. CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges. arXiv preprint arXiv:2401.07339 (2024).","DOI":"10.18653\/v1\/2024.acl-long.737"},{"key":"e_1_3_1_68_2","unstructured":"Kexun Zhang Weiran Yao Zuxin Liu Yihao Feng Zhiwei Liu Rithesh Murthy Tian Lan Lei Li Renze Lou Jiacheng Xu et al. 2024. Diversity empowers intelligence: Integrating expertise of software engineering agents. arXiv preprint arXiv:2408.07060 (2024)."},{"key":"e_1_3_1_69_2","unstructured":"Mengxiao Zhang Yongqiang Tian Zhenyang Xu Yiwen Dong Shin Hwei Tan and Chengnian Sun. 2023. Lampr: Boosting the Effectiveness of Language-Generic Program Reduction via Large Language Models. arXiv preprint arXiv:2312.13064 (2023)."},{"key":"e_1_3_1_70_2","doi-asserted-by":"crossref","unstructured":"Mengxiao Zhang Yongqiang Tian Zhenyang Xu Yiwen Dong Shin Hwei Tan and Chengnian Sun. 2024. LPR: Large Language Models-Aided Program Reduction. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 261\u2013273.","DOI":"10.1145\/3650212.3652126"},{"key":"e_1_3_1_71_2","doi-asserted-by":"crossref","unstructured":"Yuntong Zhang Haifeng Ruan Zhiyu Fan and Abhik Roychoudhury. 2024. Autocoderover: Autonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1592\u20131604.","DOI":"10.1145\/3650212.3680384"},{"key":"e_1_3_1_72_2","unstructured":"Yuwei Zhao Ziyang Luo Yuchen Tian Hongzhan Lin Weixiang Yan Annan Li and Jing Ma. 2024. CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding? arXiv preprint arXiv:2408.10718 (2024)."},{"key":"e_1_3_1_73_2","doi-asserted-by":"crossref","unstructured":"Tianyu Zheng Ge Zhang Tianhao Shen Xueling Liu Bill Yuchen Lin Jie Fu Wenhu Chen and Xiang Yue. 2024. Opencodeinterpreter: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402.14658 (2024).","DOI":"10.18653\/v1\/2024.findings-acl.762"},{"key":"e_1_3_1_74_2","unstructured":"Ming Zhu and Yi Zhou. 2024. MOSS: Enabling Code-Driven Evolution and Context Management for AI Agents. arXiv preprint arXiv:2409.16120 (2024)."},{"key":"e_1_3_1_75_2","unstructured":"Qiming Zhu Jialun Cao Yaojie Lu Hongyu Lin Xianpei Han Le Sun and Shing-Chi Cheung. 2024. DOMAINEVAL: An Auto-Constructed Benchmark for Multi-Domain Code Generation. arXiv preprint arXiv:2408.13204 (2024)."},{"key":"e_1_3_1_76_2","article-title":"DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence","author":"Zhu Qihao","year":"2024","unstructured":"Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. 2024. DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence. arXiv preprint arXiv:2406.11931 (2024).","journal-title":"arXiv preprint arXiv:2406.11931 (2024)"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T13:54:48Z","timestamp":1787493288000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,22]]},"references-count":75,"journal-issue":{"issue":"ISSTA","published-print":{"date-parts":[[2025,6,22]]}},"alternative-id":["10.1145\/3728981"],"URL":"https:\/\/doi.org\/10.1145\/3728981","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,22]]}}}