{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T14:10:02Z","timestamp":1784815802443,"version":"3.55.0"},"reference-count":171,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T00:00:00Z","timestamp":1748044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>Integrating Large Language Models (LLMs) into autonomous agents marks a significant shift in the research landscape by offering cognitive abilities that are competitive with human planning and reasoning. This article explores the transformative potential of integrating Large Language Models into Multi-Agent (LMA) systems for addressing complex challenges in software engineering (SE). By leveraging the collaborative and specialized abilities of multiple agents, LMA systems enable autonomous problem-solving, improve robustness, and provide scalable solutions for managing the complexity of real-world software projects. In this article, we conduct a systematic review of recent primary studies to map the current landscape of LMA applications across various stages of the software development lifecycle (SDLC). To illustrate current capabilities and limitations, we perform two case studies to demonstrate the effectiveness of state-of-the-art LMA frameworks. Additionally, we identify critical research gaps and propose a comprehensive research agenda focused on enhancing individual agent capabilities and optimizing agent synergy. Our work outlines a forward-looking vision for developing fully autonomous, scalable, and trustworthy LMA systems, laying the foundation for the evolution of Software Engineering 2.0.<\/jats:p>","DOI":"10.1145\/3712003","type":"journal-article","created":{"date-parts":[[2025,1,13]],"date-time":"2025-01-13T11:03:31Z","timestamp":1736766211000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":171,"title":["LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3370-8585","authenticated-orcid":false,"given":"Junda","family":"He","sequence":"first","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6919-2149","authenticated-orcid":false,"given":"Christoph","family":"Treude","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,24]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Sahar Abdelnabi Amr Gomaa Sarath Sivaprasad Lea Sch\u00f6nherr and Mario Fritz. 2023. LLM-deliberation: Evaluating LLMs with interactive multi-agent negotiation games. arXiv:2309.17234. Retrieved from https:\/\/arxiv.org\/abs\/2309.17234"},{"key":"e_1_3_2_3_2","unstructured":"Pekka Abrahamsson Outi Salo Jussi Ronkainen and Juhani Warsta. 2017. Agile software development methods: Review and analysis. arXiv:1709.08439. Retrieved from https:\/\/arxiv.org\/abs\/1709.08439"},{"key":"e_1_3_2_4_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et al. 2023. Gpt-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_2_5_2","volume-title":"LLM-Coordination: Developing Coordinating Agents with Large Language Models","author":"Agashe Saaket","year":"2023","unstructured":"Saaket Agashe. 2023. LLM-Coordination: Developing Coordinating Agents with Large Language Models. University of California, Santa Cruz."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.3991\/ijim.v14i11.13269"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.01.002"},{"key":"e_1_3_2_8_2","unstructured":"Antonis Antoniades Albert \u00d6rwall Kexun Zhang Yuxi Xie Anirudh Goyal and William Wang. 2024. SWE-search: Enhancing software agents with Monte Carlo tree search and iterative refinement. arXiv:2410.20285. Retrieved from https:\/\/arxiv.org\/abs\/2410.20285"},{"key":"e_1_3_2_9_2","unstructured":"Daman Arora Atharv Sonwane Nalin Wadhwa Abhav Mehrotra Saiteja Utpala Ramakrishna Bairi Aditya Kanade and Nagarajan Natarajan. 2024. MASAI: Modular architecture for software-engineering AI agents. arXiv:2406.11638. Retrieved from https:\/\/arxiv.org\/abs\/2406.11638"},{"key":"e_1_3_2_10_2","unstructured":"Mohammadmehdi Ataei Hyunmin Cheong Daniele Grandi Ye Wang Nigel Morris and Alexander Tessier. 2024. Elicitron: An LLM agent-based simulation framework for design requirements elicitation. arXiv:2404.16045. Retrieved from https:\/\/arxiv.org\/abs\/2404.16045"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2311878121"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1137\/0217014"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453146"},{"key":"e_1_3_2_14_2","volume-title":"Multi-Agent Oriented Programming: Programming Multi-Agent Systems Using JaCaMo","author":"Boissier Olivier","year":"2020","unstructured":"Olivier Boissier, Rafael H. Bordini, Jomi Hubner, and Alessandro Ricci. 2020. Multi-Agent Oriented Programming: Programming Multi-Agent Systems Using JaCaMo. Mit Press."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/1571641"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1147\/sj.352.0151"},{"key":"e_1_3_2_17_2","unstructured":"Weilin Cai Juyong Jiang Fan Wang Jing Tang Sunghun Kim and Jiayi Huang. 2024. A survey on mixture of experts. arXiv:2407.06204. Retrieved from https:\/\/arxiv.org\/abs\/2407.06204"},{"key":"e_1_3_2_18_2","unstructured":"Yuzhe Cai Shaoguang Mao Wenshan Wu Zehua Wang Yaobo Liang Tao Ge Chenfei Wu Wang You Ting Song Yan Xia et al. 2023. Low-code LLM: Visual programming over LLMs. arXiv:2304.08103. Retrieved from https:\/\/arxiv.org\/abs\/2304.08103"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3702973"},{"key":"e_1_3_2_20_2","unstructured":"Dong Chen Shaoxin Lin Muhan Zeng Daoguang Zan Jian-Gang Wang Anton Cheshkov Jun Sun Hao Yu Guoliang Dong Artem Aliev et al. 2024. CodeR: Issue resolving with multi-agent and task graphs. arXiv:2406.01304. Retrieved from https:\/\/arxiv.org\/abs\/2406.01304"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3363562"},{"key":"e_1_3_2_22_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Chen Weize","year":"2023","unstructured":"Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. 2023. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In Proceedings of the 12th International Conference on Learning Representations."},{"key":"e_1_3_2_23_2","unstructured":"Yongchao Chen Jacob Arkin Yang Zhang Nicholas Roy and Chuchu Fan. 2023. Scalable multi-robot collaboration with large language models: Centralized or decentralized systems? arXiv:2309.15943. Retrieved from https:\/\/arxiv.org\/abs\/2309.15943"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3652155"},{"key":"e_1_3_2_25_2","unstructured":"Yuheng Cheng Ceyao Zhang Zhengwen Zhang Xiangrui Meng Sirui Hong Wenhao Li Zihao Wang Zekai Wang Feng Yin Junhua Zhao et al. 2024. Exploring large language model based intelligent agents: definitions methods and prospects. arXiv:2401.03428. Retrieved from https:\/\/arxiv.org\/abs\/2401.03428"},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Michael G. Christel and Kyo C. Kang. 1992. Issues in requirements elicitation.","DOI":"10.21236\/ADA258932"},{"issue":"03","key":"e_1_3_2_27_2","first-page":"1","article-title":"An introduction to agile methods","volume":"62","author":"Cohen David","year":"2004","unstructured":"David Cohen, Mikael Lindvall, and Patricia Costa. 2004. An introduction to agile methods. Advances in Computing 62, 03 (2004), 1\u201366.","journal-title":"Advances in Computing"},{"key":"e_1_3_2_28_2","first-page":"93","article-title":"Big data and due process: Toward a framework to redress predictive privacy Harms","volume":"55","author":"Crawford Kate","year":"2014","unstructured":"Kate Crawford and Jason Schultz. 2014. Big data and due process: Toward a framework to redress predictive privacy Harms. Boston College Law Review 55 (2014), 93.","journal-title":"Boston College Law Review"},{"key":"e_1_3_2_29_2","unstructured":"DBLP computer Science bibliography. 2024. DBLP: Computer Science bibliography. Retrieved November 13 2024 from https:\/\/dblp.org"},{"key":"e_1_3_2_30_2","unstructured":"Gelei Deng Yi Liu V\u00edctor Mayoral-Vilches Peng Liu Yuekang Li Yuan Xu Tianwei Zhang Yang Liu Martin Pinzger and Stefan Rass. 2023. Pentestgpt: An llm-empowered automatic penetration testing tool. arXiv:2308.06782. Retrieved from https:\/\/arxiv.org\/abs\/2308.06782"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/773153.773173"},{"key":"e_1_3_2_32_2","unstructured":"Yihong Dong Xue Jiang Zhi Jin and Ge Li. 2023. Self-collaboration code generation via ChatGPT. arXiv:2304.07590. Retrieved from https:\/\/arxiv.org\/abs\/2304.07590"},{"key":"e_1_3_2_33_2","unstructured":"Yilun Du Shuang Li Antonio Torralba Joshua B. Tenenbaum and Igor Mordatch. 2023. Improving factuality and reasoning in language models through multiagent debate. arXiv:2305.14325. Retrieved from https:\/\/arxiv.org\/abs\/2305.14325"},{"key":"e_1_3_2_34_2","unstructured":"Zhuoyun Du Chen Qian Wei Liu Zihao Xie Yifei Wang Yufan Dang Weize Chen and Cheng Yang. 2024. Multi-agent software development through cross-team collaboration. arXiv:2406.08979. Retrieved from https:\/\/arxiv.org\/abs\/2406.08979"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/11787006_1"},{"key":"e_1_3_2_36_2","unstructured":"Gang Fan Xiaoheng Xie Xunjin Zheng Yinan Liang and Peng Di. 2023. Static code analysis in the AI era: An in-depth exploration of the concept function and potential of intelligent code analysis agents. arXiv:2310.08837. Retrieved from https:\/\/arxiv.org\/abs\/2310.08837"},{"key":"e_1_3_2_37_2","first-page":"21","volume-title":"International Workshop on Agent Theories, Architectures, and Languages","author":"Franklin Stan","year":"1996","unstructured":"Stan Franklin and Art Graesser. 1996. Is it an agent, or just a program?: A taxonomy for autonomous agents. In International Workshop on Agent Theories, Architectures, and Languages. Springer, 21\u201335."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC60848.2023.00085"},{"key":"e_1_3_2_39_2","unstructured":"Yunfan Gao Yun Xiong Xinyu Gao Kangxiang Jia Jinliu Pan Yuxi Bi Yi Dai Jiawei Sun and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv:2312.10997. Retrieved from https:\/\/arxiv.org\/abs\/2312.10997"},{"key":"e_1_3_2_40_2","first-page":"152","volume-title":"Proceedings of the 1993 IEEE International Symposium on Requirements Engineering","author":"Goguen Joseph A.","year":"1993","unstructured":"Joseph A. Goguen and Charlotte Linde. 1993. Techniques for requirements elicitation. In Proceedings of the 1993 IEEE International Symposium on Requirements Engineering. IEEE, 152\u2013164."},{"issue":"110","key":"e_1_3_2_41_2","first-page":"1","article-title":"Secure multi-party computation","volume":"78","author":"Goldreich Oded","year":"1998","unstructured":"Oded Goldreich. 1998. Secure multi-party computation. Manuscript. Preliminary Version 78, 110 (1998), 1\u2013108.","journal-title":"Manuscript. Preliminary Version"},{"key":"e_1_3_2_42_2","unstructured":"Taicheng Guo Xiuying Chen Yaqi Wang Ruidi Chang Shichao Pei Nitesh V. Chawla Olaf Wiest and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. arXiv:2402.01680. Retrieved from https:\/\/arxiv.org\/abs\/2402.01680"},{"key":"e_1_3_2_43_2","unstructured":"Junda He Bowen Xu Zhou Yang DongGyun Han Chengran Yang Jiakun Liu Zhipeng Zhao and David Lo. 2024. PTM4Tag+: Tag recommendation of stack overflow posts with pre-trained models. arXiv:2408.02311. Retrieved from https:\/\/arxiv.org\/abs\/2408.02311"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527897"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3635711"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/940804"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1080\/07421222.2004.11045786"},{"key":"e_1_3_2_48_2","unstructured":"Samuel Holt Max Ruiz Luyten and Mihaela van der Schaar. 2023. L2mac: Large language model automatic computer for unbounded code generation. arXiv:2310.02003. Retrieved from https:\/\/arxiv.org\/abs\/2310.02003"},{"key":"e_1_3_2_49_2","unstructured":"Sirui Hong Xiawu Zheng Jonathan Chen Yuheng Cheng Jinlin Wang Ceyao Zhang Zili Wang Steven Ka Shing Yau Zijuan Lin Liyang Zhou et al. 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv:2308.00352. Retrieved from https:\/\/arxiv.org\/abs\/2308.00352"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.3386\/w31122"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3695988"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPS-ISA58951.2023.00044"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2015.33"},{"key":"e_1_3_2_54_2","unstructured":"Yue Hu Yuzhu Cai Yaxin Du Xinyu Zhu Xiangrui Liu Zijie Yu Yuchen Hou Shuo Tang and Siheng Chen. 2024. Self-evolving multi-agent collaboration networks for software development. arXiv:2410.16946. Retrieved from https:\/\/arxiv.org\/abs\/2410.16946"},{"key":"e_1_3_2_55_2","unstructured":"Dong Huang Qingwen Bu Jie M. Zhang Michael Luck and Heming Cui. 2023. AgentCoder: Multi-agent-based code generation with iterative testing and optimisation. arXiv:2312.13010. Retrieved from https:\/\/arxiv.org\/abs\/2312.13010"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3475716.3475769"},{"key":"e_1_3_2_57_2","unstructured":"Yoichi Ishibashi and Yoshimasa Nishimura. 2024. Self-organized agents: A LLM multi-agent framework toward ultra large-scale code generation and optimization. arXiv:2404.02183. Retrieved from https:\/\/arxiv.org\/abs\/2404.02183"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.269"},{"key":"e_1_3_2_59_2","unstructured":"Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida Wang Armando Solar-Lezama Koushik Sen and Ion Stoica. 2024. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv:2403.07974. Retrieved from https:\/\/arxiv.org\/abs\/2403.07974"},{"key":"e_1_3_2_60_2","unstructured":"Dongming Jin Zhi Jin Xiaohong Chen and Chunhui Wang. 2024. MARE: Multi-agents collaboration framework for requirements engineering. arXiv:2405.03256. Retrieved from https:\/\/arxiv.org\/abs\/2405.03256"},{"key":"e_1_3_2_61_2","unstructured":"Martin Josifoski Lars Klein Maxime Peyrard Nicolas Baldwin Yifei Li Saibo Geng Julian Paul Schnitzler Yuxing Yao Jiheng Wei Debjit Paul et al. 2023. Flows: Building blocks of reasoning and collaborating AI. arXiv:2308.01285. Retrieved from https:\/\/arxiv.org\/abs\/2308.01285"},{"key":"e_1_3_2_62_2","volume-title":"Proceedings of Net Object Days (NOD), AgeS Workshop","author":"Jouvin Denis","year":"2002","unstructured":"Denis Jouvin and Salima Hassas. 2002. Role delegation as multi-agent oriented dynamic composition. In Proceedings of Net Object Days (NOD), AgeS Workshop."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000083"},{"key":"e_1_3_2_64_2","volume-title":"Metrics and Models in Software Quality Engineering","author":"Kan Stephen H.","year":"2003","unstructured":"Stephen H. Kan. 2003. Metrics and Models in Software Quality Engineering. Addison-Wesley Professional."},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","unstructured":"Sungmin Kang Bei Chen Shin Yoo and Jian-Guang Lou. 2023. Explainable automated debugging via large language model-driven scientific debugging. arXiv:2304.02195. Retrieved from https:\/\/arxiv.org\/abs\/2304.02195","DOI":"10.1007\/s10664-024-10594-x"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2023.102274"},{"issue":"1","key":"e_1_3_2_67_2","first-page":"1","article-title":"Competency mapping: A gap analysis","volume":"1","author":"Kaur Jaideep","year":"2013","unstructured":"Jaideep Kaur and Vikas Kumar. 2013. Competency mapping: A gap analysis. International Journal of Education and Research 1, 1 (2013), 1\u20139.","journal-title":"International Journal of Education and Research"},{"key":"e_1_3_2_68_2","unstructured":"Omar Khattab Arnav Singhvi Paridhi Maheshwari Zhiyuan Zhang Keshav Santhanam Sri Vardhamanan Saiful Haq Ashutosh Sharma Thomas T. Joshi Hanna Moazam et al. 2023. Dspy: Compiling declarative language model calls into self-improving pipelines. arXiv:2310.03714. Retrieved from https:\/\/arxiv.org\/abs\/2310.03714"},{"key":"e_1_3_2_69_2","volume-title":"Agile and Iterative Development: A Manager\u2019s Guide","author":"Larman Craig","year":"2004","unstructured":"Craig Larman. 2004. Agile and Iterative Development: A Manager\u2019s Guide. Addison-Wesley Professional."},{"key":"e_1_3_2_70_2","unstructured":"Hung Le Yingbo Zhou Caiming Xiong Silvio Savarese and Doyen Sahoo. 2024. INDICT: Code generation with internal dialogues of critiques for both security and helpfulness. arXiv:2407.02518. Retrieved from https:\/\/arxiv.org\/abs\/2407.02518"},{"key":"e_1_3_2_71_2","unstructured":"Cheryl Lee Chunqiu Steven Xia Jen-tse Huang Zhouruixin Zhu Lingming Zhang and Michael R Lyu. 2024. A unified debugging approach via LLM-based multi-agent synergy. arXiv:2404.17153. Retrieved from https:\/\/arxiv.org\/abs\/2404.17153"},{"key":"e_1_3_2_72_2","volume-title":"Managing Software Requirements: A Unified Aapproach","author":"Leffingwell Dean","year":"2000","unstructured":"Dean Leffingwell and Don Widrig. 2000. Managing Software Requirements: A Unified Aapproach. Addison-Wesley Professional."},{"key":"e_1_3_2_73_2","unstructured":"Bin Lei Yuchen Li and Qiuwu Chen. 2024. AutoCoder: Enhancing code large language model with \\(\\backslash\\) textsc {AIEV-instruct}. arXiv:2405.14906. Retrieved from https:\/\/arxiv.org\/abs\/2405.14906"},{"key":"e_1_3_2_74_2","unstructured":"Bin Lei Yuchen Li Yiming Zeng Tao Ren Yi Luo Tianyu Shi Zitian Gao Zeyu Hu Weitai Kang and Qiuwu Chen. 2024. Infant agent: A tool-integrated logic-driven agent with cost-effective API usage. arXiv:2411.01114. Retrieved from https:\/\/arxiv.org\/abs\/2411.01114"},{"key":"e_1_3_2_75_2","unstructured":"Ludvig Lemner Linnea Wahlgren Gregory Gay Nasser Mohammadiha Jingxiong Liu and Joakim Wennerberg. 2024. Exploring the integration of large language models in industrial test maintenance processes. arXiv:2409.06416. Retrieved from https:\/\/arxiv.org\/abs\/2409.06416"},{"key":"e_1_3_2_76_2","first-page":"51991","article-title":"Camel: Communicative agents for \u201cmind\u201d exploration of large language model society","volume":"36","author":"Li Guohao","year":"2024","unstructured":"Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2024. Camel: Communicative agents for \u201cmind\u201d exploration of large language model society. Advances in Neural Information Processing Systems 36 (2024), 51991\u201352008.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_77_2","unstructured":"Jierui Li Hung Le Yinbo Zhou Caiming Xiong Silvio Savarese and Doyen Sahoo. 2024. CodeTree: Agent-guided tree search for code generation with large language models. arXiv:2411.04329. Retrieved from https:\/\/arxiv.org\/abs\/2411.04329"},{"key":"e_1_3_2_78_2","unstructured":"Junyou Li Qin Zhang Yangbin Yu Qiang Fu and Deheng Ye. 2024. More agents is all you need. arXiv:2402.05120. Retrieved from https:\/\/arxiv.org\/abs\/2402.05120"},{"key":"e_1_3_2_79_2","unstructured":"Yuan Li Yixuan Zhang and Lichao Sun. 2023. MetaAgents: Simulating interactions of human behaviors for LLM-based task-oriented coordination via collaborative generative agents. arXiv:2310.06500. Retrieved from https:\/\/arxiv.org\/abs\/2310.06500"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i9.28934"},{"key":"e_1_3_2_81_2","unstructured":"Tian Liang Zhiwei He Wenxiang Jiao Xing Wang Yan Wang Rui Wang Yujiu Yang Zhaopeng Tu and Shuming Shi. 2023. Encouraging divergent thinking in large language models through multi-agent debate. arXiv:2305.19118. Retrieved from https:\/\/arxiv.org\/abs\/2305.19118"},{"key":"e_1_3_2_82_2","doi-asserted-by":"crossref","unstructured":"Feng Lin Dong Jae Kim et al. 2024. SOEN-101: Code generation by emulating software process models using large language model agents. arXiv:2403.15852. Retrieved from https:\/\/arxiv.org\/abs\/2403.15852","DOI":"10.1109\/ICSE55347.2025.00140"},{"key":"e_1_3_2_83_2","unstructured":"Leilei Lin Yingming Zhou Wenlong Chen and Chen Qian. 2024. Think-on-process: Dynamic process generation for collaborative development of multi-agent system. arXiv:2409.06568. Retrieved from https:\/\/arxiv.org\/abs\/2409.06568"},{"key":"e_1_3_2_84_2","first-page":"1950","article-title":"Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning","volume":"35","author":"Liu Haokun","year":"2022","unstructured":"Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A. Raffel. 2022. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems 35 (2022), 1950\u20131965.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_85_2","first-page":"21558","article-title":"Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation","volume":"36","author":"Liu Jiawei","year":"2024","unstructured":"Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2024. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2024), 21558\u201321572.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_86_2","unstructured":"Xiangyan Liu Bo Lan Zhiyuan Hu Yang Liu Zhicheng Zhang Wenmeng Zhou Fei Wang and Michael Shieh. 2024. CodexGraph: Bridging large language models and code repositories via code graph databases. arXiv:2408.03910. Retrieved from https:\/\/arxiv.org\/abs\/2408.03910"},{"key":"e_1_3_2_87_2","unstructured":"Yizhou Liu Pengfei Gao Xinchen Wang Chao Peng and Zhao Zhang. 2024. MarsCode agent: AI-native automated bug fixing. arXiv:2409.00899. Retrieved from https:\/\/arxiv.org\/abs\/2409.00899"},{"key":"e_1_3_2_88_2","unstructured":"Zihan Liu Ruinan Zeng Dongxia Wang Gengyun Peng Jingyi Wang Qiang Liu Peiyu Liu and Wenhai Wang. 2024. Agents4PLC: Automating closed-loop PLC code generation and verification in industrial control systems using LLM-based agents. arXiv:2410.14209. Retrieved from https:\/\/arxiv.org\/abs\/2410.14209"},{"key":"e_1_3_2_89_2","unstructured":"Zijun Liu Yanzhe Zhang Peng Li Yang Liu and Diyi Yang. 2023. Dynamic LLM-agent network: An LLM-agent collaboration framework with agent team optimization. arXiv:2310.02170. Retrieved from https:\/\/arxiv.org\/abs\/2310.02170"},{"key":"e_1_3_2_90_2","volume-title":"Proceedings of the 1st Conference on Language Modeling","author":"Liu Zijun","year":"2024","unstructured":"Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2024. A dynamic LLM-powered agent network for task-oriented agent collaboration. In Proceedings of the 1st Conference on Language Modeling."},{"key":"e_1_3_2_91_2","unstructured":"David Lo. 2023. Trustworthy and synergistic artificial intelligence for software engineering: Vision and roadmaps. arXiv:2309.04142. Retrieved from https:\/\/arxiv.org\/abs\/2309.04142"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE59067.2024.00011"},{"key":"e_1_3_2_93_2","unstructured":"Yingwei Ma Qingping Yang Rongyu Cao Binhua Li Fei Huang and Yongbin Li. 2024. How to understand whole software repository? arXiv:2406.01422. Retrieved from https:\/\/arxiv.org\/abs\/2406.01422"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1162\/artl.1993.1.135"},{"key":"e_1_3_2_95_2","unstructured":"Zhenyu Mao Jialong Li Munan Li and Kenji Tei. 2024. Multi-role consensus through LLMs discussions for vulnerability detection. arXiv:2403.14274. Retrieved from https:\/\/arxiv.org\/abs\/2403.14274"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2022.100056"},{"key":"e_1_3_2_97_2","unstructured":"Noble Saji Mathews and Meiyappan Nagappan. 2024. Test-driven development for code generation. arXiv:2402.13521. Retrieved from https:\/\/arxiv.org\/abs\/2402.13521"},{"key":"e_1_3_2_98_2","unstructured":"Harrison Chase. 2022. LangChain. (2022). Retrieved from https:\/\/github.com\/langchain-ai\/langchain"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1093\/0195150430.001.0001"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1044\/cicsd_33_S_21"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-017-9360-z"},{"key":"e_1_3_2_102_2","unstructured":"Minh Huynh Nguyen Thang Phan Chau Phong X. Nguyen and Nghi D. Q. Bui. 2024. AgileCoder: Dynamic collaborative agents for software development based on agile methodology. arXiv:2406.11912. Retrieved from https:\/\/arxiv.org\/abs\/2406.11912"},{"key":"e_1_3_2_103_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Olausson Theo X.","year":"2023","unstructured":"Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. Is self-repair a silver bullet for code generation? In Proceedings of the 12th International Conference on Learning Representations."},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICGSE.2008.38"},{"key":"e_1_3_2_105_2","first-page":"68","article-title":"The California consumer privacy act: Towards a European-style privacy regime in the United States","volume":"23","author":"Pardau Stuart L.","year":"2018","unstructured":"Stuart L. Pardau. 2018. The California consumer privacy act: Towards a European-style privacy regime in the United States. Journal of Technology Law & Policy 23 (2018), 68.","journal-title":"Journal of Technology Law & Policy"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02152-7_29"},{"key":"e_1_3_2_107_2","unstructured":"Huy Nhat Phan Tien N. Nguyen Phong X. Nguyen and Nghi D. Q. Bui. 2024. Hyperagent: Generalist software engineering agents to solve coding tasks at scale. arXiv:2409.16299. Retrieved from https:\/\/arxiv.org\/abs\/2409.16299"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.305"},{"key":"e_1_3_2_109_2","unstructured":"Chen Qian Jiahao Li Yufan Dang Wei Liu YiFei Wang Zihao Xie Weize Chen Cheng Yang Yingli Zhang Zhiyuan Liu et al. 2024. Iterative experience refinement of software-developing agents. arXiv:2405.04219. Retrieved from https:\/\/arxiv.org\/abs\/2405.04219"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.810"},{"key":"e_1_3_2_111_2","unstructured":"Yihao Qin Shangwen Wang Yiling Lou Jinhao Dong Kaixin Wang Xiaoling Li and Xiaoguang Mao. 2024. AgentFL: Scaling LLM-based fault localization to project-level context. arXiv:2403.16362. Retrieved from https:\/\/arxiv.org\/abs\/2403.16362"},{"key":"e_1_3_2_112_2","unstructured":"Zeeshan Rasheed Malik Abdul Sami Muhammad Waseem Kai-Kristian Kemell Xiaofeng Wang Anh Nguyen Kari Syst\u00e4 and Pekka Abrahamsson. 2024. AI-powered code review with LLMs: Early results. arXiv:2404.18496. Retrieved from https:\/\/arxiv.org\/abs\/2404.18496"},{"key":"e_1_3_2_113_2","doi-asserted-by":"crossref","unstructured":"Zeeshan Rasheed Muhammad Waseem Mika Saari Kari Syst\u00e4 and Pekka Abrahamsson. 2024. Codepori: Large scale model for autonomous software development by using multi-agents. arXiv:2402.01411. Retrieved from https:\/\/arxiv.org\/abs\/2402.01411","DOI":"10.2139\/ssrn.4979510"},{"key":"e_1_3_2_114_2","unstructured":"Haifeng Ruan Yuntong Zhang and Abhik Roychoudhury. 2024. SpecRover: Code intent extraction via LLMs. arXiv:2408.02232. Retrieved from https:\/\/arxiv.org\/abs\/2408.02232"},{"key":"e_1_3_2_115_2","unstructured":"Malik Abdul Sami Muhammad Waseem Zeeshan Rasheed Mika Saari Kari Syst\u00e4 and Pekka Abrahamsson. 2024. Experimenting with multi-agent software development: Towards a unified platform. arXiv:2406.05381. Retrieved from https:\/\/arxiv.org\/abs\/2406.05381"},{"key":"e_1_3_2_116_2","unstructured":"Malik Abdul Sami Muhammad Waseem Zheying Zhang Zeeshan Rasheed Kari Syst\u00e4 and Pekka Abrahamsson. 2024. AI based multiagent approach for requirements elicitation and analysis. arXiv:2409.00038. Retrieved from https:\/\/arxiv.org\/abs\/2409.00038"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0065-2458(08)60206-5"},{"key":"e_1_3_2_118_2","first-page":"8634","article-title":"Reflexion: Language agents with verbal reinforcement learning","volume":"36","author":"Shinn Noah","year":"2024","unstructured":"Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems 36 (2024), 8634\u20138652.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(93)90034-9"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780367807542"},{"key":"e_1_3_2_121_2","unstructured":"Giriprasad Sridhara Ranjani H.G and Sourav Mazumdar. 2023. ChatGPT: A study on its utility for ubiquitous software engineering tasks. arXiv:2305.16837. Retrieved from https:\/\/arxiv.org\/abs\/2305.16837"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680347"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34957-8_9"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1145\/3613904.3642777"},{"key":"e_1_3_2_125_2","unstructured":"Daniel Tang Zhenghan Chen Kisub Kim Yewei Song Haoye Tian Saad Ezzini Yongfeng Huang and Jacques Klein Tegawende F. Bissyande. 2024. Collaborative agents for software engineering. arXiv:2402.02172. Retrieved from https:\/\/arxiv.org\/abs\/2402.02172"},{"key":"e_1_3_2_126_2","doi-asserted-by":"crossref","unstructured":"Wei Tao Yucheng Zhou Wenqiang Zhang and Yu Cheng. 2024. MAGIS: LLM-based multi-agent framework for GitHub issue resolution. arXiv:2403.17927. Retrieved from https:\/\/arxiv.org\/abs\/2403.17927","DOI":"10.52202\/079017-1647"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1002\/0471722324"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-800341-1.00001-2"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106589"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/337180.337184"},{"issue":"3152676","key":"e_1_3_2_131_2","first-page":"10","article-title":"The EU general data protection regulation (Gdpr)","volume":"10","author":"Voigt Paul","year":"2017","unstructured":"Paul Voigt and Axel Von dem Bussche. 2017. The EU general data protection regulation (Gdpr). A Practical Guide, 1st Ed., Springer International publishing 10, 3152676 (2017), 10\u20135555.","journal-title":"A Practical Guide"},{"key":"e_1_3_2_132_2","unstructured":"Guanzhi Wang Yuqi Xie Yunfan Jiang Ajay Mandlekar Chaowei Xiao Yuke Zhu Linxi Fan and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv:2305.16291. Retrieved from https:\/\/arxiv.org\/abs\/2305.16291"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.124"},{"key":"e_1_3_2_134_2","unstructured":"Luyuan Wang Yongyu Deng Yiwei Zha Guodong Mao Qinmin Wang Tianchen Min Wei Chen and Shoufa Chen. 2024. MobileAgentBench: An efficient and user-friendly benchmark for mobile LLM agents. arXiv:2406.08184. Retrieved from https:\/\/arxiv.org\/abs\/2406.08184"},{"key":"e_1_3_2_135_2","unstructured":"Lei Wang Chen Ma Xueyang Feng Zeyu Zhang Hao Yang Jingsen Zhang Zhiyuan Chen Jiakai Tang Xu Chen Yankai Lin et al. 2023. A survey on large language model based autonomous agents. arXiv:2308.11432. Retrieved from https:\/\/arxiv.org\/abs\/2308.11432"},{"key":"e_1_3_2_136_2","unstructured":"Qian Wang Tianyu Wang Qinbin Li Jingsheng Liang and Bingsheng He. 2024. MegaAgent: A practical framework for autonomous cooperation in large-scale LLM agent systems. arXiv:2408.09955. Retrieved from https:\/\/arxiv.org\/abs\/2408.09955"},{"key":"e_1_3_2_137_2","unstructured":"Siyuan Wang Zhuohan Long Zhihao Fan Zhongyu Wei and Xuanjing Huang. 2024. Benchmark self-evolving: A multi-agent framework for dynamic LLM evaluation. arXiv:2402.11443. Retrieved from https:\/\/arxiv.org\/abs\/2402.11443"},{"key":"e_1_3_2_138_2","unstructured":"Zihao Wang Shaofei Cai Guanzhou Chen Anji Liu Xiaojian Ma and Yitao Liang. 2023. Describe explain plan and select: Interactive planning with large language models enables open-world multi-task agents. arXiv:2302.01560. Retrieved from https:\/\/arxiv.org\/abs\/2302.01560"},{"key":"e_1_3_2_139_2","unstructured":"Zefan Wang Zichuan Liu Yingying Zhang Aoxiao Zhong Lunting Fan Lingfei Wu and Qingsong Wen. 2023. RCAgent: Cloud root cause analysis by autonomous agents with tool-augmented large language models. arXiv:2310.16340. Retrieved from https:\/\/arxiv.org\/abs\/2310.16340"},{"key":"e_1_3_2_140_2","unstructured":"Zhitao Wang Wei Wang Zirao Li Long Wang Can Yi Xinjie Xu Luyang Cao Hanjing Su Shouzhi Chen and Jun Zhou. 2024. XUAT-Copilot: Multi-agent collaborative system for automated user acceptance testing with large language model. arXiv:2401.02705. Retrieved from https:\/\/arxiv.org\/abs\/2401.02705"},{"key":"e_1_3_2_141_2","unstructured":"Zekun Moore Wang Zhongyuan Peng Haoran Que Jiaheng Liu Wangchunshu Zhou Yuhan Wu Hongcheng Guo Ruitong Gan Zehao Ni Man Zhang et al. 2023. Rolellm: Benchmarking eliciting and enhancing role-playing abilities of large language models. arXiv:2310.00746. Retrieved from https:\/\/arxiv.org\/abs\/2310.00746"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1145\/382192.383004"},{"key":"e_1_3_2_143_2","unstructured":"Ratnadira Widyasari David Lo and Lizi Liao. 2024. Beyond ChatGPT: Enhancing software quality assurance tasks with diverse LLMs and validation techniques. arXiv:2409.01001. Retrieved from https:\/\/arxiv.org\/abs\/2409.01001"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.5555\/1695886"},{"key":"e_1_3_2_146_2","unstructured":"Qingyun Wu Gagan Bansal Jieyu Zhang Yiran Wu Shaokun Zhang Erkang Zhu Beibin Li Li Jiang Xiaoyun Zhang and Chi Wang. 2023. Autogen: Enabling next-Gen LLM applications via multi-agent conversation framework. arXiv:2308.08155. Retrieved from https:\/\/arxiv.org\/abs\/2308.08155"},{"key":"e_1_3_2_147_2","unstructured":"Yiran Wu Feiran Jia Shaokun Zhang Qingyun Wu Hangyu Li Erkang Zhu Yue Wang Yin Tat Lee Richard Peng and Chi Wang. 2023. An empirical study on challenging math problem solving with gpt-4. arXiv:2306.01337. Retrieved from https:\/\/arxiv.org\/abs\/2306.01337"},{"key":"e_1_3_2_148_2","unstructured":"Zengqing Wu Shuyuan Zheng Qianying Liu Xu Han Brian Inhyuk Kwon Makoto Onizuka Shaojie Tang Run Peng and Chuan Xiao. 2024. Shall we talk: Exploring spontaneous collaborations of competing LLM agents. arXiv:2402.12327. Retrieved from https:\/\/arxiv.org\/abs\/2402.12327"},{"key":"e_1_3_2_149_2","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang Junzhe Wang Senjie Jin Enyu Zhou et al. 2023. The rise and potential of large language model based agents: A survey. arXiv:2309.07864. Retrieved from https:\/\/arxiv.org\/abs\/2309.07864"},{"key":"e_1_3_2_150_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639121"},{"key":"e_1_3_2_151_2","unstructured":"Chenyuan Yang Yinlin Deng Runyu Lu Jiayi Yao Jiawei Liu Reyhaneh Jabbarvand and Lingming Zhang. 2023. White-Box compiler fuzzing empowered by large language models. arXiv:2310.15991. Retrieved from https:\/\/arxiv.org\/abs\/2310.15991"},{"key":"e_1_3_2_152_2","unstructured":"Chengran Yang Jiakun Liu Bowen Xu Christoph Treude Yunbo Lyu Ming Li and David Lo. 2023. APIDocBooster: An extract-then-abstract framework leveraging large language models for augmenting API documentation. arXiv:2312.10934. Retrieved from https:\/\/arxiv.org\/abs\/2312.10934"},{"key":"e_1_3_2_153_2","unstructured":"Weiqing Yang Hanbin Wang Zhenghao Liu Xinze Li Yukun Yan Shuo Wang Yu Gu Minghe Yu Zhiyuan Liu and Ge Yu. 2024. Enhancing the code debugging ability of LLMs via communicative agent based data refinement. arXiv:2408.05006. Retrieved from https:\/\/arxiv.org\/abs\/2408.05006"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.5555\/2695622"},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST60714.2024.00020"},{"key":"e_1_3_2_156_2","unstructured":"Daoguang Zan Ailun Yu Wei Liu Dong Chen Bo Shen Wei Li Yafen Yao Yongshun Gong Xiaolin Chen Bei Guan et al. 2024. CodeS: Natural language to code repository via multi-layer sketch. arXiv:2403.16443. Retrieved from https:\/\/arxiv.org\/abs\/2403.16443"},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534390"},{"key":"e_1_3_2_158_2","unstructured":"Guibin Zhang Yanwei Yue Xiangguo Sun Guancheng Wan Miao Yu Junfeng Fang Kun Wang and Dawei Cheng. 2024. G-designer: Architecting multi-agent communication topologies via graph neural networks. arXiv:2410.11782. Retrieved from https:\/\/arxiv.org\/abs\/2410.11782"},{"key":"e_1_3_2_159_2","doi-asserted-by":"crossref","unstructured":"Huan Zhang Wei Cheng Yuhan Wu and Wei Hu. 2024. A pair programming framework for code generation via multi-plan exploration and feedback-driven refinement. arXiv:2409.05001. Retrieved from https:\/\/arxiv.org\/abs\/2409.05001","DOI":"10.1145\/3691620.3695506"},{"key":"e_1_3_2_160_2","unstructured":"Kexun Zhang Weiran Yao Zuxin Liu Yihao Feng Zhiwei Liu Rithesh Murthy Tian Lan Lei Li Renze Lou Jiacheng Xu et al. 2024. Diversity empowers intelligence: Integrating expertise of software engineering agents. arXiv:2408.07060. Retrieved from https:\/\/arxiv.org\/abs\/2408.07060"},{"key":"e_1_3_2_161_2","unstructured":"Lyuye Zhang Kaixuan Li Kairan Sun Daoyuan Wu Ye Liu Haoye Tian and Yang Liu. 2024. Acfix: Guiding LLMs with mined common rbac practices for context-aware repair of access control vulnerabilities in smart contracts. arXiv:2403.06838. Retrieved from https:\/\/arxiv.org\/abs\/2403.06838"},{"key":"e_1_3_2_162_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11390-018-1864-x"},{"key":"e_1_3_2_163_2","unstructured":"Simiao Zhang Jiaping Wang Guoliang Dong Jun Sun Yueling Zhang and Geguang Pu. 2024. Experimenting a new programming practice with LLMs. arXiv:2401.01062. Retrieved from https:\/\/arxiv.org\/abs\/2401.01062"},{"key":"e_1_3_2_164_2","unstructured":"Sai Zhang Zhenchang Xing Ronghui Guo Fangzhou Xu Lei Chen Zhaoyuan Zhang Xiaowang Zhang Zhiyong Feng and Zhiqiang Zhuang. 2024. Empowering agile-based generative software development through human-AI teamwork. arXiv:2407.15568. Retrieved from https:\/\/arxiv.org\/abs\/2407.15568"},{"key":"e_1_3_2_165_2","unstructured":"Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu Tingchen Fu Xinting Huang Enbo Zhao Yu Zhang Yulong Chen et al. 2023. Siren\u2019s song in the AI ocean: A survey on hallucination in large language models. arXiv:2309.01219. Retrieved from https:\/\/arxiv.org\/abs\/2309.01219"},{"key":"e_1_3_2_166_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680384"},{"key":"e_1_3_2_167_2","unstructured":"Zhonghan Zhao Kewei Chen Dongxu Guo Wenhao Chai Tian Ye Yanting Zhang and Gaoang Wang. 2024. Hierarchical auto-organizing system for open-ended multi-agent navigation. arXiv:2403.08282. Retrieved from https:\/\/arxiv.org\/abs\/2403.08282"},{"key":"e_1_3_2_168_2","doi-asserted-by":"publisher","DOI":"10.1504\/IJWGS.2018.095647"},{"key":"e_1_3_2_169_2","unstructured":"Xin Zhou Sicong Cao Xiaobing Sun and David Lo. 2024. Large language model for vulnerability detection and repair: Literature review and the road ahead. arXiv:2404.02525. Retrieved from https:\/\/arxiv.org\/abs\/2404.02525"},{"key":"e_1_3_2_170_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME58846.2023.00028"},{"key":"e_1_3_2_171_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.emnlp-main.890"},{"key":"e_1_3_2_172_2","unstructured":"Terry Yue Zhuo Minh Chien Vu Jenny Chim Han Hu Wenhao Yu Ratnadira Widyasari Imam Nur Bani Yusuf Haolan Zhan Junda He Indraneil Paul et al. 2024. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. arXiv:2406.15877. Retrieved from https:\/\/arxiv.org\/abs\/2406.15877"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3712003","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3712003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,20]],"date-time":"2025-11-20T16:03:31Z","timestamp":1763654611000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3712003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,24]]},"references-count":171,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3712003"],"URL":"https:\/\/doi.org\/10.1145\/3712003","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,24]]},"assertion":[{"value":"2024-04-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}