{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T15:27:41Z","timestamp":1784302061459,"version":"3.55.0"},"reference-count":118,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2025,5,26]],"date-time":"2025-05-26T00:00:00Z","timestamp":1748217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Marsden Fund Council"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>A paradigm shift is underway in Software Engineering, with AI systems such as LLMs playing an increasingly important role in boosting software development productivity. This trend is anticipated to persist. In the next years, we expect a growing symbiotic partnership between human software developers and AI. The Software Engineering research community cannot afford to overlook this trend; we must address the key research challenges posed by the integration of AI into the software development process. In this article, we present our vision of the future of software development in an AI-driven world and explore the key challenges that our research community should address to realize this vision.<\/jats:p>","DOI":"10.1145\/3715003","type":"journal-article","created":{"date-parts":[[2025,1,24]],"date-time":"2025-01-24T15:25:13Z","timestamp":1737732313000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":48,"title":["The Future of AI-Driven Software Engineering"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5885-9297","authenticated-orcid":false,"given":"Valerio","family":"Terragni","sequence":"first","affiliation":[{"name":"The University of Auckland, Auckland, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5180-9618","authenticated-orcid":false,"given":"Annie","family":"Vella","sequence":"additional","affiliation":[{"name":"The University of Auckland, Auckland, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9654-5678","authenticated-orcid":false,"given":"Partha","family":"Roop","sequence":"additional","affiliation":[{"name":"The University of Auckland, Auckland, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4092-9706","authenticated-orcid":false,"given":"Kelly","family":"Blincoe","sequence":"additional","affiliation":[{"name":"The University of Auckland, Auckland, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,26]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"277","volume-title":"Proceedings of the 2020 20th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA)","author":"Abdelnabi Esra A.","year":"2020","unstructured":"Esra A. Abdelnabi, Abdelsalam M. Maatuk, Tawfig M. Abdelaziz, and Salwa M. Elakeili. 2020. Generating UML class diagram using NLP techniques and heuristic rules. In Proceedings of the 2020 20th International Conference on Sciences and Techniques of Automatic Control and Computer Engineering (STA), 277\u2013282. DOI: 10.1109\/STA50679.2020.9329301"},{"key":"e_1_3_2_3_2","first-page":"140","volume-title":"Proceedings of the 43rd International Conference on Software Engineering","author":"Ahlgren John","year":"2021","unstructured":"John Ahlgren, Maria Eugenia Berezin, Kinga Bojarczuk, Elena Dulskyte, Inna Dvortsova, Johann George, Natalija Gucevska, Mark Harman, Maria Lomeli, Erik Meijer, et al. 2021. Testing web enabled simulation at scale using metamorphic testing. In Proceedings of the 43rd International Conference on Software Engineering, 140\u2013149. DOI: 10.1109\/ICSE-SEIP52600.2021.00023"},{"key":"e_1_3_2_4_2","first-page":"185","volume-title":"Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE \u201924)","author":"Alshahwan Nadia","year":"2024","unstructured":"Nadia Alshahwan, Jubin Chheda, Anastasia Finogenova, Beliz Gokkaya, Mark Harman, Inna Harper, Alexandru Marginean, Shubho Sengupta, and Eddy Wang. 2024. Automated unit test improvement using large language models at meta. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (FSE \u201924). ACM, 185\u2013196. DOI: 10.1145\/3663529.3663839"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1007\/978-3-031-55642-5_6","volume-title":"Generative AI for Effective Software Development","author":"Arora Chetan","year":"2024","unstructured":"Chetan Arora, John Grundy, and Mohamed Abdelrazek. 2024. Advancing requirements engineering through generative AI: Assessing the role of LLMs. In Generative AI for Effective Software Development. A. Nguyen-Duc, P. Abrahamsson, and F. Khomh (Eds.), Springer, 129\u2013148."},{"key":"e_1_3_2_6_2","first-page":"1264","volume-title":"Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE \u201921)","author":"Ayerdi Jon","year":"2021","unstructured":"Jon Ayerdi, Valerio Terragni, Aitor Arrieta, Paolo Tonella, Goiuria Sagardui, and Maite Arratibel. 2021. Generating metamorphic relations for cyber-physical systems with genetic programming: An industrial case study. In Proceedings of the 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE \u201921). ACM, 1264\u20131274. DOI: 10.1145\/3468264.3473920"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3407840"},{"issue":"5","key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"488","DOI":"10.1016\/0010-4809(70)90010-8","article-title":"High-level programming languages","volume":"3","author":"Barnett G. Octo","year":"1970","unstructured":"G. Octo Barnett and Robert A. Greenes. 1970. High-level programming languages. Computers and Biomedical Research 3, 5 (1970), 488\u2013494.","journal-title":"Computers and Biomedical Research"},{"key":"e_1_3_2_9_2","first-page":"58","article-title":"Copyright conundrums in generative AI: Github Copilot\u2019s not-so-fair use of open-source licensed code","volume":"7","author":"Basanagoudar Vivek","year":"2023","unstructured":"Vivek Basanagoudar and Abhijay Srekanth. 2023. Copyright conundrums in generative AI: Github Copilot\u2019s not-so-fair use of open-source licensed code. Journal of Intellectual Property Studies 7 (2023), 58\u201368.","journal-title":"Journal of Intellectual Property Studies"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1145\/3643795.3648396","volume-title":"Proceedings of the 1st International Workshop on Large Language Models for Code","author":"Bhatia Shreya","year":"2024","unstructured":"Shreya Bhatia, Tarushi Gandhi, Dhruv Kumar, and Pankaj Jalote. 2024. Unit test generation using generative AI: A comparative performance analysis of autogeneration tools. In Proceedings of the 1st International Workshop on Large Language Models for Code, 54\u201361. DOI: 10.1145\/3643795.3648396"},{"key":"e_1_3_2_11_2","first-page":"2336","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201924)","author":"Bi Zhangqian","year":"2024","unstructured":"Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. 2024. Iterative refinement of project-level code context for precise code generation with compiler feedback. In Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201924). ACL, 2336\u20132353. DOI: 10.18653\/V1\/2024.FINDINGS-ACL.138"},{"key":"e_1_3_2_12_2","unstructured":"Soneya Binta Hossain and Matthew Dwyer. 2024. TOGLL: Correct and strong test oracle generation with LLMs. arXiv:2405.03786. Retrieved from https:\/\/arxiv.org\/abs\/2405.03786"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.JSS.2021.111041"},{"issue":"3","key":"e_1_3_2_14_2","first-page":"3","article-title":"The copyleft movement: Creative commons licensing","volume":"26","author":"Broussard Sharee L.","year":"2007","unstructured":"Sharee L. Broussard. 2007. The copyleft movement: Creative commons licensing. Communication Research Trends 26, 3 (2007), 3\u201318.","journal-title":"Communication Research Trends"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/1015864.1015889"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/IWC\/IWV046"},{"key":"e_1_3_2_17_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_2_18_2","volume-title":"Metamorphic Testing: A New Approach for Generating Next Test Cases","author":"Chen T. Y.","year":"1998","unstructured":"T. Y. Chen, S. C. Cheung, and S. M. Yiu. 1998. Metamorphic Testing: A New Approach for Generating Next Test Cases. Technical Report. Technical Report HKUST-CS98-01, Department of Computer Science, The Hong Kong University of Science and Technology."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3143561"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"1487","DOI":"10.1145\/3468264.3473136","volume-title":"Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Chen Tsong Yueh","year":"2021","unstructured":"Tsong Yueh Chen and T. H. Tse. 2021. New visions on metamorphic testing after a quarter of a century of inception. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 1487\u20131490."},{"key":"e_1_3_2_21_2","unstructured":"Leuson Da Silva Jordan Samhi and Foutse Khomh. 2024. ChatGPT vs LLaMA: Impact reliability and challenges in stack overflow discussions. arXiv:2402.08801. Retrieved from https:\/\/arxiv.org\/abs\/2402.08801"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.INFSOF.2024.107468"},{"key":"e_1_3_2_23_2","first-page":"142","volume-title":"Proceedings of the 2019 IEEE 27th International Requirements Engineering Conference (RE)","author":"Dalpiaz Fabiano","year":"2019","unstructured":"Fabiano Dalpiaz, Davide Dell\u2019Anna, Fatma Basak Aydemir, and Sercan \u00c7evikol. 2019. Requirements classification with interpretable machine learning and dependency parsing. In Proceedings of the 2019 IEEE 27th International Requirements Engineering Conference (RE), 142\u2013152. DOI: 10.1109\/RE.2019.00025"},{"key":"e_1_3_2_24_2","first-page":"865","volume-title":"Proceedings of the 2024 IEEE\/ACM 46th International Conference on Software Engineering (ICSE)","author":"Du Xueying","year":"2024","unstructured":"Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2024. Evaluating large language models in class-level code generation. In Proceedings of the 2024 IEEE\/ACM 46th International Conference on Software Engineering (ICSE). IEEE, 865\u2013865. DOI: 10.1145\/3597503.3639219"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3561048"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2023.3265877"},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems","author":"Medeiros Eler Marcelo","year":"2019","unstructured":"Marcelo Medeiros Eler, Leandro Orlandin, and Alberto Dumont Alves Oliveira. 2019. Do Android app users care about accessibility? An analysis of user reviews on the Google play store. In Proceedings of the 18th Brazilian Symposium on Human Factors in Computing Systems, 1\u201311. DOI: 10.1145\/3357155.3358477"},{"key":"e_1_3_2_28_2","first-page":"31","volume-title":"Proceedings of the 2023 IEEE\/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE)","author":"Fan Angela","year":"2023","unstructured":"Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. Zhang. 2023. Large language models for software engineering: Survey and open problems. In Proceedings of the 2023 IEEE\/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE). IEEE, 31\u201353. DOI: 10.1109\/ICSE-FOSE59343.2023.00008"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1145\/2025113.2025179","volume-title":"Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering","author":"Fraser Gordon","year":"2011","unstructured":"Gordon Fraser and Andrea Arcuri. 2011. EvoSuite: Automatic test suite generation for object-oriented software. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering, 416\u2013419."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-34144-1"},{"key":"e_1_3_2_31_2","first-page":"1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Geng Mingyang","year":"2024","unstructured":"Mingyang Geng, Shangwen Wang, Dezun Dong, Haotian Wang, Ge Li, Zhi Jin, Xiaoguang Mao, and Xiangke Liao. 2024. Large language models are few-shot summarizers: Multi-intent comment generation via in-context learning. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, 1\u201313. DOI: 10.1145\/3597503.3608134"},{"key":"e_1_3_2_32_2","unstructured":"Stefan Goetz and Andreas Schaad. 2024. \u201cYou still have to study\u201d\u2014On the security of LLM generated code. arXiv:2408.07106. Retrieved from https:\/\/arxiv.org\/abs\/2408.07106"},{"issue":"1","key":"e_1_3_2_33_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2500000010","article-title":"Program synthesis","volume":"4","author":"Gulwani Sumit","year":"2017","unstructured":"Sumit Gulwani, Oleksandr Polozov, Rishabh Singh. 2017. Program synthesis. Foundations and Trends\u00ae in Programming Languages 4, 1\u20132 (2017), 1\u2013119.","journal-title":"Foundations and Trends\u00ae in Programming Languages"},{"key":"e_1_3_2_34_2","first-page":"1073","volume-title":"Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201924)","author":"Guo Lianghong","year":"2024","unstructured":"Lianghong Guo, Yanlin Wang, Ensheng Shi, Wanjun Zhong, Hongyu Zhang, Jiachi Chen, Ruikai Zhang, Yuchi Ma, and Zibin Zheng. 2024. When to stop? Towards efficient code generation in LLMs with excess token prevention. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201924), 1073\u20131085. DOI: 10.1145\/3650212.3680343"},{"key":"e_1_3_2_35_2","unstructured":"Junda He Christoph Treude and David Lo. 2024. LLM-based multi-agent systems for software engineering: Vision and the road ahead. arXiv:2404.04834. Retrieved from https:\/\/arxiv.org\/abs\/2404.04834"},{"key":"e_1_3_2_36_2","unstructured":"Wenpin Hou and Zhicheng Ji. 2024. A systematic evaluation of large language models for generating programming code. arXiv:2403.00894. Retrieved from https:\/\/arxiv.org\/abs\/2403.00894"},{"issue":"8","key":"e_1_3_2_37_2","first-page":"1","article-title":"Large language models for software engineering: A systematic literature review","volume":"33","author":"Hou Xinyi","year":"2023","unstructured":"Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. 2023. Large language models for software engineering: A systematic literature review. ACM Transactions on Software Engineering and Methodology 33, 8 (2023), 1\u201379.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_38_2","unstructured":"Dong Huang Qingwen Bu Jie Zhang Xiaofei Xie Junjie Chen and Heming Cui. 2023. Bias assessment and mitigation in LLM-based code generation. arXiv:2309.14345. Retrieved from https:\/\/arxiv.org\/abs\/2309.14345"},{"key":"e_1_3_2_39_2","unstructured":"Yuan Huang Yinan Chen Xiangping Chen Junqi Chen Rui Peng Zhicao Tang Jinbo Huang Furen Xu and Zibin Zheng. 2024. Generative software engineering. arXiv:2403.02583. Retrieved from https:\/\/arxiv.org\/abs\/2403.02583"},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1145\/2931037.2931062","volume-title":"Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA \u201916)","author":"Jahangirova Gunel","year":"2016","unstructured":"Gunel Jahangirova, David Clark, Mark Harman, and Paolo Tonella. 2016. Test oracle assessment and improvement. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA \u201916). ACM, 247\u2013258. DOI: 10.1145\/2931037.2931062"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2934409"},{"key":"e_1_3_2_42_2","first-page":"61","volume-title":"Proceedings of the 2023 IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT)","author":"Jahangirova Gunel","year":"2023","unstructured":"Gunel Jahangirova and Valerio Terragni. 2023. SBFT tool competition 2023\u2014Java test case generation track. In Proceedings of the 2023 IEEE\/ACM International Workshop on Search-Based and Fuzz Testing (SBFT), 61\u201364. DOI: 10.1109\/sbft59156.2023.00025"},{"issue":"2","key":"e_1_3_2_43_2","first-page":"1","article-title":"An extended study of syntactic breaking changes in the wild","volume":"30","author":"Jayasuriya Dhanushka","year":"2024","unstructured":"Dhanushka Jayasuriya, Samuel Ou, Saakshi Hegde, Valerio Terragni, Jens Dietrich, and Kelly Blincoe. 2024. An extended study of syntactic breaking changes in the wild. Empirical Software Engineering 30, 2, (2024), 1\u201344.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_44_2","first-page":"1238","volume-title":"Proceedings of the 1st ACM on Software Engineering (PACMSE)","author":"Jayasuriya Dhanushka","year":"2024","unstructured":"Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, and Kelly Blincoe. 2024. Understanding the impact of APIs behavioral breaking changes on client applications. In Proceedings of the 1st ACM on Software Engineering (PACMSE), 1238\u20131261."},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1145\/3597926.3598147","volume-title":"Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Jayasuriya Dhanushka","year":"2023","unstructured":"Dhanushka Jayasuriya, Valerio Terragni, Jens Dietrich, Samuel Ou, and Kelly Blincoe. 2023. Understanding breaking changes in the wild. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis, 1433\u20131444."},{"key":"e_1_3_2_46_2","first-page":"1646","volume-title":"Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Jin Matthew","year":"2023","unstructured":"Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. 2023. Inferfix: End-to-end program repair with LLMs. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 1646\u20131656."},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Gurusha Juneja Subhabrata Dutta Soumen Chakrabarti Sunny Manchanda and Tanmoy Chakraborty. 2023. Small language models fine-tuned to coordinate larger language models improve complex reasoning. arXiv:2310.18338. Retrieved from https:\/\/arxiv.org\/abs\/2310.18338","DOI":"10.18653\/v1\/2023.emnlp-main.225"},{"key":"e_1_3_2_48_2","first-page":"213","volume-title":"Proceedings of the 2014 IEEE 22nd International Requirements Engineering Conference (RE)","author":"Knauss Eric","year":"2014","unstructured":"Eric Knauss, Daniela Damian, Alessia Knauss, and Arber Borici. 2014. Openness and requirements: Opportunities and tradeoffs in software ecosystems. In Proceedings of the 2014 IEEE 22nd International Requirements Engineering Conference (RE). IEEE, 213\u2013222."},{"key":"e_1_3_2_49_2","unstructured":"Michael Konstantinou Renzo Degiovanni and Mike Papadakis. 2024. Do LLMs generate test oracles that capture the actual or the expected program behaviour? arXiv:2410.21136. Retrieved from https:\/\/arxiv.org\/abs\/2410.21136"},{"issue":"8","key":"e_1_3_2_50_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3470133","article-title":"A systematic review of API evolution literature","volume":"54","author":"Lamothe Maxime","year":"2021","unstructured":"Maxime Lamothe, Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc, and Weiyi Shang. 2021. A systematic review of API evolution literature. ACM Computing Surveys 54, 8 (2021), 1\u201336.","journal-title":"ACM Computing Surveys"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318162"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1007\/BFb0017737","volume-title":"European Workshop On Software Process Technology","author":"Lehman Manny M.","year":"1996","unstructured":"Manny M. Lehman. 1996. Laws of software evolution revisited. In European Workshop On Software Process Technology. C. Montangero (Ed.), Springer, 108\u2013124."},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1109\/ICSE48619.2023.00085","volume-title":"Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Lemieux Caroline","year":"2023","unstructured":"Caroline Lemieux, Jeevana Priya Inala, Shuvendu K. Lahiri, and Siddhartha Sen. 2023. Codamosa: Escaping coverage plateaus in test generation with pre-trained large language models. In Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 919\u2013931."},{"key":"e_1_3_2_54_2","first-page":"103942","article-title":"Evaluating large language models for software testing","volume":"93","author":"Li Yihao","year":"2024","unstructured":"Yihao Li, Pan Liu, Haiyang Wang, Jie Chu, and W. Eric Wong. 2024. Evaluating large language models for software testing. Computer Standards and Interfaces 93 (2024), 103942.","journal-title":"Computer Standards and Interfaces"},{"key":"e_1_3_2_55_2","first-page":"1","volume-title":"Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE \u201924)","author":"Liang Jenny T.","year":"2024","unstructured":"Jenny T. Liang, Chenyang Yang, and Brad A. Myers. 2024. A large-scale survey on the usability of AI programming assistants: Successes and challenges. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 1\u201313. DOI: 10.1145\/3597503.3608128"},{"key":"e_1_3_2_56_2","unstructured":"Junwei Liu Kaixin Wang Yixuan Chen Xin Peng Zhenpeng Chen Lingming Zhang and Yiling Lou. 2024. Large language model-based agents for software engineering: A survey. arXiv:2409.02977. Retrieved from https:\/\/arxiv.org\/abs\/2409.02977"},{"key":"e_1_3_2_57_2","first-page":"21558","article-title":"Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation","volume":"36","author":"Liu Jiawei","year":"2024","unstructured":"Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2024. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2024), 21558\u201321572.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"110817","DOI":"10.1016\/j.jss.2020.110817","article-title":"A critical review on the evaluation of automated program repair systems","volume":"171","author":"Liu Kui","year":"2021","unstructured":"Kui Liu, Li Li, Anil Koyuncu, Dongsun Kim, Zhe Liu, Jacques Klein, and Tegawend\u00e9 F. Bissyand\u00e9. 2021. A critical review on the evaluation of automated program repair systems. Journal of Systems and Software 171 (2021), 110817.","journal-title":"Journal of Systems and Software"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","unstructured":"Andrea Lops Fedelucio Narducci Azzurra Ragone Michelantonio Trizio and Claudio Bartolini. 2024. A system for automated unit test generation using large language models and assessment of generated test suites. arXiv:2408.07846. Retrieved from https:\/\/arxiv.org\/abs\/2408.07846","DOI":"10.1109\/ICSTW64639.2025.10962454"},{"key":"e_1_3_2_60_2","first-page":"389","volume-title":"Proceedings of the 2024 IEEE 32nd International Requirements Engineering Conference (RE)","author":"Lubos Sebastian","year":"2024","unstructured":"Sebastian Lubos, Alexander Felfernig, Thi Ngoc Trang Tran, Damian Garber, Merfat El Mansi, Seda Polat Erdeniz, and Viet-Man Le. 2024. Leveraging LLMs for the quality assurance of software requirements. In Proceedings of the 2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 389\u2013397."},{"key":"e_1_3_2_61_2","first-page":"168","volume-title":"Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering","author":"Lukasczyk Stephan","year":"2022","unstructured":"Stephan Lukasczyk and Gordon Fraser. 2022. Pynguin: Automated unit test generation for python. In Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering, 168\u2013172."},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","unstructured":"Qinyu Luo Yining Ye Shihao Liang Zhong Zhang Yujia Qin Yaxi Lu Yesai Wu Xin Cong Yankai Lin Yingli Zhang et al. 2024. RepoAgent: An LLM-powered open-source framework for repository-level code documentation generation. arXiv:2402.16667. Retrieved from https:\/\/arxiv.org\/abs\/2402.16667","DOI":"10.18653\/v1\/2024.emnlp-demo.46"},{"key":"e_1_3_2_63_2","unstructured":"Quang-Hung Luu Huai Liu and Tsong Yueh Chen. 2023. Can ChatGPT advance software testing intelligence? An experience report on metamorphic testing. arXiv:2310.19204. Retrieved from https:\/\/arxiv.org\/abs\/2310.19204"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"106798","DOI":"10.1016\/j.infsof.2021.106798","article-title":"Prioritizing user concerns in app reviews\u2014A study of requests for new features, enhancements and bug fixes","volume":"144","author":"Malgaonkar Saurabh","year":"2022","unstructured":"Saurabh Malgaonkar, Sherlock A Licorish, and Bastin Tony Roy Savarimuthu. 2022. Prioritizing user concerns in app reviews\u2014A study of requests for new features, enhancements and bug fixes. Information and Software Technology 144 (2022), 106798.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_65_2","unstructured":"Garima Malik Mucahit Cevik Devang Parikh and Ayse Basar. 2024. Supervised semantic similarity-based conflict detection algorithm: S3CDA. arXiv:2206.13690. Retrieved from https:\/\/arxiv.org\/abs\/2206.13690"},{"key":"e_1_3_2_66_2","author":"Mao Ke","year":"2015","unstructured":"Ke Mao, Licia Capra, Mark Harman, and Yue Jia. 2015. A Survey of the Use of Crowdsourcing in Software Engineering. Research Note RN\/15\/01.","journal-title":"A Survey of the Use of Crowdsourcing in Software Engineering"},{"issue":"6","key":"e_1_3_2_67_2","doi-asserted-by":"crossref","first-page":"180","DOI":"10.3390\/fi16060180","article-title":"Using ChatGPT in software requirements engineering: A comprehensive review","volume":"16","author":"Marques Nuno","year":"2024","unstructured":"Nuno Marques, Rodrigo Rocha Silva, and Jorge Bernardino. 2024. Using ChatGPT in software requirements engineering: A comprehensive review. Future Internet 16, 6 (2024), 180.","journal-title":"Future Internet"},{"key":"e_1_3_2_68_2","first-page":"1223","volume-title":"Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE)","author":"Molina Facundo","year":"2021","unstructured":"Facundo Molina, Pablo Ponzio, Nazareno Aguirre, and Marcelo Frias. 2021. EvoSpex: An evolutionary algorithm for learning postconditions. In Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), 1223\u20131235. DOI: 10.1109\/ICSE43902.2021.00112"},{"issue":"9","key":"e_1_3_2_69_2","doi-asserted-by":"crossref","first-page":"2240","DOI":"10.1109\/TSE.2024.3428324","article-title":"Towards efficient fine-tuning of language models with organizational data for automated software review","volume":"50","author":"Nashaat Mona","year":"2024","unstructured":"Mona Nashaat and James Miller. 2024. Towards efficient fine-tuning of language models with organizational data for automated software review. IEEE Transactions on Software Engineering 50, 9 (2024), 2240\u20132253.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_70_2","first-page":"1","volume-title":"Proceedings of the 19th International Conference on Mining Software Repositories","author":"Nguyen Nhan","year":"2022","unstructured":"Nhan Nguyen and Sarah Nadi. 2022. An empirical evaluation of GitHub copilot\u2019s code suggestions. In Proceedings of the 19th International Conference on Mining Software Repositories, 1\u20135."},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"2464","DOI":"10.1145\/3691620.3695330","volume-title":"Proceedings of the 39th IEEE\/ACM International Conference on Automated Software Engineering","author":"Ouedraogo Wendkuuni C.","year":"2024","unstructured":"Wendkuuni C. Ouedraogo, Kader Kabore, Haoye Tian, Yewei Song, Anil Koyuncu, Jacques Klein, David Lo, and Tegawende F. Bissyande. 2024. LLMs and prompting for unit test generation: A large-scale evaluation. In Proceedings of the 39th IEEE\/ACM International Conference on Automated Software Engineering, 2464\u20132465."},{"key":"e_1_3_2_72_2","first-page":"75","volume-title":"Proceedings of the 29th International Conference on Software Engineering (ICSE\u201907)","author":"Pacheco Carlos","year":"2007","unstructured":"Carlos Pacheco, Shuvendu K. Lahiri, Michael D. Ernst, and Thomas Ball. 2007. Feedback-directed random test generation. In Proceedings of the 29th International Conference on Software Engineering (ICSE\u201907). IEEE, 75\u201384."},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1109\/SP46214.2022.9833571","volume-title":"Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP)","author":"Pearce Hammond","year":"2022","unstructured":"Hammond Pearce, Baleegh Ahmad, Benjamin Tan, Brendan Dolan-Gavitt, and Ramesh Karri. 2022. Asleep at the keyboard? Assessing the security of GitHub Copilot\u2019s code contributions. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP), 754\u2013768."},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"1361","DOI":"10.1145\/2145204.2145407","volume-title":"Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work","author":"Philip Kavita","year":"2012","unstructured":"Kavita Philip, Medha Umarji, Megha Agarwala, Susan Elliott Sim, Rosalva Gallardo-Valencia, Cristina V. Lopes, and Sukanya Ratanotayanon. 2012. Software reuse through methodical component reuse and a methodical snippet remixing. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work, 1361\u20131370."},{"key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"107523","DOI":"10.1016\/j.infsof.2024.107523","article-title":"Fine-tuning and prompt engineering for large language models-based code review automation","volume":"175","author":"Pornprasit Chanathip","year":"2024","unstructured":"Chanathip Pornprasit and Chakkrit Tantithamthavorn. 2024. Fine-tuning and prompt engineering for large language models-based code review automation. Information and Software Technology 175 (2024), 107523.","journal-title":"Information and Software Technology"},{"key":"e_1_3_2_76_2","first-page":"1","volume-title":"Proceedings of the 17th Innovations in Software Engineering Conference","author":"Rajbhoj Asha","year":"2024","unstructured":"Asha Rajbhoj, Akanksha Somase, Piyush Kulkarni, and Vinay Kulkarni. 2024. Accelerating software development using generative AI: ChatGPT case study. In Proceedings of the 17th Innovations in Software Engineering Conference, 1\u201311."},{"key":"e_1_3_2_77_2","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1109\/ICSME55016.2022.00054","volume-title":"Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Ribeiro Eric","year":"2022","unstructured":"Eric Ribeiro, Ronan Nascimento, Igor Steinmacher, Laerte Xavier, Marco Gerosa, Hugo de Paula, and Mairieli Wessel. 2022. Together or apart? Investigating a mediator bot to aggregate bot\u2019s comments on pull requests. In Proceedings of the 2022 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 434\u2013438."},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1145\/2786805.2786855","volume-title":"Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering","author":"Sadowski Caitlin","year":"2015","unstructured":"Caitlin Sadowski, Kathryn T Stolee, and Sebastian Elbaum. 2015. How developers search for code: A case study. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 191\u2013201."},{"key":"e_1_3_2_79_2","first-page":"1","volume-title":"Proceedings of the 2023 IEEE High Performance Extreme Computing Conference (HPEC)","author":"Samsi Siddharth","year":"2023","unstructured":"Siddharth Samsi, Dan Zhao, Joseph McDonald, Baolin Li, Adam Michaleas, Michael Jones, William Bergeron, Jeremy Kepner, Devesh Tiwari, and Vijay Gadepally. 2023. From words to watts: Benchmarking the energy costs of large language model inference. In Proceedings of the 2023 IEEE High Performance Extreme Computing Conference (HPEC). IEEE, 1\u20139."},{"key":"e_1_3_2_80_2","unstructured":"Max Sch\u00e4fer Sarah Nadi Aryaz Eghbali and Frank Tip. 2023. Adaptive test generation using a large language model. arXiv:2302.06527. Retrieved from https:\/\/arxiv.org\/abs\/2302.06527"},{"issue":"1","key":"e_1_3_2_81_2","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/TSE.2023.3334955","article-title":"An empirical evaluation of using large language models for automated unit test generation","volume":"50","author":"Sch\u00e4fer Max","year":"2023","unstructured":"Max Sch\u00e4fer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. 2023. An empirical evaluation of using large language models for automated unit test generation. IEEE Transactions on Software Engineering 50, 1 (2023), 85\u2013105.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2532875"},{"key":"e_1_3_2_83_2","first-page":"201","volume-title":"Proceedings of the 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201915)","author":"Shamshiri Sina","year":"2015","unstructured":"Sina Shamshiri, Ren\u00e9 Just, Jos\u00e9 Miguel Rojas, Gordon Fraser, Phil McMinn, and Andrea Arcuri. 2015. Do automatically generated unit tests find real faults? An empirical study of effectiveness and challenges. In Proceedings of the 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201915). IEEE Press, 201\u2013211. DOI: 10.1109\/ASE.2015.86"},{"key":"e_1_3_2_84_2","unstructured":"Seung Yeob Shin Fabrizio Pastore Domenico Bianculli and Alexandra Baicoianu. 2024. Towards generating executable metamorphic relations using large language models. arXiv:2401.17019. Retrieved from https:\/\/arxiv.org\/abs\/2401.17019"},{"key":"e_1_3_2_85_2","first-page":"313","volume-title":"Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering","author":"Latif Siddiq Mohammed","year":"2024","unstructured":"Mohammed Latif Siddiq, Joanna Cecilia Da Silva Santos, Ridwanul Hasan Tanvir, Noshin Ulfat, Fahmid Al Rifat, and Vin\u00edcius Carvalho Lopes. 2024. Using large language models to generate junit tests: An empirical study. In Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering, 313\u2013322."},{"issue":"1","key":"e_1_3_2_86_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2063239.2063243","article-title":"How well do search engines support code retrieval on the web?","volume":"21","author":"Sim Susan Elliott","year":"2011","unstructured":"Susan Elliott Sim, Medha Umarji, Sukanya Ratanotayanon, and Cristina V Lopes. 2011. How well do search engines support code retrieval on the web? ACM Transactions on Software Engineering and Methodology 21, 1 (2011), 1\u201325.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_87_2","doi-asserted-by":"crossref","unstructured":"Dananjay Srinivas Rohan Das Saeid Tizpaz-Niari Ashutosh Trivedi and Maria Leonor Pacheco. 2023. On the potential and limitations of few-shot in-context learning to generate metamorphic specifications for tax preparation software. arXiv:2311.11979. Retrieved from https:\/\/arxiv.org\/abs\/2311.11979","DOI":"10.18653\/v1\/2023.nllp-1.23"},{"issue":"3","key":"e_1_3_2_88_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2581377","article-title":"Solving the search for source code","volume":"23","author":"Stolee Kathryn T.","year":"2014","unstructured":"Kathryn T. Stolee, Sebastian Elbaum, and Daniel Dobos. 2014. Solving the search for source code. ACM Transactions on Software Engineering and Methodology 23, 3 (2014), 1\u201345.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_89_2","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1145\/2593882.2593887","volume-title":"Proceedings of the Future of Software Engineering","author":"Storey Margaret-Anne","year":"2014","unstructured":"Margaret-Anne Storey, Leif Singer, Brendan Cleary, Fernando Figueira Filho, and Alexey Zagalsky. 2014. The (R)Evolution of social media in software engineering. In Proceedings of the Future of Software Engineering, 100\u2013116."},{"key":"e_1_3_2_90_2","first-page":"1178","volume-title":"Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Terragni Valerio","year":"2020","unstructured":"Valerio Terragni, Gunel Jahangirova, Paolo Tonella, and Mauro Pezz\u00e8. 2020. Evolutionary improvement of assertion oracles. In Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 1178\u20131189."},{"key":"e_1_3_2_91_2","first-page":"85","volume-title":"Proceedings of the 43rd IEEE\/ACM International Conference on Software Engineering Companion (ICSE \u201921)","author":"Terragni Valerio","year":"2021","unstructured":"Valerio Terragni, Gunel Jahangirova, Paolo Tonella, and Mauro Pezz\u00e8. 2021. GAssert: A fully automated tool to improve assertion oracles. In Proceedings of the 43rd IEEE\/ACM International Conference on Software Engineering Companion (ICSE \u201921), 85\u201388."},{"key":"e_1_3_2_92_2","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1145\/2931037.2931058","volume-title":"Proceedings of the 25th International Symposium on Software Testing and Analysis","author":"Terragni Valerio","year":"2016","unstructured":"Valerio Terragni, Yepang Liu, and Shing-Chi Cheung. 2016. CSNIPPEX: Automated synthesis of compilable code snippets from Q&A sites. In Proceedings of the 25th International Symposium on Software Testing and Analysis, 118\u2013129."},{"key":"e_1_3_2_93_2","first-page":"542","volume-title":"Proceedings of the 36th IEEE\/ACM International Conference on Automated Software Engineering","author":"Terragni Valerio","year":"2021","unstructured":"Valerio Terragni and Pasquale Salza. 2021. APIzation: Generating reusable APIs from stackoverflow code snippets. In Proceedings of the 36th IEEE\/ACM International Conference on Automated Software Engineering, 542\u2013554."},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3219458"},{"key":"e_1_3_2_95_2","first-page":"678","volume-title":"Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)","author":"Tsigkanos Christos","year":"2023","unstructured":"Christos Tsigkanos, Pooja Rani, Sebastian M\u00fcller, and Timo Kehrer. 2023. Large language models: The next frontier for variable discovery within metamorphic testing? In Proceedings of the 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 678\u2013682."},{"key":"e_1_3_2_96_2","first-page":"221","volume-title":"Proceedings of the 2021 IEEE 29th International Requirements Engineering Conference (RE)","author":"Van Oordt Simon","year":"2021","unstructured":"Simon Van Oordt and Emitza Guzman. 2021. On the role of user feedback in software evolution: A practitioners\u2019 perspective. In Proceedings of the 2021 IEEE 29th International Requirements Engineering Conference (RE). IEEE, 221\u2013232."},{"issue":"4","key":"e_1_3_2_97_2","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1109\/TSE.2024.3368208","article-title":"Software testing with large language models: Survey, landscape, and vision","volume":"50","author":"Wang Junjie","year":"2024","unstructured":"Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. 2024. Software testing with large language models: Survey, landscape, and vision. In IEEE Transactions on Software Engineering 50, 4 (2024), 911\u2013936.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_98_2","first-page":"716","volume-title":"Proceedings of the 28th INCOSE International Symposium","author":"Wheatcraft Louis S.","year":"2018","unstructured":"Louis S. Wheatcraft and Michael J. Ryan. 2018. Communicating requirements\u2014Effectively! In Proceedings of the 28th INCOSE International Symposium. Wiley Online Library, 716\u2013732."},{"issue":"6","key":"e_1_3_2_99_2","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MC.2003.1204373","article-title":"Agile software development: It\u2019s about feedback and change","volume":"36","author":"Williams Laurie","year":"2003","unstructured":"Laurie Williams and Alistair Cockburn. 2003. Agile software development: It\u2019s about feedback and change. Computer 36, 6 (2003), 39\u201343.","journal-title":"Computer"},{"key":"e_1_3_2_100_2","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang Junzhe Wang Senjie Jin Enyu Zhou et al. 2023. The rise and potential of large language model based agents: A survey. arXiv:2309.07864. Retrieved from https:\/\/arxiv.org\/abs\/2309.07864"},{"key":"e_1_3_2_101_2","first-page":"1482","volume-title":"Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Steven Xia Chunqiu","year":"2023","unstructured":"Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE). IEEE, 1482\u20131494."},{"key":"e_1_3_2_102_2","first-page":"959","volume-title":"Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","author":"Steven Xia Chunqiu","year":"2022","unstructured":"Chunqiu Steven Xia and Lingming Zhang. 2022. Less training, more repairing please: revisiting automated program repair via zero-shot learning. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 959\u2013971."},{"key":"e_1_3_2_103_2","unstructured":"Zhuokui Xie Yinghao Chen Chen Zhi Shuiguang Deng and Jianwei Yin. 2023. ChatUniTest: A ChatGPT-based automated unit test generation tool. arXiv:2305.04764. Retrieved from https:\/\/arxiv.org\/abs\/2305.04764"},{"key":"e_1_3_2_104_2","first-page":"557","volume-title":"Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering","author":"Xu Congying","year":"2024","unstructured":"Congying Xu, Songqiang Chen, Jiarong Wu, Shing-Chi Cheung, Valerio Terragni, Hengcheng Zhu, and Jialun Cao. 2024. MR-Adopt: Automatic deduction of input transformation function for metamorphic testing. In Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering, 557\u2013569. DOI: 10.1145\/3691620.3696020"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3656340"},{"key":"e_1_3_2_106_2","first-page":"563","volume-title":"Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC \u201919), Part II","author":"Xu Feiyu","year":"2019","unstructured":"Feiyu Xu, Hans Uszkoreit, Yangzhou Du, Wei Fan, Dongyan Zhao, and Jun Zhu. 2019. Explainable AI: A brief survey on history, research areas, approaches and challenges. In Proceedings of the 8th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC \u201919), Part II. Springer, 563\u2013574."},{"key":"e_1_3_2_107_2","unstructured":"Biwei Yan Kun Li Minghui Xu Yueyan Dong Yue Zhang Zhaochun Ren and Xiuzhen Cheng. 2024. On protecting the data privacy of large language models (LLMs): A survey. arXiv:2403.05156. Retrieved from https:\/\/arxiv.org\/abs\/2403.05156"},{"issue":"6","key":"e_1_3_2_108_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3649506","article-title":"Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond","volume":"18","author":"Yang Jingfeng","year":"2024","unstructured":"Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Shaochen Zhong, Bing Yin, and Xia Hu. 2024. Harnessing the power of LLMs in practice: A survey on ChatGPT and beyond. ACM Transactions on Knowledge Discovery from Data 18, 6 (2024), 1\u201332.","journal-title":"ACM Transactions on Knowledge Discovery from Data"},{"key":"e_1_3_2_109_2","doi-asserted-by":"crossref","first-page":"1607","DOI":"10.1145\/3691620.3695529","volume-title":"Proceedings of the 39th IEEE\/ACM International Conference on Automated Software Engineering","author":"Yang Lin","year":"2024","unstructured":"Lin Yang, Chen Yang, Shutao Gao, Weijing Wang, Bo Wang, Qihao Zhu, Xiao Chu, Jianyi Zhou, Guangtai Liang, Qianxiang Wang, et al. 2024. On the evaluation of large language models in unit test generation. In Proceedings of the 39th IEEE\/ACM International Conference on Automated Software Engineering, 1607\u20131619."},{"issue":"2","key":"e_1_3_2_110_2","doi-asserted-by":"crossref","first-page":"100211","DOI":"10.1016\/j.hcc.2024.100211","article-title":"A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly","volume":"4","author":"Yao Yifan","year":"2024","unstructured":"Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly. High-Confidence Computing 4, 2 (2024), 100211.","journal-title":"High-Confidence Computing"},{"key":"e_1_3_2_111_2","unstructured":"Burak Yeti\u015ftiren I\u015f\u0131k \u00d6zsoy Miray Ayerdem and Eray T\u00fcz\u00fcn. 2023. Evaluating the code quality of AI-assisted code generation tools: An empirical study on GitHub Copilot Amazon CodeWhisperer and ChatGPT. arXiv:2304.10778. Retrieved from https:\/\/arxiv.org\/abs\/2304.10778"},{"key":"e_1_3_2_112_2","volume-title":"The Requirements Engineering Handbook","author":"Young Ralph Rowland","year":"2004","unstructured":"Ralph Rowland Young. 2004. The Requirements Engineering Handbook. Artech House."},{"key":"e_1_3_2_113_2","unstructured":"Zhiqiang Yuan Yiling Lou Mingwei Liu Shiji Ding Kaixin Wang Yixuan Chen and Xin Peng. 2023. No more manual tests? Evaluating and improving ChatGPT for unit test generation. arXiv:2305.04207. Retrieved from https:\/\/arxiv.org\/abs\/2305.04207"},{"key":"e_1_3_2_114_2","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1109\/ICSME.2019.00035","volume-title":"Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME)","author":"Zhang Bo","year":"2019","unstructured":"Bo Zhang, Hongyu Zhang, Junjie Chen, Dan Hao, and Pablo Moscato. 2019. Automatic discovery and cleansing of numerical metamorphic relations. In Proceedings of the 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME), 235\u2013245. DOI: 10.1109\/ICSME.2019.00035"},{"key":"e_1_3_2_115_2","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1145\/2642937.2642994","volume-title":"Proceedings of the 29th ACM\/IEEE International Conference on Automated Software Engineering (ASE \u201914)","author":"Zhang J.","year":"2014","unstructured":"J. Zhang, J. Chen, D. Hao, Y. Xiong, B. Xie, L. Zhang, and H. Mei. 2014. Search-based inference of polynomial metamorphic relations. In Proceedings of the 29th ACM\/IEEE International Conference on Automated Software Engineering (ASE \u201914). ACM, New York, NY, 701\u2013712. DOI: 10.1145\/2642937.2642994"},{"key":"e_1_3_2_116_2","doi-asserted-by":"crossref","unstructured":"Quanjun Zhang Weifeng Sun Chunrong Fang Bowen Yu Hongyan Li Meng Yan Jianyi Zhou and Zhenyu Chen. 2024. Exploring automated assertion generation via large language models. ACM Transactions on Software Engineering and Methodology. Retrieved from https:\/\/dx.doi.org\/10.1145\/3699598","DOI":"10.1145\/3699598"},{"key":"e_1_3_2_117_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong et al. 2023. A survey of large language models. arXiv:2303.18223. Retrieved from https:\/\/arxiv.org\/abs\/2303.18223"},{"issue":"4","key":"e_1_3_2_118_2","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1109\/TSE.2017.2782280","article-title":"An empirical study on API usages","volume":"45","author":"Zhong Hao","year":"2017","unstructured":"Hao Zhong and Hong Mei. 2017. An empirical study on API usages. IEEE Transactions on Software Engineering 45, 4 (2017), 319\u2013334.","journal-title":"IEEE Transactions on Software Engineering"},{"issue":"3","key":"e_1_3_2_119_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1145\/3633453","article-title":"Measuring GitHub Copilot\u2019s impact on productivity","volume":"67","author":"Ziegler Albert","year":"2024","unstructured":"Albert Ziegler, Eirini Kalliamvakou, X. Alice Li, Andrew Rice, Devon Rifkin, Shawn Simister, Ganesh Sittampalam, and Edward Aftandilian. 2024. Measuring GitHub Copilot\u2019s impact on productivity. Communications of the ACM 67, 3 (2024), 54\u201363.","journal-title":"Communications of the ACM"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715003","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:57Z","timestamp":1750295877000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715003"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,26]]},"references-count":118,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3715003"],"URL":"https:\/\/doi.org\/10.1145\/3715003","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,26]]},"assertion":[{"value":"2024-03-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}