{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T08:33:29Z","timestamp":1773131609079,"version":"3.50.1"},"reference-count":270,"publisher":"Wiley","issue":"11","license":[{"start":{"date-parts":[[2025,7,29]],"date-time":"2025-07-29T00:00:00Z","timestamp":1753747200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Softw Pract Exp"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:title>ABSTRACT<\/jats:title><jats:sec><jats:title>Context<\/jats:title><jats:p>Generative artificial intelligence (GenAI) tools have become increasingly prevalent in software development, offering assistance to various managerial and technical project activities. Notable examples of these tools include OpenAI's ChatGPT, GitHub Copilot, and Amazon CodeWhisperer.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objective<\/jats:title><jats:p>Although many recent publications have explored and evaluated the application of GenAI, a comprehensive understanding of the current development, applications, limitations, and open challenges remains unclear to many. Particularly, we do not have an overall picture of the current state of GenAI technology in practical software engineering usage scenarios.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>We conducted a literature review and focus groups for a duration of five months to develop a research agenda on GenAI for software engineering.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We identified 78 open research questions (RQs) in 11 areas of software engineering. Our results show that it is possible to explore the adoption of GenAI in partial automation and support decision\u2010making in all software development activities. While the current literature is skewed toward software implementation, quality assurance and software maintenance, other areas, such as requirements engineering, software design, and software engineering education, would need further research attention. Common considerations when implementing GenAI include industry\u2010level assessment, dependability and accuracy, data accessibility, transparency, and sustainability aspects associated with the technology.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>GenAI is bringing significant changes to the field of software engineering. Nevertheless, the state of research on the topic still remains immature. We believe that this research agenda holds significance and practical value for informing both researchers and practitioners about current applications and guiding future research.<\/jats:p><\/jats:sec>","DOI":"10.1002\/spe.70005","type":"journal-article","created":{"date-parts":[[2025,7,29]],"date-time":"2025-07-29T10:23:51Z","timestamp":1753784631000},"page":"1806-1843","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Generative Artificial Intelligence for Software Engineering\u2014A Research Agenda"],"prefix":"10.1002","volume":"55","author":[{"given":"Anh","family":"Nguyen\u2010Duc","sequence":"first","affiliation":[{"name":"University of South Eastern Norway  B\u00f8I Telemark Norway"},{"name":"Norwegian University of Science and Technology  Trondheim Norway"}]},{"given":"Beatriz","family":"Cabrero\u2010Daniel","sequence":"additional","affiliation":[{"name":"University of Gothenburg  Gothenburg Sweden"}]},{"given":"Adam","family":"Przybylek","sequence":"additional","affiliation":[{"name":"Gdansk University of Technology  Gdansk Poland"}]},{"given":"Chetan","family":"Arora","sequence":"additional","affiliation":[{"name":"Monash University  Melbourne Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4760-5560","authenticated-orcid":false,"given":"Dron","family":"Khanna","sequence":"additional","affiliation":[{"name":"Free University of Bozen\u2010Bolzano  Bolzano Italy"}]},{"given":"Tomas","family":"Herda","sequence":"additional","affiliation":[{"name":"Austrian Post\u2014Konzern IT  Vienna Austria"}]},{"given":"Usman","family":"Rafiq","sequence":"additional","affiliation":[{"name":"Free University of Bozen\u2010Bolzano  Bolzano Italy"}]},{"given":"Jorge","family":"Melegati","sequence":"additional","affiliation":[{"name":"Free University of Bozen\u2010Bolzano  Bolzano Italy"}]},{"given":"Eduardo","family":"Guerra","sequence":"additional","affiliation":[{"name":"Free University of Bozen\u2010Bolzano  Bolzano Italy"}]},{"given":"Kai\u2010Kristian","family":"Kemell","sequence":"additional","affiliation":[{"name":"University of Helsinki  Helsinki Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7677-2355","authenticated-orcid":false,"given":"Mika","family":"Saari","sequence":"additional","affiliation":[{"name":"Tampere University  Tampere Finland"}]},{"given":"Zheying","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tampere University  Tampere Finland"}]},{"given":"Huy","family":"Le","sequence":"additional","affiliation":[{"name":"Ho Chi Minh City University of Technology  Ho Chi Minh City Vietnam"},{"name":"Vietnam National University Ho Chi Minh City  Ho Chi Minh City Vietnam"}]},{"given":"Tho","family":"Quan","sequence":"additional","affiliation":[{"name":"Ho Chi Minh City University of Technology  Ho Chi Minh City Vietnam"},{"name":"Vietnam National University Ho Chi Minh City  Ho Chi Minh City Vietnam"}]},{"given":"Pekka","family":"Abrahamsson","sequence":"additional","affiliation":[{"name":"Tampere University  Tampere Finland"}]}],"member":"311","published-online":{"date-parts":[[2025,7,29]]},"reference":[{"key":"e_1_2_10_2_1","unstructured":"W. X.Zhao K.Zhou J.Li et al. \u201cA Survey of Large Language Models \u201d2023 arXiv:2303.18223 [cs] https:\/\/doi.org\/10.48550\/arXiv.2303.18223."},{"key":"e_1_2_10_3_1","unstructured":"Z.Xi W.Chen X.Guo et al. \u201cThe Rise and Potential of Large Language Model Based Agents: A Survey \u201d2023 arXiv:2309.07864 [cs] https:\/\/doi.org\/10.48550\/arXiv.2309.07864."},{"key":"e_1_2_10_4_1","unstructured":"R. A.Poldrack T.Lu andG.Begu\u0161 \u201cAI\u2010Assisted Coding: Experiments With GPT\u20104 \u201d2023 arXiv:2304.13187 [cs] https:\/\/doi.org\/10.48550\/arXiv.2304.13187."},{"key":"e_1_2_10_5_1","doi-asserted-by":"crossref","unstructured":"P.Denny V.Kumar andN.Giacaman \u201cConversing With Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Language \u201d2022 arXiv:2210.15157 [cs] https:\/\/doi.org\/10.48550\/arXiv.2210.15157.","DOI":"10.1145\/3545945.3569823"},{"key":"e_1_2_10_6_1","doi-asserted-by":"crossref","unstructured":"J.\u2010B.D\u00f6derlein M.Acher D. E.Khelladi andB.Combemale \u201cPiloting Copilot and Codex: Hot Temperature Cold Prompts or Black Magic?\u201d2022 arXiv:2210.14699 [cs] https:\/\/doi.org\/10.48550\/arXiv.2210.14699.","DOI":"10.2139\/ssrn.4496380"},{"key":"e_1_2_10_7_1","unstructured":"Y.Dong J.Xue Z.Jin andG.Li \u201cSelf\u2010Collaboration Code Generation via ChatGPT \u201d2023 arXiv:2304.07590 [cs] https:\/\/doi.org\/10.48550\/arXiv.2304.07590."},{"key":"e_1_2_10_8_1","unstructured":"S.Ouyang J. M.Zhang M.Harman andM.Wang \u201cLLM is Like a Box of Chocolates: The Non\u2010Determinism of ChatGPT in Code Generation \u201d2023 arXiv:2308.02828 [cs] https:\/\/doi.org\/10.48550\/arXiv.2308.02828."},{"key":"e_1_2_10_9_1","unstructured":"Y.Liu G.Deng Z.Xu et al. \u201cJailbreaking ChatGPT via Prompt Engineering: An Empirical Study \u201d2023 arXiv:2305.13860 [cs] https:\/\/doi.org\/10.48550\/arXiv.2305.13860."},{"key":"e_1_2_10_10_1","unstructured":"W.Sun C.Fang Y.You et al. \u201cAutomatic Code Summarization via ChatGPT: How Far Are We?\u201d2023 arXiv:2305.12865 [cs] https:\/\/doi.org\/10.48550\/arXiv.2305.12865."},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.35179"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2022.3187912"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-COMPANION.2009.5070978"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134402"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-7998-5101-1.ch001"},{"key":"e_1_2_10_16_1","unstructured":"I.SriramandA.Khajeh\u2010Hosseini \u201cResearch Agenda in Cloud Technologies \u201d2010 https:\/\/doi.org\/10.48550\/arXiv.1001.3259."},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOSE.2007.14"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218843008001816"},{"key":"e_1_2_10_19_1","unstructured":"V.Jackson B.Vasilescu D.Russo et al. \u201cCreativity Generative AI and Software Development: A Research Agenda \u201d2024 arXiv:2406.01966 [cs] https:\/\/doi.org\/10.48550\/arXiv.2406.01966."},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-FoSE59343.2023.00008"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.06741"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3712003"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.02479"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3695988"},{"key":"e_1_2_10_25_1","unstructured":"J.Liu K.Wang Y.Chen et al. \u201cLarge Language Model\u2010Based Agents for Software Engineering: A Survey \u201d2024 arXiv:2409.02977 [cs] https:\/\/doi.org\/10.48550\/arXiv.2409.02977."},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2024.1436350"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3686803"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1186\/s42467-020-00005-4"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3487043"},{"key":"e_1_2_10_30_1","doi-asserted-by":"crossref","unstructured":"Z.Kotti R.Galanopoulou andD.Spinellis \u201cMachine Learning for Software Engineering: A Tertiary Study \u201d55 no.122022 256:1\u2013256:39 https:\/\/doi.org\/10.1145\/3572905.","DOI":"10.1145\/3572905"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics10212706"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2022.3192720"},{"key":"e_1_2_10_33_1","first-page":"1877","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems (Red Hook, NY, USA, 2020\u201012\u201006) (NIPS'20)","author":"Brown T. B.","year":"2020"},{"key":"e_1_2_10_34_1","unstructured":"A.RadfordandK.Narasimhan \u201cImproving Language Understanding by Generative Pre\u2010Training \u201d2018 https:\/\/www.semanticscholar.org\/paper\/Improving\u2010Language\u2010Understanding\u2010by\u2010Generative\u2010Radford\u2010Narasimhan\/cd18800a0fe0b668a1cc19f2ec95b5003d0a5035."},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1108\/LHTN-01-2023-0009"},{"key":"e_1_2_10_36_1","unstructured":"T.Brants A. C.Popat P.Xu F. J.Och andJ.Dean \u201cLarge Language Models in Machine Translation \u201d2007."},{"key":"e_1_2_10_37_1","first-page":"690","volume-title":"Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Heafield K.","year":"2013"},{"key":"e_1_2_10_38_1","first-page":"932","article-title":"A Neural Probabilistic Language Model","volume":"13","author":"Bengio Y.","year":"2000","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2010-343"},{"key":"e_1_2_10_40_1","unstructured":"T.Mikolov K.Chen G.Corrado andJ.Dean \u201cEfficient Estimation of Word Representations in Vector Space \u201d2013 arXiv Preprint arXiv:1301.3781."},{"key":"e_1_2_10_41_1","first-page":"5998","article-title":"Attention Is All You Need","volume":"30","author":"Vaswani A.","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_10_42_1","unstructured":"J. D. M. C. K.LeeandK.Toutanova \u201cPre\u2010Training of Deep Bidirectional Transformers for Language Understanding \u201d2018 arXiv Preprint arXiv:1810.04805."},{"issue":"8","key":"e_1_2_10_43_1","first-page":"9","article-title":"Language Models Are Unsupervised Multitask Learners","volume":"1","author":"Radford A.","year":"2019","journal-title":"OpenAI Blog"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1186\/1748\u20105908\u20105\u201056"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1186\/2046\u20104053\u20101\u201010"},{"key":"e_1_2_10_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32489-6_13"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/RCIS.2013.6577679"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-84800-044-5_4"},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISESE.2004.1334914"},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-38314-4_4"},{"key":"e_1_2_10_51_1","volume-title":"Focus Group Research Handbook","author":"Edmunds H.","year":"1998"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_2_10_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106294"},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-67084-9_12"},{"key":"e_1_2_10_55_1","doi-asserted-by":"publisher","DOI":"10.5220\/0006681900540064"},{"key":"e_1_2_10_56_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2014.10.046"},{"key":"e_1_2_10_57_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2575.2007.00259.x"},{"key":"e_1_2_10_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2023.110455"},{"key":"e_1_2_10_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2023.107176"},{"key":"e_1_2_10_60_1","doi-asserted-by":"crossref","unstructured":"J.White S.Hays F.Quchen J.Spencer\u2010Smith andD. C.Schmidt \u201cChatGPT Prompt Patterns for Improving Code Quality Refactoring Requirements Elicitation and Software Design \u201d2023 arXiv:2303.07839 [cs] https:\/\/doi.org\/10.48550\/arXiv.2303.07839.","DOI":"10.1007\/978-3-031-55642-5_4"},{"key":"e_1_2_10_61_1","doi-asserted-by":"crossref","unstructured":"K.Ronanki C.Berger andJ.Horkoff \u201cInvestigating ChatGPT's Potential to Assist in Requirements Elicitation Processes \u201d(2023) 354\u2010361.","DOI":"10.1109\/SEAA60479.2023.00061"},{"key":"e_1_2_10_62_1","unstructured":"Q.Zhang M.Chen A.Bukharin et al. \u201cAdaptive Budget Allocation for Parameter\u2010Efficient Fine\u2010Tuning \u201d2023 arXiv:2303.10512 [cs.CL]."},{"key":"e_1_2_10_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/MODELS-C59198.2023.00096"},{"key":"e_1_2_10_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510157"},{"key":"e_1_2_10_65_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2023.102994"},{"key":"e_1_2_10_66_1","doi-asserted-by":"crossref","unstructured":"C.Arora J.Grundy andM.Abdelrazek \u201cAdvancing Requirements Engineering Through Generative AI: Assessing the Role of LLMs \u201d2023 arXiv:2310.13976 [Cs.SE].","DOI":"10.1007\/978-3-031-55642-5_6"},{"key":"e_1_2_10_67_1","doi-asserted-by":"crossref","unstructured":"J.Zhang Y.Chen N.Niu andC.Liu \u201cA Preliminary Evaluation of ChatGPT in Requirements Information Retrieval \u201d2023 arXiv Preprint arXiv:2304.12562.","DOI":"10.2139\/ssrn.4450322"},{"key":"e_1_2_10_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA60479.2023.00017"},{"key":"e_1_2_10_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE54965.2022.00011"},{"key":"e_1_2_10_70_1","first-page":"1277","volume-title":"Proceedings of the 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE)","author":"Ezzini S.","year":"2023"},{"key":"e_1_2_10_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/RE48521.2020.00028"},{"key":"e_1_2_10_72_1","volume-title":"Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering (Rochester, MI, USA) (ASE \u201822)","author":"Luo X.","year":"2023"},{"key":"e_1_2_10_73_1","volume-title":"MoDRE Workshop at Requirement Engineering Conference","author":"Chen B.","year":"2023"},{"key":"e_1_2_10_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3546943"},{"key":"e_1_2_10_75_1","unstructured":"L.Yang S.Zhang J.Sun et al. \u201cTackling Cooperative Incompatibility for Zero\u2010Shot Human\u2010AI Coordination \u201d2023 arXiv:2306.03034 [cs] https:\/\/doi.org\/10.48550\/arXiv.2306.03034."},{"key":"e_1_2_10_76_1","doi-asserted-by":"publisher","DOI":"10.7763\/LNSE.2015.V3.185"},{"key":"e_1_2_10_77_1","first-page":"35","volume-title":"Proceedings of the 2016 5th International Conference on Informatics, Electronics and Vision (ICIEV)","author":"Nahar N.","year":"2016"},{"key":"e_1_2_10_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-019-05855-6"},{"key":"e_1_2_10_79_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-75417-8_3"},{"key":"e_1_2_10_80_1","first-page":"279","volume-title":"Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (Oulu, Finland) (EASE \u201823), New York, NY","author":"Ahmad A.","year":"2023"},{"key":"e_1_2_10_81_1","unstructured":"S.Herold C.Knieke M.Schindler andA.Rausch \u201cTowards Improving Software Architecture Degradation Mitigation by Machine Learning \u201d2020 https:\/\/urn.kb.se\/resolve?urn=urn:nbn:se:kau:diva\u201081120."},{"key":"e_1_2_10_82_1","volume-title":"AI\u2010Assisted Agile Software Development Workshop at XP2023 Conference","author":"Maranh\u00e3o Junior J. J.","year":"2023"},{"key":"e_1_2_10_83_1","unstructured":"T.StojanovicandS. D.Lazarevi\u0107 \u201cThe Application of ChatGPT for Identification of Microservices \u201d3 no.1(2023):99\u2013105 Number: 1 https:\/\/ebt.rs\/journals\/index.php\/conf\u2010proc\/article\/view\/181."},{"key":"e_1_2_10_84_1","doi-asserted-by":"crossref","unstructured":"R.Feldt S.Kang J.Yoon andS.Yoo \u201cTowards Autonomous Testing Agents via Conversational Large Language Models \u201d2023 arXiv:2306.05152 [cs] https:\/\/doi.org\/10.48550\/arXiv.2306.05152.","DOI":"10.1109\/ASE56229.2023.00148"},{"key":"e_1_2_10_85_1","volume-title":"Design Patterns","author":"Gamma E.","year":"1995"},{"key":"e_1_2_10_86_1","first-page":"77","volume-title":"International Conference on Agile Software Development","author":"Viviani L.","year":"2023"},{"key":"e_1_2_10_87_1","unstructured":"J.White F.Quchen S.Hays et al. \u201cA Prompt Pattern Catalog to Enhance Prompt Engineering With ChatGPT \u201d2023 arXiv:2302.11382 [cs] https:\/\/doi.org\/10.48550\/arXiv.2302.11382."},{"key":"e_1_2_10_88_1","unstructured":"J.Hoffmann S.Borgeaud A.Mensch et al. \u201cTraining Compute\u2010Optimal Large Language Models \u201d2022 arXiv:2203.15556 [cs.CL]."},{"key":"e_1_2_10_89_1","unstructured":"H.Touvron T.Lavril G.Izacard et al. \u201cLLaMA: Open and Efficient Foundation Language Models \u201d2023 arXiv:2302.13971 [cs.CL]."},{"key":"e_1_2_10_90_1","unstructured":"Andersen v. Stability AI Ltd \u201cCourtListener \u201d2023 https:\/\/www.courtlistener.com\/docket\/66732129\/parties\/andersen\u2010v\u2010stability\u2010ai\u2010ltd\/."},{"key":"e_1_2_10_91_1","unstructured":"Doe v. GitHub Inc \u201cCourtListener \u201d2022 https:\/\/www.courtlistener.com\/docket\/65669506\/doe\u20101\u2010v\u2010github\u2010inc\/."},{"key":"e_1_2_10_92_1","doi-asserted-by":"crossref","unstructured":"A. M.Dakhel V.Majdinasab A.Nikanjam F.Khomh M. C.Desmarais andZ. M. J.Jiang \u201cGitHub Copilot AI Pair Programmer: Asset or Liability? \u201d203(n. d):111734 https:\/\/doi.org\/10.1016\/j.jss.2023.111734.","DOI":"10.1016\/j.jss.2023.111734"},{"key":"e_1_2_10_93_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion55297.2022.9793778"},{"key":"e_1_2_10_94_1","first-page":"56","volume-title":"Proceedings of the 2023 IEEE\/ACM 2nd International Workshop on Natural Language\u2010Based Software Engineering (NLBSE) (2023\u201305)","author":"Siddiq M. L.","year":"2023"},{"key":"e_1_2_10_95_1","unstructured":"S.Jiang Y.Wang andW.Yu \u201cSelfEvolve: A Code Evolution Framework via Large Language Models \u201d2023 arXiv:2306.02907 [cs] https:\/\/doi.org\/10.48550\/arXiv.2306.02907."},{"key":"e_1_2_10_96_1","unstructured":"Y.Dong J.Xue Z.Jin andG.Li \u201cSelf\u2010Collaboration Code Generation via ChatGPT \u201d2023 arXiv:2304.07590 [cs.SE]."},{"key":"e_1_2_10_97_1","doi-asserted-by":"crossref","unstructured":"A.Borji \u201cA Categorical Archive of ChatGPT Failures \u201d2023 https:\/\/doi.org\/10.48550\/arXiv.2302.03494.","DOI":"10.21203\/rs.3.rs-2895792\/v1"},{"key":"e_1_2_10_98_1","first-page":"212","volume-title":"Proeedings of the 27th International Conference on Intelligent User Interfaces (New York, NY, USA, 2022\u201003\u201022) (IUI \u201822)","author":"Jiao Sun Q.","year":"2022"},{"key":"e_1_2_10_99_1","unstructured":"H.Li H.Yu Y.Zhai andZ.Qian \u201cThe Hitchhiker's Guide to Program Analysis: A Journey With Large Language Models \u201d2023 arXiv:2308.00245 [cs.SE]."},{"key":"e_1_2_10_100_1","unstructured":"B.Yeti\u015ftiren I.\u00d6zsoy M.Ayerdem andE.T\u00fcz\u00fcn \u201cEvaluating the Code Quality of AI\u2010Assisted Code Generation Tools: An Empirical Study on GitHub Copilot Amazon CodeWhisperer and ChatGPT \u201d2023 arXiv:2304.10778 [cs] https:\/\/doi.org\/10.48550\/arXiv.2304.10778."},{"key":"e_1_2_10_101_1","first-page":"1804","volume-title":"On the Effectiveness of Transfer Learning for Code Search, 49, no. 4","author":"Salza P.","year":"2021"},{"key":"e_1_2_10_102_1","doi-asserted-by":"crossref","unstructured":"F.Chen F.Fard D.Lo andT.Bryksin \u201cOn the Transferability of Pre\u2010Trained Language Models for Low\u2010Resource Programming Languages \u201d2022 arXiv:2204.09653 [cs] https:\/\/doi.org\/10.48550\/arXiv.2204.09653.","DOI":"10.1145\/3524610.3527917"},{"key":"e_1_2_10_103_1","unstructured":"Z.Feng D.Guo D.Tang et al. \u201cCodeBERT: A Pre\u2010Trained Model for Programming and Natural Languages \u201d(2002) arXiv:2002.08155 [cs] https:\/\/doi.org\/10.48550\/arXiv.2002.08155."},{"key":"e_1_2_10_104_1","unstructured":"D.Guo S.Ren S.Lu et al. \u201cGraphCodeBERT: Pre\u2010Training Code Representations With Data Flow \u201d(2009) arXiv:2009.08366 [cs] https:\/\/doi.org\/10.48550\/arXiv.2009.08366."},{"key":"e_1_2_10_105_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2021.04.019"},{"key":"e_1_2_10_106_1","unstructured":"L.Dong Y.Shen R.Jin Y.Mao K.Wang andW.Chen \u201cGeneration\u2010Augmented Query Expansion for Code Retrieval \u201d(n.d.) arXiv:2212.10692 [cs] https:\/\/doi.org\/10.48550\/arXiv.2212.10692."},{"key":"e_1_2_10_107_1","first-page":"376","volume-title":"Proceedings of the 2022 IEEE\/ACM 44th International Conference on Software Engineering (ICSE) (2022\u201305)","author":"Wei M.","year":"2022"},{"key":"e_1_2_10_108_1","unstructured":"K.Zhang H.Zhang G.Li J.Li Z.Li andZ.Jin \u201cToolCoder: Teach Code Generation Models to Use API Search Tools \u201d(n.d.) arXiv:2305.04032 [cs] https:\/\/doi.org\/10.48550\/arXiv.2305.04032."},{"key":"e_1_2_10_109_1","unstructured":"S. G.Patil T.Zhang X.Wang andJ. E.Gonzalez \u201cGorilla: Large Language Model Connected With Massive APIs \u201d(n.d.) arXiv:2305.15334 [cs] https:\/\/doi.org\/10.48550\/arxiv.2305.15334."},{"key":"e_1_2_10_110_1","first-page":"336","volume-title":"Proceedings of the 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE)","author":"Mastropaolo A.","year":"2021"},{"key":"e_1_2_10_111_1","unstructured":"M.Geng S.Wang D.Dong et al. \u201cLarge Language Models Are Few\u2010Shot Summarizers: Multi\u2010Intent Comment Generation via In\u2010Context Learning \u201d(n.d.) arXiv:2304.11384 [cs] https:\/\/doi.org\/10.48550\/arXiv.2304.11384."},{"key":"e_1_2_10_112_1","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519665"},{"key":"e_1_2_10_113_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586030"},{"key":"e_1_2_10_114_1","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534864"},{"key":"e_1_2_10_115_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"e_1_2_10_116_1","doi-asserted-by":"crossref","unstructured":"C.Bird D.Ford T.Zimmermann et al. \u201cTaking Flight With Copilot: Early Insights and Opportunities of AI\u2010Powered Pair\u2010Programming Tools \u201d20 no.6(2023) 10:35\u201310:57 https:\/\/doi.org\/10.1145\/3582083.","DOI":"10.1145\/3582083"},{"key":"e_1_2_10_117_1","unstructured":"E. J.Hu Y.Shen P.Wallis et al. \u201cLoRA: Low\u2010Rank Adaptation of Large Language Models \u201d(2021) arXiv:2106.09685 [cs.CL]."},{"key":"e_1_2_10_118_1","first-page":"4582","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Li X. L.","year":"2021"},{"key":"e_1_2_10_119_1","doi-asserted-by":"crossref","unstructured":"X.Liu K.Ji F.Yicheng et al. \u201cP\u2010Tuning v2: Prompt Tuning Can Be Comparable to Fine\u2010Tuning Universally Across Scales and Tasks \u201d(2022) arXiv:2110.07602 [cs.CL].","DOI":"10.18653\/v1\/2022.acl-short.8"},{"key":"e_1_2_10_120_1","unstructured":"X.Liu Y.Zheng D.Zhengxiao et al. \u201cGPT Understands Too \u201d(2021) arXiv:2103.10385 [cs.CL]."},{"key":"e_1_2_10_121_1","doi-asserted-by":"crossref","unstructured":"B.Lester R.Al\u2010Rfou andN.Constant \u201cThe Power of Scale for Parameter\u2010Efficient Prompt Tuning \u201d(2021) arXiv:2104.08691 [cs.CL].","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_2_10_122_1","unstructured":"Z.Wang R.Panda L.Karlinsky R.Feris H.Sun andY.Kim \u201cMultitask Prompt Tuning Enables Parameter\u2010Efficient Transfer Learning \u201d(2023) arXiv:2303.02861 [cs.CL]."},{"key":"e_1_2_10_123_1","unstructured":"H.Liu D.Tam M.Muqeeth et al. \u201cFew\u2010Shot Parameter\u2010Efficient Fine\u2010Tuning is Better and Cheaper Than In\u2010Context Learning \u201d(2022) arXiv:2205.05638 [cs.LG]."},{"key":"e_1_2_10_124_1","unstructured":"R.Li L. B.Allal Y.Zi et al. \u201cStarCoder: May the Source be With You!\u201d(2023) arXiv:2305.06161 [cs.CL]."},{"issue":"2002","key":"e_1_2_10_125_1","article-title":"The Economic Impacts of Inadequate Infrastructure for Software Testing","volume":"1","author":"Planning S.","year":"2002","journal-title":"National Institute of Standards and Technology"},{"key":"e_1_2_10_126_1","unstructured":"TestRail \u201cThe 2023 Software Testing and Quality Report \u201d(2023) TestRail https:\/\/www.testrail.com\/resource\/the\u20102023\u2010software\u2010testing\u2010quality\u2010report\/."},{"issue":"4","key":"e_1_2_10_127_1","article-title":"Autotestgpt: A System for the Automated Generation of Software Test Cases Based on Chatgpt","volume":"19","author":"Liu H.","year":"2021","journal-title":"Journal of Software"},{"key":"e_1_2_10_128_1","unstructured":"Z.Yuan Y.Lou M.Liu et al. \u201cNo More Manual Tests? Evaluating and Improving ChatGPT for Unit Test Generation \u201d(2023) arXiv Preprint arXiv:2305.04207."},{"key":"e_1_2_10_129_1","doi-asserted-by":"publisher","DOI":"10.1145\/3572905"},{"key":"e_1_2_10_130_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW58534.2023.00078"},{"key":"e_1_2_10_131_1","unstructured":"Q.Zhang C.Fang X.Yang et al. \u201cA Survey on Large Language Models for Software Engineering \u201d(2023) arXiv:2312.15223 [cs.SE]."},{"key":"e_1_2_10_132_1","unstructured":"J.Wang Y.Huang C.Chen Z.Liu S.Wang andQ.Wang \u201cSoftware Testing With Large Language Model: Survey Landscape and Vision \u201d(2023) arXiv Preprint arXiv:2307.07221."},{"key":"e_1_2_10_133_1","unstructured":"W.Ma S.Liu W.Wang et al. \u201cThe Scope of ChatGPT in Software Engineering: A Thorough Investigation \u201d(2023) arXiv Preprint arXiv:2305.12138."},{"key":"e_1_2_10_134_1","doi-asserted-by":"crossref","unstructured":"J.White S.Hays F.Quchen J.Spencer\u2010Smith andD. C.Schmidt \u201cChatgpt Prompt Patterns for Improving Code Quality Refactoring Requirements Elicitation and Software Design \u201d(2023) arXiv Preprint arXiv:2303.07839.","DOI":"10.1007\/978-3-031-55642-5_4"},{"key":"e_1_2_10_135_1","unstructured":"M. A.Akbar A. A.Khan andP.Liang \u201cEthical Aspects of ChatGPT in Software Engineering Research \u201d(2023) arXiv Preprint arXiv:2306.07557."},{"key":"e_1_2_10_136_1","doi-asserted-by":"crossref","unstructured":"S.Alagarsamy C.Tantithamthavorn andA.Aleti \u201cA3Test: Assertion\u2010Augmented Automated Test Case Generation \u201d(2023) arXiv:2302.10352 [Cs].","DOI":"10.2139\/ssrn.4724885"},{"key":"e_1_2_10_137_1","doi-asserted-by":"crossref","unstructured":"J.Shin S.Hashtroudi H.Hemmati andS.Wang \u201cDomain Adaptation for Deep Unit Test Case Generation \u201d(2024) arXiv:2308.08033 [cs] https:\/\/doi.org\/10.48550\/arXiv.2308.08033.","DOI":"10.1145\/3650212.3680354"},{"key":"e_1_2_10_138_1","unstructured":"B.Steenhoek M.Tufano N.Sundaresan andA.Svyatkovskiy \u201cReinforcement Learning From Automatic Feedback for High\u2010Quality Unit Test Generation \u201d(2023) arXiv:2310.02368 [cs] https:\/\/doi.org\/10.48550\/arXiv.2310.02368."},{"key":"e_1_2_10_139_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2016.09.002"},{"key":"e_1_2_10_140_1","unstructured":"A.Mastropaolo E.Aghajani L.Pascarella andG.Bavota \u201cAn Empirical Study on Code Comment Completion \u201d(n.d.) arXiv:2107.10544 [cs] https:\/\/doi.org\/10.48550\/arXiv.2107.10544."},{"key":"e_1_2_10_141_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3183297"},{"key":"e_1_2_10_142_1","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527220"},{"key":"e_1_2_10_143_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00178"},{"key":"e_1_2_10_144_1","doi-asserted-by":"crossref","unstructured":"Z.Zheng K.Ning J.Chen et al. \u201cTowards an Understanding of Large Language Models in Software Engineering Tasks \u201d(2023) arXiv:2308.11396 [cs.SE].","DOI":"10.1007\/s10664-024-10602-0"},{"key":"e_1_2_10_145_1","doi-asserted-by":"crossref","unstructured":"R.Feldt S.Kang J.Yoon andS.Yoo \u201cTowards Autonomous Testing Agents via Conversational Large Language Models \u201d(2023) arXiv Preprint arXiv:2306.05152.","DOI":"10.1109\/ASE56229.2023.00148"},{"key":"e_1_2_10_146_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664\u2010017\u20109580\u20107"},{"key":"e_1_2_10_147_1","doi-asserted-by":"publisher","DOI":"10.1201\/9781420013115"},{"key":"e_1_2_10_148_1","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2007.21"},{"key":"e_1_2_10_149_1","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389258"},{"key":"e_1_2_10_150_1","unstructured":"W.Sun C.Fang Y.You et al. \u201cAutomatic Code Summarization via ChatGPT: How Far Are We?\u201d(2023) arXiv:2305.12865 [cs.SE]."},{"key":"e_1_2_10_151_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227188"},{"key":"e_1_2_10_152_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2734091"},{"key":"e_1_2_10_153_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2006.07.007"},{"key":"e_1_2_10_154_1","unstructured":"D.NoeverandK.Williams \u201cChatbots as Fluent Polyglots: Revisiting Breakthrough Code Snippets \u201d(n.d.) arXiv:2301.03373 [cs] https:\/\/doi.org\/10.48550\/arXiv.2301.03373."},{"key":"e_1_2_10_155_1","volume-title":"Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering (Rochester, MI, USA) (ASE'22)","author":"Ahmed T.","year":"2023"},{"key":"e_1_2_10_156_1","doi-asserted-by":"publisher","DOI":"10.1002\/spe.3355"},{"key":"e_1_2_10_157_1","doi-asserted-by":"crossref","unstructured":"Y.Wang H.Le A. D.Gotmare N. D. Q.Bui J.Li andS. C. H.Hoi \u201cCodeT5+: Open Code Large Language Models for Code Understanding and Generation \u201d(2023) arXiv:2305.07922 [cs.CL].","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"e_1_2_10_158_1","doi-asserted-by":"crossref","unstructured":"R.Pan A. R.Ibrahimzada R.Krishna et al. \u201cLost in Translation: A Study of Bugs Introduced by Large Language Models While Translating Code \u201d(2023) arXiv:2308.03109 [cs.SE].","DOI":"10.1145\/3597503.3639226"},{"key":"e_1_2_10_159_1","doi-asserted-by":"crossref","unstructured":"G.Yang Z.Yu X.Zhang X.Chen T.Han andT.Chen \u201cAssessing and Improving Syntactic Adversarial Robustness of Pre\u2010Trained Models for Code Translation \u201d(2023) arXiv:2310.18587 [cs.SE].","DOI":"10.2139\/ssrn.4623115"},{"key":"e_1_2_10_160_1","unstructured":"A.Madaan A.Shypula U.Alon et al. \u201cLearning Performance\u2010Improving Code Edits \u201d(2023) arXiv:2302.07867 [cs.SE]."},{"key":"e_1_2_10_161_1","doi-asserted-by":"publisher","DOI":"10.1002\/SPE.3158"},{"key":"e_1_2_10_162_1","unstructured":"Q.Zhang C.Fang W.Sun et al. \u201cBoosting Automated Patch Correctness Prediction via Pre\u2010Trained Language Model \u201d(2023) arXiv:2301.12453 [cs.SE]."},{"key":"e_1_2_10_163_1","unstructured":"C. S.Xia Y.Wei andL.Zhang \u201cPractical Program Repair in the Era of Large Pre\u2010Trained Language Models \u201d(2022) arXiv:2210.14179 [cs.SE]."},{"key":"e_1_2_10_164_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179420"},{"key":"e_1_2_10_165_1","doi-asserted-by":"crossref","unstructured":"O.Asare M.Nagappan andN.Asokan \u201cIs GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?\u201d(2023) arXiv:2204.04741 [cs.SE].","DOI":"10.1007\/s10664-023-10380-1"},{"key":"e_1_2_10_166_1","article-title":"Execution\u2010Based Code Generation Using Deep Reinforcement Learning","author":"Shojaee P.","year":"2023","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_2_10_167_1","unstructured":"X.Hou Y.Zhao Y.Liu et al. \u201cLarge Language Models for Software Engineering: A Systematic Literature Review \u201d(2023) arXiv:2308.10620 [cs.SE]."},{"key":"e_1_2_10_168_1","volume-title":"Software Engineering","author":"Sommerville I.","year":"2016"},{"key":"e_1_2_10_169_1","doi-asserted-by":"publisher","DOI":"10.1109\/SEAA.2016.21"},{"key":"e_1_2_10_170_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2015.06.063"},{"key":"e_1_2_10_171_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC57700.2023.00117"},{"key":"e_1_2_10_172_1","first-page":"402","volume-title":"Proceedings of the 26th International Conference on Intelligent User Interfaces (New York, NY, USA, 2021\u201004\u201014) (IUI \u201821)","author":"Weisz J. D.","year":"2021"},{"key":"e_1_2_10_173_1","first-page":"491","volume-title":"Proceedings of the 28th International Conference on Intelligent User Interfaces (New York, NY, USA, 2023\u201003\u201027) (IUI \u201823)","author":"Steven I.","year":"2023"},{"key":"e_1_2_10_174_1","doi-asserted-by":"publisher","DOI":"10.1109\/IcETRAN59631.2023.10192161"},{"key":"e_1_2_10_175_1","doi-asserted-by":"publisher","DOI":"10.1145\/3583131.3590351"},{"key":"e_1_2_10_176_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3282111"},{"key":"e_1_2_10_177_1","unstructured":"N.Nascimento P.Alencar andD.Cowan \u201cComparing Software Developers With ChatGPT: An Empirical Investigation \u201d2023 arXiv:2305.11837 [cs] https:\/\/doi.org\/10.48550\/arXiv.2305.11837."},{"key":"e_1_2_10_178_1","volume-title":"Essentials of Project and Systems Engineering Management","author":"Eisner H.","year":"2008"},{"key":"e_1_2_10_179_1","doi-asserted-by":"publisher","DOI":"10.1177\/0008125619867910"},{"issue":"4","key":"e_1_2_10_180_1","first-page":"3","article-title":"Ammeter and Janet M Dukerich. 2002. Leadership, Team Building, and Team Member Characteristics in High Performance Project Teams","volume":"14","author":"Anthony P.","year":"2002","journal-title":"Engineering Management Journal"},{"key":"e_1_2_10_181_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEM.2008.927818"},{"key":"e_1_2_10_182_1","volume-title":"System Engineering Management","author":"Blanchard B. S.","year":"2004"},{"key":"e_1_2_10_183_1","volume-title":"Engineering Project Management","author":"Smith N. J.","year":"2002"},{"key":"e_1_2_10_184_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00170-021-06882-1"},{"key":"e_1_2_10_185_1","doi-asserted-by":"publisher","DOI":"10.1177\/87569728211061779"},{"key":"e_1_2_10_186_1","doi-asserted-by":"publisher","DOI":"10.26417\/667hri67"},{"key":"e_1_2_10_187_1","first-page":"41","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE\u2010NIER)","author":"Dam H. K.","year":"2019"},{"issue":"1","key":"e_1_2_10_188_1","first-page":"183","article-title":"Elmousalami. 2020. Comparison of Artificial Intelligence Techniques for Project Conceptual Cost Prediction: A Case Study and Comparative Analysis","volume":"68","author":"Haytham H.","year":"2020","journal-title":"IEEE Transactions on Engineering Management"},{"key":"e_1_2_10_189_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3158252"},{"key":"e_1_2_10_190_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME55016.2022.00020"},{"key":"e_1_2_10_191_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3297447"},{"key":"e_1_2_10_192_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2023.3265877"},{"key":"e_1_2_10_193_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.autcon.2020.103517"},{"key":"e_1_2_10_194_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jobe.2021.103299"},{"key":"e_1_2_10_195_1","doi-asserted-by":"crossref","unstructured":"N. A.Parikh \u201cEmpowering Business Transformation: The Positive Impact and Ethical Considerations of Generative AI in Software Product Management\u2013A Systematic Literature Review \u201d(2023) arXiv Preprint arXiv:2306.04605.","DOI":"10.4018\/979-8-3693-1634-4.ch016"},{"key":"e_1_2_10_196_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEM.2021.3109983"},{"key":"e_1_2_10_197_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-19-9948-2_2"},{"key":"e_1_2_10_198_1","doi-asserted-by":"publisher","DOI":"10.1145\/3502771.3502775"},{"key":"e_1_2_10_199_1","doi-asserted-by":"publisher","DOI":"10.1109\/UYMS50627.2020.9247027"},{"key":"e_1_2_10_200_1","doi-asserted-by":"publisher","DOI":"10.1109\/INCET57972.2023.10170591"},{"key":"e_1_2_10_201_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICE.2019.8792673"},{"key":"e_1_2_10_202_1","doi-asserted-by":"publisher","DOI":"10.1080\/23311886.2023.2176363"},{"key":"e_1_2_10_203_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2022.e10300"},{"key":"e_1_2_10_204_1","doi-asserted-by":"publisher","DOI":"10.1080\/09585192.2020.1871399"},{"key":"e_1_2_10_205_1","doi-asserted-by":"crossref","unstructured":"J.Zhao T.Wang M.Yatskar V.Ordonez andK.\u2010W.Chang \u201cMen Also Like Shopping: Reducing Gender Bias Amplification Using Corpus\u2010Level Constraints \u201d(2017) arXiv:1707.09457 [cs.AI].","DOI":"10.18653\/v1\/D17-1323"},{"key":"e_1_2_10_206_1","unstructured":"OpenAI \u201cGPT\u20104 Technical Report \u201d(2023) arXiv:2303.08774 [cs.CL]."},{"key":"e_1_2_10_207_1","unstructured":"D.Adiwardana M.\u2010T.Luong D. R.So et al. \u201cTowards a Human\u2010Like Open\u2010Domain Chatbot. CoRR Abs\/2001.09977 \u201d(2020) arXiv:2001.09977 https:\/\/arxiv.org\/abs\/2001.09977."},{"key":"e_1_2_10_208_1","volume-title":"EUR\u2010Lex \u2010 52021PC0206 \u2010 EN \u2010 EUR\u2010Lex","author":"European Commission","year":"2021"},{"key":"e_1_2_10_209_1","doi-asserted-by":"crossref","unstructured":"A.Luis A. N.Gonz\u00e1lez I.Contreras\u2010McKay andD.Molina \u201cImproving Learning Experiences in Software Engineering Capstone Courses Using Artificial Intelligence Virtual Assistants \u201d30 no.5(n. d):1370\u20131389 https:\/\/doi.org\/10.1002\/cae.22526.","DOI":"10.1002\/cae.22526"},{"key":"e_1_2_10_210_1","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2018.8659282"},{"key":"e_1_2_10_211_1","first-page":"367","volume-title":"Information and Software Technologies (Cham, 2021) (Communications in Computer and Information Science)","author":"Binkis M.","year":"2021"},{"key":"e_1_2_10_212_1","doi-asserted-by":"publisher","DOI":"10.1109\/IMITEC45504.2019.9015857"},{"key":"e_1_2_10_213_1","doi-asserted-by":"publisher","DOI":"10.1109\/SIIE56031.2022.9982349"},{"key":"e_1_2_10_214_1","doi-asserted-by":"crossref","unstructured":"J. C.Paiva J. P.Leal and\u00c1.Figueira \u201cAutomated Assessment in Computer Science Education: A State\u2010of\u2010the\u2010Art Review \u201d22 no.3(n. d):34:1\u201334:40 https:\/\/doi.org\/10.1145\/3513140.","DOI":"10.1145\/3513140"},{"key":"e_1_2_10_215_1","doi-asserted-by":"crossref","unstructured":"M.\u0106uku\u0161i\u0107 \u017d.Gara\u010da andM.Jadri\u0107 \u201cOnline Self\u2010Assessment and Students Success in Higher Education Institutions \u201d72(2013):100\u2013109 https:\/\/doi.org\/10.1016\/j.compedu.2013.10.018.","DOI":"10.1016\/j.compedu.2013.10.018"},{"key":"e_1_2_10_216_1","doi-asserted-by":"publisher","DOI":"10.1109\/EDUCON54358.2023.10125121"},{"key":"e_1_2_10_217_1","doi-asserted-by":"publisher","DOI":"10.1145\/3587102.3588815"},{"key":"e_1_2_10_218_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2023.3300574"},{"key":"e_1_2_10_219_1","first-page":"147","volume-title":"Proceedings of the 2023 IEEE 35th International Conference on Software Engineering Education and Training (CSEE&T) (2023\u201308)","author":"Berrezueta\u2010Guzman J.","year":"2003"},{"key":"e_1_2_10_220_1","doi-asserted-by":"publisher","DOI":"10.23919\/MIPRO57284.2023.10159727"},{"key":"e_1_2_10_221_1","doi-asserted-by":"publisher","DOI":"10.1049\/icp.2023.1537"},{"key":"e_1_2_10_222_1","doi-asserted-by":"publisher","DOI":"10.1145\/3636243.3636249"},{"key":"e_1_2_10_223_1","doi-asserted-by":"publisher","DOI":"10.1145\/3636243.3636252"},{"key":"e_1_2_10_224_1","doi-asserted-by":"publisher","DOI":"10.1145\/3576882.3617921"},{"key":"e_1_2_10_225_1","first-page":"1","volume-title":"Proceedings of the 23rd Koli Calling International Conference on Computing Education Research (Koli Calling \u201823)","author":"Kazemitabaar M.","year":"2024"},{"key":"e_1_2_10_226_1","doi-asserted-by":"crossref","unstructured":"J.Savelka A.Agarwal C.Bogart andM.Sakr \u201cLarge Language Models (GPT) Struggle to Answer Multiple\u2010Choice Questions About Code \u201d2023 arXiv:2303.08033 [cs] https:\/\/doi.org\/10.48550\/arXiv.2303.08033.","DOI":"10.5220\/0011996900003470"},{"key":"e_1_2_10_227_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2023.100147"},{"key":"e_1_2_10_228_1","doi-asserted-by":"crossref","unstructured":"W.Lyu Y.Wang C.Tingting Y.Sun andY.Zhang \u201cEvaluating the Effectiveness of LLMs in Introductory Computer Science Education: A Semester\u2010Long Field Study \u201d(2024) arXiv:2404.13414 [cs] https:\/\/doi.org\/10.1145\/3657604.3662036.","DOI":"10.1145\/3657604.3662036"},{"key":"e_1_2_10_229_1","doi-asserted-by":"crossref","unstructured":"S.Rasnayaka G.Wang R.Shariffdeen andG.Neelakanta Iyer \u201cAn Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project \u201d(2024) arXiv:2401.16186 [cs] https:\/\/doi.org\/10.48550\/arXiv.2401.16186.","DOI":"10.1145\/3643795.3648379"},{"key":"e_1_2_10_230_1","first-page":"49","volume-title":"Proceedings of the 16th Annual ACM India Compute Conference (COMPUTE \u201823)","author":"Balse R.","year":"2023"},{"key":"e_1_2_10_231_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610406"},{"key":"e_1_2_10_232_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jfineco.2018.03.001"},{"key":"e_1_2_10_233_1","unstructured":"S.Blank \u201cThe Four Steps to the Epiphany: Successful Strategies for Products That Win \u201d(2007) Cafepress.Com."},{"key":"e_1_2_10_234_1","volume-title":"The Lean Startup: How Today's Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses","author":"Ries E.","year":"2011"},{"key":"e_1_2_10_235_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2020.2985621"},{"key":"e_1_2_10_236_1","doi-asserted-by":"publisher","DOI":"10.28945\/5078"},{"key":"e_1_2_10_237_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11846\u2010023\u201000696\u2010z"},{"key":"e_1_2_10_238_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41746\u2010023\u201000873\u20100"},{"key":"e_1_2_10_239_1","doi-asserted-by":"crossref","unstructured":"L.Zhong Z.Wang andJ.Shang \u201cDebug Like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step\u2010by\u2010Step \u201d(2024) arXiv:2402.16906 [cs] https:\/\/doi.org\/10.48550\/arXiv.2402.16906.","DOI":"10.18653\/v1\/2024.findings-acl.49"},{"key":"e_1_2_10_240_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_2_10_241_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.12269"},{"key":"e_1_2_10_242_1","unstructured":"W.Ma S.Liu W.Wang et al. \u201cThe Scope of ChatGPT in Software Engineering: A Thorough Investigation \u201d2023 arXiv:2305.12138 [cs] https:\/\/doi.org\/10.48550\/arXiv.2305.12138."},{"key":"e_1_2_10_243_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13054\u2010023\u201004393\u2010x"},{"key":"e_1_2_10_244_1","doi-asserted-by":"crossref","unstructured":"P.Manakul A.Liusie andM. J. F.Gales \u201cSelfCheckGPT: Zero\u2010Resource Black\u2010Box Hallucination Detection for Generative Large Language Models \u201d2023 arXiv:2303.08896 [cs] https:\/\/doi.org\/10.48550\/arXiv.2303.08896.","DOI":"10.18653\/v1\/2023.emnlp-main.557"},{"key":"e_1_2_10_245_1","unstructured":"C.Barrett B.Boyd E.Burzstein et al. \u201cIdentifying and Mitigating the Security Risks of Generative AI \u201d2023 arXiv:2308.14840 [cs] https:\/\/doi.org\/10.48550\/arXiv.2308.14840."},{"key":"e_1_2_10_246_1","unstructured":"D.Li Y.Wang X.Xing et al. \u201cQuantifying and Attributing the Hallucination of Large Language Models via Association Analysis \u201d(2023) arXiv:2309.05217 [cs] https:\/\/doi.org\/10.48550\/arXiv.2309.05217."},{"key":"e_1_2_10_247_1","unstructured":"Y.Tian W.Yan Q.Yang et al. \u201cCodeHalu: Code Hallucinations in LLMs Driven by Execution\u2010Based Verification \u201d(2024) arXiv:2405.00253 [cs] https:\/\/doi.org\/10.48550\/arXiv.2405.00253."},{"key":"e_1_2_10_248_1","unstructured":"F.Liu L.Yang S.Lin et al. \u201cExploring and Evaluating Hallucinations in LLM\u2010Powered Code Generation \u201d2024 arXiv:2404.00971 [cs] https:\/\/doi.org\/10.48550\/arXiv.2404.00971."},{"key":"e_1_2_10_249_1","doi-asserted-by":"crossref","unstructured":"L.Chen M.Zaharia andJ.Zou \u201cHow is ChatGPT's Behavior Changing Over Time?\u201d2023 arXiv:2307.09009 [cs.CL].","DOI":"10.1162\/99608f92.5317da47"},{"key":"e_1_2_10_250_1","doi-asserted-by":"publisher","DOI":"10.1145\/3639475.3640097"},{"key":"e_1_2_10_251_1","doi-asserted-by":"crossref","unstructured":"A.Hamid H. R.Samidi T.Finin P.Pappachan andR.Yus \u201cGenAIPABench: A Benchmark for Generative AI\u2010Based Privacy Assistants \u201d2023 arXiv:2309.05138 [cs] https:\/\/doi.org\/10.48550\/arXiv.2309.05138.","DOI":"10.56553\/popets-2024-0081"},{"key":"e_1_2_10_252_1","doi-asserted-by":"crossref","unstructured":"E.Kasneci K.Sessler S.K\u00fcchemann et al. \u201cChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education \u201d103(2023):102274 https:\/\/doi.org\/10.1016\/j.lindif.2023.102274.","DOI":"10.1016\/j.lindif.2023.102274"},{"key":"e_1_2_10_253_1","doi-asserted-by":"crossref","unstructured":"M. T.Ribeiro S.Singh andC.Guestrin \u201cWhy Should I Trust You?: Explaining the Predictions of Any Classifier \u201d(2016) arXiv:1602.04938 [cs.LG].","DOI":"10.18653\/v1\/N16-3020"},{"key":"e_1_2_10_254_1","doi-asserted-by":"crossref","unstructured":"D.Levin R.Cross andL.Abrams \u201cWhy Should I Trust You? Predictors of Interpersonal Trust in a Knowledge Transfer Context. (09 2002) \u201d(2002).","DOI":"10.5465\/apbpp.2002.7517527"},{"key":"e_1_2_10_255_1","doi-asserted-by":"publisher","DOI":"10.1145\/3639372"},{"key":"e_1_2_10_256_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-04083-2_2"},{"key":"e_1_2_10_257_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597512.3599715"},{"key":"e_1_2_10_258_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985964"},{"key":"e_1_2_10_259_1","first-page":"32","volume-title":"Sustainability in Software Engineering: A Systematic Literature Review","author":"Penzenstadler B.","year":"2012"},{"key":"e_1_2_10_260_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-09228-7_11"},{"key":"e_1_2_10_261_1","doi-asserted-by":"crossref","unstructured":"A. A.Chien \u201cGenAI: Giga$$$ TeraWatt\u2010Hours and GigaTons of CO2 \u201d66 no.8(n. d):5 https:\/\/doi.org\/10.1145\/3606254.","DOI":"10.1145\/3606254"},{"key":"e_1_2_10_262_1","doi-asserted-by":"publisher","DOI":"10.1145\/3604930.3605705"},{"key":"e_1_2_10_263_1","unstructured":"P.Henderson J.Hu J.Romoff E.Brunskill D.Jurafsky andJ.Pineau \u201cTowards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning \u201d(2002) arXiv:2002.05651 [cs] https:\/\/doi.org\/10.48550\/arXiv.2002.05651."},{"key":"e_1_2_10_264_1","unstructured":"J.Kaplan S.McCandlish T.Henighan et al. \u201cScaling Laws for Neural Language Models \u201d2020 arXiv:2001.08361 [cs stat] https:\/\/doi.org\/10.48550\/arXiv.2001.08361."},{"key":"e_1_2_10_265_1","unstructured":"J. W.Rae S.Borgeaud T.Cai et al. \u201cScaling Language Models: Methods Analysis & Insights From Training Gopher \u201d2021 arXiv:2112.11446 [cs] https:\/\/doi.org\/10.48550\/arXiv.2112.11446."},{"key":"e_1_2_10_266_1","unstructured":"R. K. E.Bellamy K.Dey M.Hind et al. \u201cAI Fairness 360: An Extensible Toolkit for Detecting Understanding and Mitigating Unwanted Algorithmic Bias \u201d(2018) Issue: arXiv:1810.01943 arXiv:1810.01943 [cs] https:\/\/doi.org\/10.48550\/arXiv.1810.01943."},{"key":"e_1_2_10_267_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494672"},{"key":"e_1_2_10_268_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeai.2023.100152"},{"key":"e_1_2_10_269_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-32436-9_1"},{"key":"e_1_2_10_270_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256\u2010019\u20100088\u20102"},{"key":"e_1_2_10_271_1","doi-asserted-by":"crossref","unstructured":"X.HuandH. K.Dam \u201cFuture of Software Engineering @ ICSE 2023 \u201d(2023) https:\/\/www.computer.org\/csdl\/proceedings\u2010article\/icse\u2010fose\/2023\/249600a001\/1V5Q5qMnr1e.","DOI":"10.1109\/ICSE-FoSE59343.2023.00005"}],"container-title":["Software: Practice and Experience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/spe.70005","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T01:19:28Z","timestamp":1759713568000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/spe.70005"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,29]]},"references-count":270,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1002\/spe.70005"],"URL":"https:\/\/doi.org\/10.1002\/spe.70005","archive":["Portico"],"relation":{},"ISSN":["0038-0644","1097-024X"],"issn-type":[{"value":"0038-0644","type":"print"},{"value":"1097-024X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,29]]},"assertion":[{"value":"2024-11-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-06-16","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}