{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T20:15:59Z","timestamp":1777407359875,"version":"3.51.4"},"reference-count":103,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T00:00:00Z","timestamp":1755820800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T00:00:00Z","timestamp":1755820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012306","name":"Universit\u00e0 degli Studi di Trieste","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100012306","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SN COMPUT. SCI."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>In recent years, the field of Natural Language Processing (NLP) has made considerable progress with the development of neural network-based models, leading to the creation of various Large Language Models (LLMs). These models have demonstrated strong performance in various NLP tasks, such as language translation, sentiment analysis, and named entity recognition. One notable application of LLMs is their ability to generate code automatically from simple problem descriptions. However, even advanced LLMs frequently generate incorrect code. To address this issue, we extend a recently proposed method that aims to improve the correctness of code generated by LLMs using an evolutionary approach known as Genetic Improvement (GI). Our method involves constructing a dynamic grammar based on the LLM-generated code and using a problem-agnostic fitness function. In our experiments, we evaluated the proposed method on 25 well-known and widely-used problems across four different LLMs, both open-source and proprietary models. We demonstrate that our approach significantly improves the accuracy of code generated by LLMs. Specifically, for problems that the LLM alone does not fully solve, we show that GI significantly improves the initial LLM-generated solution in 50% to 75% of cases across the tested models. Our proposed GI approach remains effective as long as the initial LLM-generated code, despite some errors, provides a solid foundation for constructing a correct program.<\/jats:p>","DOI":"10.1007\/s42979-025-04281-x","type":"journal-article","created":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T07:25:15Z","timestamp":1755847515000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Exploring the Effect of Genetic Improvement for Large Language Models-Generated Code"],"prefix":"10.1007","volume":"6","author":[{"given":"Giovanni","family":"Pinna","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Damiano","family":"Ravalico","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2772-4095","authenticated-orcid":false,"given":"Luigi","family":"Rovito","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Manzoni","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"De Lorenzo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,8,22]]},"reference":[{"key":"4281_CR1","doi-asserted-by":"crossref","unstructured":"Vaithilingam P, Zhang T, Glassman EL. Expectation vs experience: Evaluating the usability of code generation tools powered by large language models. In: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems. CHI EA \u201922. Association for Computing Machinery, New York, NY, USA 2022.","DOI":"10.1145\/3491101.3519665"},{"key":"4281_CR2","doi-asserted-by":"crossref","unstructured":"Sobania D, Briesch M, Rothlauf F. Choose your programming copilot: A comparison of the program synthesis performance of github copilot and genetic programming. In: Proceedings of the Genetic and Evolutionary Computation Conference, 2022;1019\u20131027.","DOI":"10.1145\/3512290.3528700"},{"key":"4281_CR3","doi-asserted-by":"crossref","unstructured":"Pinna G, Ravalico D, Rovito L, Manzoni L, De Lorenzo A. Enhancing large language models-based code generation by leveraging genetic improvement. In: European Conference on Genetic Programming (Part of EvoStar), 2024;108\u2013124. Springer.","DOI":"10.1007\/978-3-031-56957-9_7"},{"key":"4281_CR4","doi-asserted-by":"publisher","unstructured":"Helmuth T, McPhee NF, Spector L. In: Riolo, R., Worzel, W.P., Kotanchek, M., Kordon, A. (eds.) Lexicase Selection for Program Synthesis: A Diversity Analysis, 2016;151\u2013167. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-319-34223-8_9.","DOI":"10.1007\/978-3-319-34223-8_9"},{"key":"4281_CR5","doi-asserted-by":"publisher","unstructured":"Spector L. Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. In: Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation. GECCO \u201912, 401\u2013408. Association for Computing Machinery, New York, NY, USA 2012. https:\/\/doi.org\/10.1145\/2330784.2330846.","DOI":"10.1145\/2330784.2330846"},{"key":"4281_CR6","doi-asserted-by":"crossref","unstructured":"Helmuth T, Kelly P. Psb2: the second program synthesis benchmark suite. In: Proceedings of the Genetic and Evolutionary Computation Conference, 2021;785\u2013794.","DOI":"10.1145\/3449639.3459285"},{"issue":"3","key":"4281_CR7","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1007\/s10710-022-09434-y","volume":"23","author":"T Helmuth","year":"2022","unstructured":"Helmuth T, Kelly P. Applying genetic programming to psb2: the next generation program synthesis benchmark suite. Genet Program Evolvable Mach. 2022;23(3):375\u2013404.","journal-title":"Genet Program Evolvable Mach"},{"key":"4281_CR8","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I. Attention is all you need. Advances in neural information processing systems 2017;30."},{"key":"4281_CR9","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 2019. https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"4281_CR10","doi-asserted-by":"crossref","unstructured":"Gillioz A, Casas J, Mugellini E, Abou Khaled O. Overview of the transformer-based models for nlp tasks. In: 2020 15th Conference on Computer Science and Information Systems (FedCSIS), 2020;179\u2013183. IEEE.","DOI":"10.15439\/2020F20"},{"key":"4281_CR11","unstructured":"AI M. LLaMA: Open and Efficient Foundation Language Models 2023. https:\/\/arxiv.org\/abs\/2302.13971"},{"issue":"6","key":"4281_CR12","first-page":"7","volume":"3","author":"R Taori","year":"2023","unstructured":"Taori R, Gulrajani I, Zhang T, Dubois Y, Li X, Guestrin C, Liang P, Hashimoto TB. Alpaca: a strong, replicable instruction-following model. Stanf Cent Res Found Models. 2023;3(6):7 (https:\/\/crfmstanford.edu\/2023\/03\/13\/alpaca.html).","journal-title":"Stanf Cent Res Found Models"},{"key":"4281_CR13","doi-asserted-by":"crossref","unstructured":"Wang Y, Kordi Y, Mishra S, Liu A, Smith NA, Khashabi D, Hajishirzi H. Self-instruct: aligning language models with self-generated instructions 2023. https:\/\/arxiv.org\/abs\/2212.10560","DOI":"10.18653\/v1\/2023.acl-long.754"},{"key":"4281_CR14","unstructured":"AI M. Llama 2: Open foundation and fine-tuned chat models 2023. https:\/\/arxiv.org\/abs\/2307.09288"},{"key":"4281_CR15","unstructured":"AI M. Code Llama: open foundation models for code 2024. https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"4281_CR16","unstructured":"AI M. The Llama 3 herd of models. arXiv 2024. https:\/\/ai.meta.com\/research\/publications\/the-llama-3-herd-of-models\/."},{"key":"4281_CR17","first-page":"1877","volume":"33","author":"T Brown","year":"2020","unstructured":"Brown T, Mann B, Ryder N, Subbiah M, Kaplan JD, Dhariwal P, Neelakantan A, Shyam P, Sastry G, Askell A, et al. Language models are few-shot learners. Adv Neural Inf Process Syst. 2020;33:1877\u2013901.","journal-title":"Adv Neural Inf Process Syst"},{"key":"4281_CR18","unstructured":"OpenAI: GPT-4 technical report 2024. https:\/\/arxiv.org\/abs\/2303.08774."},{"key":"4281_CR19","unstructured":"McAleese N, Pokorny RM, Uribe JFC, Nitishinskaya E, Trebacz M, Leike J. LLM critics help catch LLM bugs 2024. https:\/\/arxiv.org\/abs\/2407.00215"},{"issue":"4","key":"4281_CR20","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1007\/s11280-024-01276-1","volume":"27","author":"J Chen","year":"2024","unstructured":"Chen J, Liu Z, Huang X, Wu C, Liu Q, Jiang G, Pu Y, Lei Y, Chen X, Wang X, et al. When large language models meet personalization: perspectives of challenges and opportunities. World Wide Web. 2024;27(4):42.","journal-title":"World Wide Web"},{"issue":"1\u20132","key":"4281_CR21","first-page":"1","volume":"4","author":"S Gulwani","year":"2017","unstructured":"Gulwani S, Polozov O, Singh R. Program synthesis. Found Trends\u00ae Prog Lang. 2017;4(1\u20132):1\u2013119.","journal-title":"Found Trends\u00ae Prog Lang"},{"issue":"3","key":"4281_CR22","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1145\/362566.362568","volume":"14","author":"Z Manna","year":"1971","unstructured":"Manna Z, Waldinger RJ. Toward automatic program synthesis. Commun ACM. 1971;14(3):151\u201365.","journal-title":"Commun ACM"},{"issue":"2","key":"4281_CR23","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1016\/0004-3702(75)90008-9","volume":"6","author":"Z Manna","year":"1975","unstructured":"Manna Z, Waldinger R. Knowledge and reasoning in program synthesis. Artif Intell. 1975;6(2):175\u2013208.","journal-title":"Artif Intell"},{"issue":"3","key":"4281_CR24","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1016\/0004-3702(80)90050-8","volume":"14","author":"W Bibel","year":"1980","unstructured":"Bibel W. Syntax-directed, semantics-supported program synthesis. Artif Intell. 1980;14(3):243\u201361.","journal-title":"Artif Intell"},{"key":"4281_CR25","unstructured":"Ward M. Proving program refinements and transformations. PhD thesis, University of Oxford PhD Thesis 1989."},{"key":"4281_CR26","doi-asserted-by":"publisher","unstructured":"H\u00fcttel H. On program synthesis and large language models. Commun ACM. 2024. https:\/\/doi.org\/10.1145\/3680410. Online First.","DOI":"10.1145\/3680410"},{"key":"4281_CR27","first-page":"28","volume":"665","author":"A-E Rugina","year":"2008","unstructured":"Rugina A-E, Thomas D, Olive X, Veran G. Gene-auto: automatic software code generation for real-time embedded systems. DASIA 2008 Data Syst Aerosp. 2008;665:28.","journal-title":"DASIA 2008 Data Syst Aerosp"},{"key":"4281_CR28","doi-asserted-by":"crossref","unstructured":"Sun H, Nie Y, Li X, Huang M, Tian J, Kong W. An automatic code generation method based on sequence generative adversarial network. In: 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), 2022;383\u2013390.","DOI":"10.1109\/DSC55868.2022.00059"},{"issue":"2","key":"4281_CR29","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1147\/sj.352.0151","volume":"35","author":"FJ Budinsky","year":"1996","unstructured":"Budinsky FJ, Finnie MA, Vlissides JM, Yu PS. Automatic code generation from design patterns. IBM Syst J. 1996;35(2):151\u201371.","journal-title":"IBM Syst J"},{"key":"4281_CR30","doi-asserted-by":"crossref","unstructured":"M\u00e9ry D, Singh NK. Automatic code generation from event-b models. In: Proceedings of the 2nd Symposium on Information and Communication Technology, 2011;179\u2013188.","DOI":"10.1145\/2069216.2069252"},{"key":"4281_CR31","doi-asserted-by":"crossref","unstructured":"Moreira TG, Wehrmeister MA, Pereira CE, Petin J-F, Levrat E. Automatic code generation for embedded systems: From uml specifications to vhdl code. In: 2010 8th IEEE International Conference on Industrial Informatics, 2010;1085\u20131090.","DOI":"10.1109\/INDIN.2010.5549590"},{"key":"4281_CR32","doi-asserted-by":"crossref","unstructured":"Liu Z, Dou Y, Jiang J, Xu J. Automatic code generation of convolutional neural networks in FPGA implementation. In: 2016 International Conference on Field-programmable Technology (FPT), 2016;61\u201368.","DOI":"10.1109\/FPT.2016.7929190"},{"issue":"3","key":"4281_CR33","doi-asserted-by":"publisher","first-page":"56","DOI":"10.3390\/computers9030056","volume":"9","author":"G Paolone","year":"2020","unstructured":"Paolone G, Marinelli M, Paesani R, Di Felice P. Automatic code generation of MVC web applications. Computers. 2020;9(3):56.","journal-title":"Computers"},{"key":"4281_CR34","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/BF00175355","volume":"4","author":"JR Koza","year":"1994","unstructured":"Koza JR. Genetic programming as a means for programming computers by natural selection. Stat Comput. 1994;4:87\u2013112.","journal-title":"Stat Comput"},{"issue":"4","key":"4281_CR35","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1109\/4235.942529","volume":"5","author":"M O\u2019Neill","year":"2001","unstructured":"O\u2019Neill M, Ryan C. Grammatical evolution. IEEE Trans Evol Comput. 2001;5(4):349\u201358.","journal-title":"IEEE Trans Evol Comput"},{"key":"4281_CR36","doi-asserted-by":"crossref","unstructured":"Ryan C, Collins JJ, Neill MO. Grammatical evolution: Evolving programs for an arbitrary language. In: Genetic Programming: First European Workshop, EuroGP\u201998 Paris, France, April 14\u201315, 1998 Proceedings 1, 1998;83\u201396. Springer.","DOI":"10.1007\/BFb0055930"},{"key":"4281_CR37","doi-asserted-by":"crossref","unstructured":"Karpuzcu UR. Automatic verilog code generation through grammatical evolution. In: Proceedings of the 7th Annual Workshop on Genetic and Evolutionary Computation, 2005;394\u2013397.","DOI":"10.1145\/1102256.1102346"},{"key":"4281_CR38","doi-asserted-by":"publisher","unstructured":"L\u00f6ppenberg M, Schwung A. Self optimisation and automatic code generation by evolutionary algorithms in plc based controlling processes. In: 2023 IEEE 21st International Conference on Industrial Informatics (INDIN), 2023;1\u20136. https:\/\/doi.org\/10.1109\/INDIN51400.2023.10218168.","DOI":"10.1109\/INDIN51400.2023.10218168"},{"key":"4281_CR39","doi-asserted-by":"crossref","unstructured":"Zhang Y, Li Y, Wang X. An optimized hybrid evolutionary algorithm for accelerating automatic code optimization. In Third International Seminar on Artificial Intelligence Networking and Information Technology (AINIT 2022). 2023;12587:488\u201396. SPIE.","DOI":"10.1117\/12.2667392"},{"key":"4281_CR40","unstructured":"Zheng L, Jia C, Sun M, Wu Z, Yu CH, Haj-Ali A, Wang Y, Yang J, Zhuo D, Sen K, et al. Ansor: Generating $$\\{$$High-Performance$$\\}$$ tensor programs for deep learning. In: 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20), 2020;863\u2013879."},{"key":"4281_CR41","unstructured":"Chen T, Moreau T, Jiang Z, Zheng L, Yan E, Shen H, Cowan M, Wang L, Hu Y, Ceze L, et al. $$\\{$$TVM$$\\}$$: An automated $$\\{$$End-to-End$$\\}$$ optimizing compiler for deep learning. In: 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), 2018;578\u2013594."},{"key":"4281_CR42","doi-asserted-by":"crossref","unstructured":"Sandnes FE, Megson GM. A hybrid genetic algorithm applied to automatic parallel controller code generation. In: Proceedings of the Eighth Euromicro Workshop on Real-Time Systems, 1996;70\u201375. IEEE.","DOI":"10.1109\/EMWRTS.1996.557799"},{"key":"4281_CR43","doi-asserted-by":"crossref","unstructured":"Miller JF, Harding SL. Cartesian genetic programming. In: Proceedings of the 10th Annual Conference Companion on Genetic and Evolutionary Computation, 2008;2701\u20132726.","DOI":"10.1145\/1388969.1389075"},{"key":"4281_CR44","doi-asserted-by":"crossref","unstructured":"Walker JA, Liu Y, Tempesti G, Tyrrell AM. Automatic code generation on a move processor using cartesian genetic programming. In: Evolvable Systems: From Biology to Hardware: 9th International Conference, ICES 2010, York, UK, September 6-8, 2010. Proceedings 9, 2010;238\u2013249. Springer.","DOI":"10.1007\/978-3-642-15323-5_21"},{"key":"4281_CR45","doi-asserted-by":"crossref","unstructured":"Serruto WF, Casas LA. Automatic code generation for microcontroller-based system using multi-objective linear genetic programming. In: 2017 International Conference on Computational Science and Computational Intelligence (CSCI), 2017;279\u2013285. IEEE.","DOI":"10.1109\/CSCI.2017.47"},{"key":"4281_CR46","doi-asserted-by":"crossref","unstructured":"Bahrini A, Khamoshifar M, Abbasimehr H, Riggs RJ, Esmaeili M, Majdabadkohne RM, Pasehvar M. Chatgpt: Applications, opportunities, and threats. In: 2023 Systems and Information Engineering Design Symposium (SIEDS), 2023;274\u2013279.","DOI":"10.1109\/SIEDS58326.2023.10137850"},{"issue":"6","key":"4281_CR47","doi-asserted-by":"publisher","first-page":"1548","DOI":"10.1109\/TSE.2024.3392499","volume":"50","author":"Z Liu","year":"2024","unstructured":"Liu Z, Tang Y, Luo X, Zhou Y, Zhang LF. No need to lift a finger anymore? assessing the quality of code generation by ChatGPT. IEEE Trans Software Eng. 2024;50(6):1548\u201384. https:\/\/doi.org\/10.1109\/TSE.2024.3392499.","journal-title":"IEEE Trans Software Eng"},{"key":"4281_CR48","unstructured":"Ouyang S, Zhang JM, Harman M, Wang M. LLM is Like a Box of Chocolates: the Non-determinism of ChatGPT in Code Generation 2023. https:\/\/arxiv.org\/abs\/2308.02828."},{"key":"4281_CR49","unstructured":"Austin J, Odena A, Nye M, Bosma M, Michalewski H, Dohan D, Jiang E, Cai C, Terry M, Le Q, Sutton C. Program synthesis with large language models 2021. https:\/\/arxiv.org\/abs\/2108.07732."},{"key":"4281_CR50","doi-asserted-by":"publisher","unstructured":"Sobania D, Petke J, Briesch M, Rothlauf F. A comparison of large language models and genetic programming for program synthesis. IEEE Transactions on Evolutionary Computation, 2024;1\u20131 https:\/\/doi.org\/10.1109\/TEVC.2024.3410873.","DOI":"10.1109\/TEVC.2024.3410873"},{"key":"4281_CR51","doi-asserted-by":"publisher","unstructured":"Hsu T-H, Chang C-H, Yu T-L. Program synthesis on single-layer loop behavior in pure functional programming. In: 2024 IEEE Congress on Evolutionary Computation (CEC), 2024;1\u20138. https:\/\/doi.org\/10.1109\/CEC60901.2024.10612128.","DOI":"10.1109\/CEC60901.2024.10612128"},{"key":"4281_CR52","doi-asserted-by":"publisher","unstructured":"Vella Zarb D, Parks G, Kipouros T. Synergistic utilization of llms for program synthesis. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201924 Companion, 539\u2013542. Association for Computing Machinery, New York, NY, USA 2024. https:\/\/doi.org\/10.1145\/3638530.3654426.","DOI":"10.1145\/3638530.3654426"},{"key":"4281_CR53","doi-asserted-by":"crossref","unstructured":"Liventsev V, Grishina A, H\u00e4rm\u00e4 A, Moonen L. Fully autonomous programming with large language models. In: Proceedings of the Genetic and Evolutionary Computation Conference. GECCO \u201923, 1146\u20131155. Association for Computing Machinery, New York, NY, USA 2023.","DOI":"10.1145\/3583131.3590481"},{"key":"4281_CR54","doi-asserted-by":"crossref","unstructured":"De La Torre C, Lavinas Y, Cortacero K, Luga H, Wilson DG, Cussat-Blanc S. Multimodal adaptive graph evolution for program synthesis. In: International Conference on Parallel Problem Solving from Nature, 2024;306\u2013321. Springer.","DOI":"10.1007\/978-3-031-70055-2_19"},{"key":"4281_CR55","unstructured":"Chen M, Tworek J, et al. Evaluating Large Language Models Trained on Code 2021. https:\/\/arxiv.org\/abs\/2107.03374."},{"key":"4281_CR56","doi-asserted-by":"crossref","unstructured":"Custode LL, Migliore Rambaldi CC, Roveri M, Iacca G. Comparing large language models and grammatical evolution for code generation. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201924 Companion, 1830\u20131837. Association for Computing Machinery, New York, NY, USA 2024. DOIurlhttps:\/\/doi.org\/10.1145\/3638530.3664162.","DOI":"10.1145\/3638530.3664162"},{"key":"4281_CR57","first-page":"21558","volume":"36","author":"J Liu","year":"2024","unstructured":"Liu J, Xia CS, Wang Y, Zhang L. Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation. Adv Neur Inform Process Syst. 2024;36:21558\u201372.","journal-title":"Adv Neur Inform Process Syst"},{"key":"4281_CR58","doi-asserted-by":"crossref","unstructured":"Li J, Li G, Zhang X, Dong Y, Jin Z. EvoCodeBench: an evolving code generation Benchmark aligned with real-world code repositories 2024. https:\/\/arxiv.org\/abs\/2404.00599.","DOI":"10.18653\/v1\/2024.findings-acl.214"},{"key":"4281_CR59","unstructured":"Wu Q, Peng C, Gao P, Hu R, Gan H, Jiang B, Tang J, Deng Z, Guan Z, Gao C, Liu X, Yang P. RepoMasterEval: evaluating code completion via real-world repositories 2024. https:\/\/arxiv.org\/abs\/2408.03519."},{"key":"4281_CR60","unstructured":"Piterbarg U, Pinto L, Fergus R. Training language models on synthetic edit sequences improves code synthesis 2024. https:\/\/arxiv.org\/abs\/2410.02749."},{"key":"4281_CR61","doi-asserted-by":"crossref","unstructured":"Langdon WB. Genetic improvement of programs. In: 2014 16th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, 2014;14\u201319. IEEE.","DOI":"10.1109\/SYNASC.2014.10"},{"issue":"1","key":"4281_CR62","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1109\/TEVC.2013.2281544","volume":"19","author":"WB Langdon","year":"2015","unstructured":"Langdon WB, Harman M. Optimizing existing software with genetic programming. IEEE Trans Evol Comput. 2015;19(1):118\u201335. https:\/\/doi.org\/10.1109\/TEVC.2013.2281544.","journal-title":"IEEE Trans Evol Comput"},{"key":"4281_CR63","doi-asserted-by":"publisher","unstructured":"Langdon WB, Ochoa G. Genetic improvement: A key challenge for evolutionary computation. In: 2016 IEEE Congress on Evolutionary Computation (CEC), 2016;3068\u20133075. https:\/\/doi.org\/10.1109\/CEC.2016.7744177.","DOI":"10.1109\/CEC.2016.7744177"},{"issue":"3","key":"4281_CR64","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1145\/3672089.3672102","volume":"49","author":"WB Langdon","year":"2024","unstructured":"Langdon WB, An G, Blot A, Nowack V, Petke J, Yoo S, Krauss O, Fredericks EM, Blackwell D. The 13th international workshop on genetic improvement(gi @ icse 2024). SIGSOFT Softw Eng Notes. 2024;49(3):42\u201350. https:\/\/doi.org\/10.1145\/3672089.3672102.","journal-title":"SIGSOFT Softw Eng Notes"},{"key":"4281_CR65","doi-asserted-by":"crossref","unstructured":"Petke J, Harman M, Langdon WB, Weimer W. Using genetic improvement and code transplants to specialise a c++ program to a problem class. In: Genetic Programming: 17th European Conference, EuroGP 2014, Granada, Spain, April 23-25, 2014, Revised Selected Papers 2014;17, 137\u2013149. Springer.","DOI":"10.1007\/978-3-662-44303-3_12"},{"issue":"6","key":"4281_CR66","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1109\/TSE.2017.2702606","volume":"44","author":"J Petke","year":"2018","unstructured":"Petke J, Harman M, Langdon WB, Weimer W. Specialising software for different downstream applications using genetic improvement and code transplantation. IEEE Trans Software Eng. 2018;44(6):574\u201394. https:\/\/doi.org\/10.1109\/TSE.2017.2702606.","journal-title":"IEEE Trans Software Eng"},{"key":"4281_CR67","doi-asserted-by":"crossref","unstructured":"Marino F, Squillero G, Tonda A. A general-purpose framework for genetic improvement. In: Parallel Problem Solving from Nature\u2013PPSN XIV: 14th International Conference, Edinburgh, UK, September 17-21, 2016, Proceedings 14, 2016;345\u2013352. Springer.","DOI":"10.1007\/978-3-319-45823-6_32"},{"key":"4281_CR68","doi-asserted-by":"crossref","unstructured":"Fenton M, McDermott J, Fagan D, Forstenlechner S, Hemberg E, O\u2019Neill M. Ponyge2: Grammatical evolution in python. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, 2017;1194\u20131201.","DOI":"10.1145\/3067695.3082469"},{"key":"4281_CR69","doi-asserted-by":"crossref","unstructured":"An G, Blot A, Petke J, Yoo S. Pyggi 2.0: Language independent genetic improvement framework. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2019;1100\u20131104.","DOI":"10.1145\/3338906.3341184"},{"key":"4281_CR70","unstructured":"Blot A, Petke J. MAGPIE: Machine automated general performance improvement via evolution of Software 2022. https:\/\/arxiv.org\/abs\/2208.02811."},{"key":"4281_CR71","doi-asserted-by":"publisher","unstructured":"Alshahwan N. Industrial experience of genetic improvement in Facebook. In: 2019 IEEE\/ACM International Workshop on Genetic Improvement (GI), 2019;1\u20131. https:\/\/doi.org\/10.1109\/GI.2019.00010.","DOI":"10.1109\/GI.2019.00010"},{"key":"4281_CR72","doi-asserted-by":"crossref","unstructured":"Callan J, Langdon WB, Petke J. On reducing network usage with genetic improvement. In: 13th International Workshop on Genetic Improvement@ ICSE 2024.","DOI":"10.1145\/3643692.3648262"},{"key":"4281_CR73","doi-asserted-by":"crossref","unstructured":"Langdon WB, Clark D. Deep mutations have little impact. In: 13th International Workshop on Genetic Improvement@ ICSE 2024.","DOI":"10.1145\/3643692.3648259"},{"key":"4281_CR74","doi-asserted-by":"publisher","unstructured":"Pluhacek M, Kazikova A, Kadavy T, Viktorin A, Senkerik R. 2023 Leveraging large language models for the generation of novel metaheuristic optimization algorithms. In: Proceedings of the Companion Conference on Genetic and Evolutionary Computation. GECCO \u201923 Companion, 1812\u20131820. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3583133.3596401.","DOI":"10.1145\/3583133.3596401"},{"key":"4281_CR75","unstructured":"Wu X, Wu S-H, Wu J, Feng L, Tan KC. Evolutionary computation in the era of large language model: survey and roadmap 2024. https:\/\/arxiv.org\/abs\/2401.10034."},{"key":"4281_CR76","doi-asserted-by":"crossref","unstructured":"Mouret J-B. Large language models help computer programs to evolve. Nature Publishing Group UK London 2024.","DOI":"10.1038\/d41586-023-03998-0"},{"issue":"7995","key":"4281_CR77","doi-asserted-by":"publisher","first-page":"468","DOI":"10.1038\/s41586-023-06924-6","volume":"625","author":"B Romera-Paredes","year":"2024","unstructured":"Romera-Paredes B, Barekatain M, Novikov A, Balog M, Kumar MP, Dupont E, Ruiz FJ, Ellenberg JS, Wang P, Fawzi O, et al. Mathematical discoveries from program search with large language models. Nature. 2024;625(7995):468\u201375.","journal-title":"Nature"},{"key":"4281_CR78","doi-asserted-by":"crossref","unstructured":"Hemberg E, Moskal S, O\u2019Reilly U-M. Evolving code with a large language model 2024. https:\/\/arxiv.org\/abs\/2401.07102.","DOI":"10.1007\/s10710-024-09494-2"},{"key":"4281_CR79","doi-asserted-by":"crossref","unstructured":"Lehman J, Gordon J, Jain S, Ndousse K, Yeh C, Stanley KO. Evolution through large models. Handbook of Evolutionary Machine Learning, 2023;331\u2013366.","DOI":"10.1007\/978-981-99-3814-8_11"},{"issue":"7","key":"4281_CR80","doi-asserted-by":"publisher","first-page":"287","DOI":"10.3390\/a17070287","volume":"17","author":"N Tao","year":"2024","unstructured":"Tao N, Ventresque A, Nallur V, Saber T. Enhancing program synthesis with large language models using many-objective grammar-guided genetic programming. Algorithms. 2024;17(7):287.","journal-title":"Algorithms"},{"key":"4281_CR81","unstructured":"Liu F, Lin X, Wang Z, Yao S, Tong X, Yuan M, Zhang Q. Large language model for multi-objective evolutionary optimization 2024. https:\/\/arxiv.org\/abs\/2310.12541."},{"key":"4281_CR82","unstructured":"Liu F, Tong X, Yuan M, Zhang Q. Algorithm evolution using large language model 2023. https:\/\/arxiv.org\/abs\/2311.15249."},{"key":"4281_CR83","unstructured":"Stein N, B\u00e4ck T. LLaMEA: a large language model evolutionary algorithm for automatically generating metaheuristics 2024. https:\/\/arxiv.org\/abs\/2405.20132."},{"key":"4281_CR84","doi-asserted-by":"crossref","unstructured":"Brahmachary S, Joshi SM, Panda A, Koneripalli K, Sagotra AK, Patel H, Sharma A, Jagtap AD, Kalyanaraman K. Large language model-based evolutionary optimizer: reasoning with elitism 2024. https:\/\/arxiv.org\/abs\/2403.02054.","DOI":"10.1016\/j.neucom.2024.129272"},{"key":"4281_CR85","unstructured":"Maddigan P, Lensen A, Xue B. Explaining genetic programming trees using large language models 2024. https:\/\/arxiv.org\/abs\/2403.03397."},{"key":"4281_CR86","doi-asserted-by":"publisher","unstructured":"Custode LL, Caraffini F, Yaman A, Iacca G. An investigation on the use of large language models for hyperparameter tuning in evolutionary algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201924 Companion, 1838\u20131845. Association for Computing Machinery, New York, NY, USA 2024. https:\/\/doi.org\/10.1145\/3638530.3664163.","DOI":"10.1145\/3638530.3664163"},{"key":"4281_CR87","doi-asserted-by":"publisher","unstructured":"Jorgensen S, Nadizar G, Pietropolli G, Manzoni L, Medvet E, O\u2019Reilly U-M, Hemberg E. Large language model-based test case generation for GP agents. In: Proceedings of the Genetic and Evolutionary Computation Conference. GECCO \u201924, 914\u2013923. Association for Computing Machinery, New York, NY, USA 2024. https:\/\/doi.org\/10.1145\/3638529.3654056.","DOI":"10.1145\/3638529.3654056"},{"key":"4281_CR88","doi-asserted-by":"publisher","unstructured":"Saletta M, Ferretti C. Exploring the prompt space of large language models through evolutionary sampling. In: Proceedings of the Genetic and Evolutionary Computation Conference. GECCO \u201924, 1345\u20131353. Association for Computing Machinery, New York, NY, USA 2024. https:\/\/doi.org\/10.1145\/3638529.3654049.","DOI":"10.1145\/3638529.3654049"},{"key":"4281_CR89","unstructured":"Chen A, Dohan D, So D. Evoprompting: language models for code-level neural architecture search. Adv Neur Inform Process Syst 2024;36."},{"key":"4281_CR90","unstructured":"Guo Q, Wang R, Guo J, Li B, Song K, Tan X, Liu G, Bian J, Yang Y. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers 2024. https:\/\/arxiv.org\/abs\/2309.08532."},{"key":"4281_CR91","unstructured":"Fernando C, Banarse D, Michalewski H, Osindero S, Rockt\u00e4schel T. Promptbreeder: self-referential self-improvement via prompt evolution 2023. https:\/\/arxiv.org\/abs\/2309.16797"},{"key":"4281_CR92","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2024.103938","volume":"92","author":"N Tao","year":"2025","unstructured":"Tao N, Ventresque A, Nallur V, Saber T. Grammar-obeying program synthesis: a novel approach using large language models and many-objective genetic programming. Comput Stand Interf. 2025;92: 103938.","journal-title":"Comput Stand Interf"},{"key":"4281_CR93","doi-asserted-by":"crossref","unstructured":"Helmuth T, Spector L. General program synthesis benchmark suite. In: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, 2015;1039\u20131046.","DOI":"10.1145\/2739480.2754769"},{"key":"4281_CR94","unstructured":"Grootendorst M. 2020 Keybert: Minimal keyword extraction with bert. Zenodo."},{"issue":"3\u20134","key":"4281_CR95","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1162\/artl_a_00341","volume":"27","author":"T Helmuth","year":"2022","unstructured":"Helmuth T, Spector L. Problem-solving benefits of down-sampled lexicase selection. Artif Life. 2022;27(3\u20134):183\u2013203.","journal-title":"Artif Life"},{"key":"4281_CR96","doi-asserted-by":"crossref","unstructured":"Helmuth T, Spector L. Explaining and exploiting the advantages of down-sampled lexicase selection. In: Artificial Life Conference Proceedings 2020;32, 341\u2013349. MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA.","DOI":"10.1162\/isal_a_00334"},{"key":"4281_CR97","doi-asserted-by":"publisher","unstructured":"Boldi R, Bao A, Briesch M, Helmuth T, Sobania D, Spector L, Lalejini A. A comprehensive analysis of down-sampling for genetic programming-based program synthesis. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201924 Companion, 487\u2013490. Association for Computing Machinery, New York, NY, USA 2024. https:\/\/doi.org\/10.1145\/3638530.3654134.","DOI":"10.1145\/3638530.3654134"},{"key":"4281_CR98","doi-asserted-by":"publisher","unstructured":"Metevier B, Saini AK, Spector L. In: Banzhaf, W., Spector, L., Sheneman, L. (eds.) Lexicase Selection Beyond Genetic Programming, 123\u2013136. Springer, Cham 2019. https:\/\/doi.org\/10.1007\/978-3-030-04735-1_7.","DOI":"10.1007\/978-3-030-04735-1_7"},{"key":"4281_CR99","doi-asserted-by":"publisher","unstructured":"Helmuth T, La Cava, W. Lexicase selection. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201922, 1385\u20131397. Association for Computing Machinery, New York, NY, USA 2022. https:\/\/doi.org\/10.1145\/3520304.3533633.","DOI":"10.1145\/3520304.3533633"},{"key":"4281_CR100","doi-asserted-by":"publisher","unstructured":"Ding L, Boldi R, Helmuth T, Spector L. Lexicase selection at scale. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO \u201922, 2054\u20132062. Association for Computing Machinery, New York, NY, USA 2022. https:\/\/doi.org\/10.1145\/3520304.3534026.","DOI":"10.1145\/3520304.3534026"},{"issue":"1","key":"4281_CR101","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1214\/aoms\/1177730491","volume":"18","author":"HB Mann","year":"1947","unstructured":"Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. 1947;18(1):50\u201360.","journal-title":"Ann Math Stat"},{"issue":"260","key":"4281_CR102","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1080\/01621459.1952.10483441","volume":"47","author":"WH Kruskal","year":"1952","unstructured":"Kruskal WH, Wallis WA. Use of ranks in one-criterion variance analysis. J Am Stat Assoc. 1952;47(260):583\u2013621.","journal-title":"J Am Stat Assoc"},{"key":"4281_CR103","first-page":"65","volume":"1","author":"S Holm","year":"1979","unstructured":"Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;1:65\u201370.","journal-title":"Scand J Stat"}],"container-title":["SN Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-025-04281-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42979-025-04281-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42979-025-04281-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,9]],"date-time":"2025-09-09T19:26:54Z","timestamp":1757446014000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42979-025-04281-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,22]]},"references-count":103,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["4281"],"URL":"https:\/\/doi.org\/10.1007\/s42979-025-04281-x","relation":{},"ISSN":["2661-8907"],"issn-type":[{"value":"2661-8907","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,22]]},"assertion":[{"value":"30 August 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not Applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Research Involving Human and\/or Animals"}},{"value":"Not Applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Informed Consent"}}],"article-number":"760"}}