{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T18:51:03Z","timestamp":1777488663814,"version":"3.51.4"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2023,7,1]],"date-time":"2023-07-01T00:00:00Z","timestamp":1688169600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T00:00:00Z","timestamp":1689292800000},"content-version":"vor","delay-in-days":13,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/P023991\/1"],"award-info":[{"award-number":["EP\/P023991\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/fund","name":"Funder name","doi-asserted-by":"publisher","award":["EP\/J017515\/1"],"award-info":[{"award-number":["EP\/J017515\/1"]}],"id":[{"id":"10.13039\/fund","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"publisher","award":["DE160100850"],"award-info":[{"award-number":["DE160100850"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automated program modification underlies two successful research areas \u2014 genetic improvement and program repair. Under the generate-and-validate strategy, automated program modification transforms a program, then validates the result against a test suite. Much work has focused on the search space of application of single fine-grained operators \u2014 <jats:sc>copy<\/jats:sc>, <jats:sc>delete<\/jats:sc>, <jats:sc>replace<\/jats:sc>, and <jats:sc>swap<\/jats:sc> at both line and statement granularity. This work explores the limits of this strategy. We scale up existing findings an order of magnitude from small corpora to 10 real-world Java programs comprising up to 500k LoC. We decisively show that the grammar-specificity of statement granular edits pays off: its pass rate triples that of line edits and uses 10% less computational resources. We confirm previous findings that <jats:sc>delete<\/jats:sc> is the most effective operator for creating test-suite equivalent program variants. We go farther than prior work by exploring the limits of <jats:sc>delete<\/jats:sc> \u2019s effectiveness by exhaustively applying it. We show this strategy is too costly in practice to be used to search for improved software variants. We further find that pass rates drop from 12\u201334% for single statement edits to 2\u20136% for 5-edit sequences, which implies that further progress will need human-inspired operators that target specific faults or improvements. A program is <jats:italic>amenable to automated modification<\/jats:italic> to the extent to which automatically editing it is likely to produce test-suite passing variants. We are the first to systematically search for a code measure that correlates with a program\u2019s amenability to automated modification. We found no strong correlations, leaving the question open.<\/jats:p>","DOI":"10.1007\/s10664-023-10344-5","type":"journal-article","created":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T02:01:31Z","timestamp":1689300091000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Program transformation landscapes for automated program modification using Gin"],"prefix":"10.1007","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7833-6044","authenticated-orcid":false,"given":"Justyna","family":"Petke","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brad","family":"Alexander","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Earl T.","family":"Barr","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander E. I.","family":"Brownlee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Wagner","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David R.","family":"White","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,7,14]]},"reference":[{"key":"10344_CR1","doi-asserted-by":"crossref","unstructured":"Ahmad H, Cashin P, Forrest S, Weimer W (2022) Digging into semantics: Where do search-based software repair methods search? URL https:\/\/web.eecs.umich.edu\/~weimerw\/p\/weimer-ppsn2022.pdf","DOI":"10.1007\/978-3-031-14721-0_1"},{"key":"10344_CR2","doi-asserted-by":"publisher","unstructured":"An G, Kim J, Yoo S (2018) Comparing line and AST granularity level for program repair using pyggi. In: Petke J, Stolee KT, Langdon WB, Weimer W (eds) Proceedings of the 4th International Genetic Improvement Workshop, GI@ICSE 2018, Gothenburg, Sweden, June 2, 2018, ACM, pp 19\u201326. https:\/\/doi.org\/10.1145\/3194810.3194814","DOI":"10.1145\/3194810.3194814"},{"key":"10344_CR3","doi-asserted-by":"publisher","unstructured":"Barr ET, Brun Y, Devanbu PT, Harman M, Sarro F (2014) The plastic surgery hypothesis. In: Cheung S, Orso A, Storey MD (eds) Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16 - 22, 2014, ACM, pp 306\u2013317. https:\/\/doi.org\/10.1145\/2635868.2635898","DOI":"10.1145\/2635868.2635898"},{"key":"10344_CR4","doi-asserted-by":"publisher","unstructured":"Bian Z, Blot A, Petke J (2021) Refining fitness functions for search-based program repair. In: 2nd IEEE\/ACM International Workshop on Automated Program Repair, APR@ICSE 2021, Madrid, Spain, June 1, 2021, IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/APR52552.2021.00008","DOI":"10.1109\/APR52552.2021.00008"},{"issue":"5","key":"10344_CR5","doi-asserted-by":"publisher","first-page":"3077","DOI":"10.1007\/s10664-018-9675-9","volume":"24","author":"DW Binkley","year":"2019","unstructured":"Binkley DW, Gold N, Islam SS, Krinke J, Yoo S (2019) A comparison of tree- and line-oriented observational slicing. Empir Softw Eng 24(5):3077\u20133113. https:\/\/doi.org\/10.1007\/s10664-018-9675-9","journal-title":"Empir Softw Eng"},{"issue":"5","key":"10344_CR6","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.1109\/TEVC.2021.3070271","volume":"25","author":"A Blot","year":"2021","unstructured":"Blot A, Petke J (2021) Empirical comparison of search heuristics for genetic improvement of software. IEEE Trans Evol Comput 25(5):1001\u20131011. https:\/\/doi.org\/10.1109\/TEVC.2021.3070271","journal-title":"IEEE Trans Evol Comput"},{"key":"10344_CR7","doi-asserted-by":"publisher","unstructured":"Brownlee AEI, Petke J, Alexander B, Barr ET, Wagner M, White DR (2019) Gin: genetic improvement research made easy. In: Auger A, St\u00fctzle T (eds) Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, Prague, Czech Republic, July 13-17, 2019, ACM, pp 985\u2013993. https:\/\/doi.org\/10.1145\/3321707.3321841","DOI":"10.1145\/3321707.3321841"},{"key":"10344_CR8","doi-asserted-by":"publisher","unstructured":"Brownlee AEI, Petke J, Rasburn AF (2020) Injecting shortcuts for faster running java code. In: IEEE Congress on Evolutionary Computation, CEC 2020, Glasgow, United Kingdom, July 19-24, 2020, IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/CEC48606.2020.9185708","DOI":"10.1109\/CEC48606.2020.9185708"},{"issue":"5","key":"10344_CR9","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1007\/s10664-022-10137-2","volume":"27","author":"J Callan","year":"2022","unstructured":"Callan J, Krauss O, Petke J, Sarro F (2022) How do android developers improve non-functional properties of software? Empir Softw Eng 27(5):113. https:\/\/doi.org\/10.1007\/s10664-022-10137-2","journal-title":"Empir Softw Eng"},{"issue":"3","key":"10344_CR10","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.1007\/s10664-016-9443-7","volume":"22","author":"R Coelho","year":"2017","unstructured":"Coelho R, Almeida L, Gousios G, van Deursen A, Treude C (2017) Exception handling bug hazards in android - results from a mining study and an exploratory survey. Empir Softw Eng 22(3):1264\u20131304. https:\/\/doi.org\/10.1007\/s10664-016-9443-7","journal-title":"Empir Softw Eng"},{"key":"10344_CR11","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"J Cohen","year":"1969","unstructured":"Cohen J (1969) Statistical Power Analysis for the Behavioral Sciences. Academic Press, NY"},{"key":"10344_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111263","volume":"188","author":"K Etemadi","year":"2022","unstructured":"Etemadi K, Tarighat N, Yadav S, Martinez M, Monperrus M (2022) Estimating the potential of program repair search spaces with commit analysis. J Syst Softw 188:111263. https:\/\/doi.org\/10.1016\/j.jss.2022.111263","journal-title":"J Syst Softw"},{"issue":"1","key":"10344_CR13","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/TSE.2017.2755013","volume":"45","author":"L Gazzola","year":"2019","unstructured":"Gazzola L, Micucci D, Mariani L (2019) Automatic software repair: A survey. IEEE Trans Software Eng 45(1):34\u201367. https:\/\/doi.org\/10.1109\/TSE.2017.2755013","journal-title":"IEEE Trans Software Eng"},{"issue":"4","key":"10344_CR14","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.2307\/797245","volume":"105","author":"P Gewirtz","year":"1996","unstructured":"Gewirtz P (1996) On \u201cI know it when I see it\u2019\u2019. Yale Law J 105(4):1023\u20131047. https:\/\/doi.org\/10.2307\/797245","journal-title":"Yale Law J"},{"key":"10344_CR15","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10100-7","author":"D Ginelli","year":"2022","unstructured":"Ginelli D, Martinez M, Mariani L, Monperrus M (2022). A comprehensive study of code-removal patches in automated program repair. https:\/\/doi.org\/10.1007\/s10664-021-10100-7","journal-title":"A comprehensive study of code-removal patches in automated program repair."},{"key":"10344_CR16","doi-asserted-by":"publisher","unstructured":"Guizzo G, Blot A, Callan J, Petke J, Sarro F (2021) Refining fitness functions for search-based automated program repair - A case study with ARJA and arja-e. In: O\u2019Reilly U, Devroey X (eds) Search-Based Software Engineering - 13th International Symposium, SSBSE 2021, Bari, Italy, October 11-12, 2021, Proceedings, Springer, Lecture Notes in Computer Science, vol 12914, pp 159\u2013165. https:\/\/doi.org\/10.1007\/978-3-030-88106-1_12","DOI":"10.1007\/978-3-030-88106-1_12"},{"key":"10344_CR17","doi-asserted-by":"publisher","unstructured":"Gulwani S (2010) Dimensions in program synthesis. In: Kutsia T, Schreiner W, Fern\u00e1ndez M (eds) Proceedings of the 12th International ACM SIGPLAN Conference on Principles and Practice of Declarative Programming, July 26-28, 2010, Hagenberg, Austria, ACM, pp 13\u201324. https:\/\/doi.org\/10.1145\/1836089.1836091","DOI":"10.1145\/1836089.1836091"},{"key":"10344_CR18","doi-asserted-by":"publisher","unstructured":"Haraldsson SO, Woodward JR, Brownlee AEI, Siggeirsdottir K (2017) Fixing bugs in your sleep: how genetic improvement became an overnight success. In: Bosman PAN (ed) Genetic and Evolutionary Computation Conference, Berlin, Germany, July 15-19, 2017, Companion Material Proceedings, ACM, pp 1513\u20131520. https:\/\/doi.org\/10.1145\/3067695.3082517","DOI":"10.1145\/3067695.3082517"},{"issue":"4","key":"10344_CR19","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1007\/s10710-019-09355-3","volume":"20","author":"N Harrand","year":"2019","unstructured":"Harrand N, Allier S, Rodriguez-Cancio M, Monperrus M, Baudry B (2019) A journey among Java neutral program variants. Genet Program Evolvable Mach 20(4):531\u2013580. https:\/\/doi.org\/10.1007\/s10710-019-09355-3","journal-title":"Genet Program Evolvable Mach"},{"key":"10344_CR20","doi-asserted-by":"publisher","unstructured":"Hassan F, Bansal C, Nagappan N, Zimmermann T, Awadallah AH (2020) An empirical study of software exceptions in the field using search logs. In: Baldassarre MT, Lanubile F, Kalinowski M, Sarro F (eds) ESEM \u201920: ACM \/ IEEE International Symposium on Empirical Software Engineering and Measurement, Bari, Italy, October 5-7, 2020, ACM, pp 4:1\u20134:12. https:\/\/doi.org\/10.1145\/3382494.3410692","DOI":"10.1145\/3382494.3410692"},{"issue":"4","key":"10344_CR21","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/MS.2021.3071086","volume":"38","author":"S Kirbas","year":"2021","unstructured":"Kirbas S, Windels E, McBello O, Kells K, Pagano MW, Szalanski R, Nowack V, Winter ER, Counsell S, Bowes D, Hall T, Haraldsson S, Woodward JR (2021) On the introduction of automatic program repair in bloomberg. IEEE Softw 38(4):43\u201351. https:\/\/doi.org\/10.1109\/MS.2021.3071086","journal-title":"IEEE Softw"},{"issue":"3","key":"10344_CR22","doi-asserted-by":"publisher","first-page":"1980","DOI":"10.1007\/s10664-019-09780-z","volume":"25","author":"A Koyuncu","year":"2020","unstructured":"Koyuncu A, Liu K, Bissyand\u00e9 TF, Kim D, Klein J, Monperrus M, Traon YL (2020) Fixminer: Mining relevant fix patterns for automated program repair. Empir Softw Eng 25(3):1980\u20132024. https:\/\/doi.org\/10.1007\/s10664-019-09780-z","journal-title":"Empir Softw Eng"},{"key":"10344_CR23","doi-asserted-by":"publisher","unstructured":"Langdon WB, Harman M (2010) Evolving a CUDA kernel from an nvidia template. In: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2010, Barcelona, Spain, 18-23 July 2010, IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/CEC.2010.5585922","DOI":"10.1109\/CEC.2010.5585922"},{"key":"10344_CR24","doi-asserted-by":"publisher","unstructured":"Langdon WB, Lam BYH (2017) Genetically improved barracuda. BioData Min 10(1):28:1\u201328:11. https:\/\/doi.org\/10.1186\/s13040-017-0149-1","DOI":"10.1186\/s13040-017-0149-1"},{"key":"10344_CR25","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1007\/978-3-319-45901-1_24","volume-title":"First Complex Systems Digital Campus World E-Conference 2015","author":"WB Langdon","year":"2017","unstructured":"Langdon WB, Petke J (2017) Software is not fragile. In: Bourgine P, Collet P, Parrend P (eds) First Complex Systems Digital Campus World E-Conference 2015. Springer, Cham, pp 203\u2013211"},{"key":"10344_CR26","doi-asserted-by":"publisher","unstructured":"Langdon WB, Lam BYH, Petke J, Harman M (2015) Improving CUDA DNA analysis software with genetic programming. In: Silva S, Esparcia-Alc\u00e1zar AI (eds) Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2015, Madrid, Spain, July 11-15, 2015, ACM, pp 1063\u20131070. https:\/\/doi.org\/10.1145\/2739480.2754652","DOI":"10.1145\/2739480.2754652"},{"key":"10344_CR27","doi-asserted-by":"publisher","unstructured":"Langdon WB, Veerapen N, Ochoa G (2017) Visualising the search landscape of the triangle program. In: McDermott J, Castelli M, Sekanina L, Haasdijk E, Garc\u00eda-S\u00e1nchez P (eds) Genetic Programming - 20th European Conference, EuroGP 2017, Amsterdam, The Netherlands, April 19-21, 2017, Proceedings, Lecture Notes in Computer Science, vol 10196, pp 96\u2013113. https:\/\/doi.org\/10.1007\/978-3-319-55696-3_7","DOI":"10.1007\/978-3-319-55696-3_7"},{"key":"10344_CR28","doi-asserted-by":"publisher","unstructured":"Le Goues C, Weimer W, Forrest S (2012) Representations and operators for improving evolutionary software repair. In: Soule T, Moore JH (eds) Genetic and Evolutionary Computation Conference, GECCO \u201912, Philadelphia, PA, USA, July 7-11, 2012, ACM, pp 959\u2013966. https:\/\/doi.org\/10.1145\/2330163.2330296","DOI":"10.1145\/2330163.2330296"},{"key":"10344_CR29","doi-asserted-by":"publisher","unstructured":"Li J, He P, Zhu J, Lyu MR (2017) Software defect prediction via convolutional neural network. In: 2017 IEEE International Conference on Software Quality, Reliability and Security, QRS 2017, Prague, Czech Republic, July 25-29, 2017, IEEE, pp 318\u2013328. https:\/\/doi.org\/10.1109\/QRS.2017.42","DOI":"10.1109\/QRS.2017.42"},{"key":"10344_CR30","doi-asserted-by":"publisher","unstructured":"Liu K, Koyuncu A, Kim D, Bissyand\u00e9 TF (2019) Tbar: revisiting template-based automated program repair. In: Zhang D, M\u00f8ller A (eds) Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2019, Beijing, China, July 15-19, 2019, ACM, pp 31\u201342. https:\/\/doi.org\/10.1145\/3293882.3330577","DOI":"10.1145\/3293882.3330577"},{"key":"10344_CR31","doi-asserted-by":"publisher","unstructured":"Marginean A, Bader J, Chandra S, Harman M, Jia Y, Mao K, Mols A, Scott A (2019) Sapfix: automated end-to-end repair at scale. In: Sharp H, Whalen M (eds) Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP) 2019, Montreal, QC, Canada, May 25-31, 2019, IEEE \/ ACM, pp 269\u2013278. https:\/\/doi.org\/10.1109\/ICSE-SEIP.2019.00039","DOI":"10.1109\/ICSE-SEIP.2019.00039"},{"issue":"1","key":"10344_CR32","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1007\/s10664-013-9282-8","volume":"20","author":"M Martinez","year":"2015","unstructured":"Martinez M, Monperrus M (2015) Mining software repair models for reasoning on the search space of automated program fixing. Empir Softw Eng 20(1):176\u2013205. https:\/\/doi.org\/10.1007\/s10664-013-9282-8","journal-title":"Empir Softw Eng"},{"key":"10344_CR33","doi-asserted-by":"publisher","unstructured":"Martinez M, Monperrus M (2018) Ultra-large repair search space with automatically mined templates: The cardumen mode of astor. In: Colanzi TE, McMinn P (eds) Search-Based Software Engineering - 10th International Symposium, SSBSE 2018, Montpellier, France, September 8-9, 2018, Proceedings, Springer, Lecture Notes in Computer Science, vol 11036, pp 65\u201386. https:\/\/doi.org\/10.1007\/978-3-319-99241-9_3","DOI":"10.1007\/978-3-319-99241-9_3"},{"key":"10344_CR34","doi-asserted-by":"publisher","unstructured":"Mechtaev S, Yi J, Roychoudhury A (2016) Angelix: scalable multiline program patch synthesis via symbolic analysis. In: Dillon LK, Visser W, Williams LA (eds) Proceedings of the 38th International Conference on Software Engineering, ICSE 2016, Austin, TX, USA, May 14-22, 2016, ACM, pp 691\u2013701. https:\/\/doi.org\/10.1145\/2884781.2884807","DOI":"10.1145\/2884781.2884807"},{"key":"10344_CR35","doi-asserted-by":"publisher","unstructured":"Meyvis T, Yoon H (2021) Adding is favoured over subtracting in problem solving. nature 592. https:\/\/doi.org\/10.1038\/d41586-021-00592-0","DOI":"10.1038\/d41586-021-00592-0"},{"key":"10344_CR36","doi-asserted-by":"publisher","unstructured":"Monperrus M (2014) A critical review of \"automatic patch generation learned from human-written patches\": essay on the problem statement and the evaluation of automatic software repair. In: Jalote P, Briand LC, van\u00a0der Hoek A (eds) 36th International Conference on Software Engineering, ICSE \u201914, Hyderabad, India - May 31 - June 07, 2014, ACM, pp 234\u2013242. https:\/\/doi.org\/10.1145\/2568225.2568324","DOI":"10.1145\/2568225.2568324"},{"issue":"2","key":"10344_CR37","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1145\/42372.42379","volume":"31","author":"BA Nejmeh","year":"1988","unstructured":"Nejmeh BA (1988) Npath: A measure of execution path complexity and its applications. Commun ACM 31(2):188\u2013200. https:\/\/doi.org\/10.1145\/42372.42379","journal-title":"Commun ACM"},{"key":"10344_CR38","doi-asserted-by":"publisher","unstructured":"Ochoa G, Tomassini M, V\u00e9rel S, Darabos C (2008) A study of NK landscapes\u2019 basins and local optima networks. In: Ryan C, Keijzer M (eds) Genetic and Evolutionary Computation Conference, GECCO 2008, Proceedings, Atlanta, GA, USA, July 12-16, 2008, ACM, pp 555\u2013562. https:\/\/doi.org\/10.1145\/1389095.1389204","DOI":"10.1145\/1389095.1389204"},{"key":"10344_CR39","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.114595","volume":"172","author":"SK Pandey","year":"2021","unstructured":"Pandey SK, Mishra RB, Tripathi AK (2021) Machine learning based methods for software fault prediction: A survey. Expert Syst Appl 172:114595. https:\/\/doi.org\/10.1016\/j.eswa.2021.114595","journal-title":"Expert Syst Appl"},{"key":"10344_CR40","doi-asserted-by":"publisher","unstructured":"Petke J (2017) New operators for non-functional genetic improvement. In: Bosman PAN (ed) Genetic and Evolutionary Computation Conference, Berlin, Germany, July 15-19, 2017, Companion Material Proceedings, ACM, pp 1541\u20131542. https:\/\/doi.org\/10.1145\/3067695.3082520","DOI":"10.1145\/3067695.3082520"},{"issue":"3","key":"10344_CR41","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1109\/TEVC.2017.2693219","volume":"22","author":"J Petke","year":"2018","unstructured":"Petke J, Haraldsson SO, Harman M, Langdon WB, White DR, Woodward JR (2018) Genetic improvement of software: A comprehensive survey. IEEE Trans Evol Comput 22(3):415\u2013432. https:\/\/doi.org\/10.1109\/TEVC.2017.2693219","journal-title":"IEEE Trans Evol Comput"},{"key":"10344_CR42","doi-asserted-by":"publisher","unstructured":"Petke J, Alexander B, Barr ET, Brownlee AEI, Wagner M, White DR (2019) A survey of genetic improvement search spaces. In: L\u00f3pez-Ib\u00e1\u00f1ez M, Auger A, St\u00fctzle T (eds) Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2019, Prague, Czech Republic, July 13-17, 2019, ACM, pp 1715\u20131721. https:\/\/doi.org\/10.1145\/3319619.3326870","DOI":"10.1145\/3319619.3326870"},{"key":"10344_CR43","doi-asserted-by":"publisher","unstructured":"Qi Z, Long F, Achour S, Rinard MC (2015) An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In: Young M, Xie T (eds) Proceedings of the 2015 International Symposium on Software Testing and Analysis, ISSTA 2015, Baltimore, MD, USA, July 12-17, 2015, ACM, pp 24\u201336. https:\/\/doi.org\/10.1145\/2771783.2771791","DOI":"10.1145\/2771783.2771791"},{"key":"10344_CR44","doi-asserted-by":"publisher","unstructured":"Reeves CR (1999) Fitness landscapes and evolutionary algorithms. In: Fonlupt C, Hao J, Lutton E, Ronald EMA, Schoenauer M (eds) Artificial Evolution, 4th European Conference, AE\u201999, Dunkerque, France, November 3-5, 1999, Selected Papers, Springer, Lecture Notes in Computer Science, vol 1829, pp 3\u201320. https:\/\/doi.org\/10.1007\/10721187_1","DOI":"10.1007\/10721187_1"},{"issue":"1","key":"10344_CR45","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1137\/S0036144501395952","volume":"44","author":"CM Reidys","year":"2002","unstructured":"Reidys CM, Stadler PF (2002) Combinatorial landscapes. SIAM Rev 44(1):3\u201354. https:\/\/doi.org\/10.1137\/S0036144501395952","journal-title":"SIAM Rev"},{"key":"10344_CR46","doi-asserted-by":"publisher","unstructured":"Renzullo J, Weimer W, Moses ME, Forrest S (2018) Neutrality and epistasis in program space. In: Petke J, Stolee KT, Langdon WB, Weimer W (eds) Proceedings of the 4th International Genetic Improvement Workshop, GI@ICSE 2018, Gothenburg, Sweden, June 2, 2018, ACM, pp 1\u20138. https:\/\/doi.org\/10.1145\/3194810.3194812","DOI":"10.1145\/3194810.3194812"},{"key":"10344_CR47","doi-asserted-by":"publisher","unstructured":"Saha S, Saha RK, Prasad MR (2019) Harnessing evolution for multi-hunk program repair. In: Atlee JM, Bultan T, Whittle J (eds) Proceedings of the 41st International Conference on Software Engineering, ICSE 2019, Montreal, QC, Canada, May 25-31, 2019, IEEE \/ ACM, pp 13\u201324. https:\/\/doi.org\/10.1109\/ICSE.2019.00020","DOI":"10.1109\/ICSE.2019.00020"},{"issue":"3","key":"10344_CR48","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1007\/s10710-013-9195-8","volume":"15","author":"EM Schulte","year":"2014","unstructured":"Schulte EM, Fry ZP, Fast E, Weimer W, Forrest S (2014) Software mutational robustness. Genet Program Evolvable Mach 15(3):281\u2013312. https:\/\/doi.org\/10.1007\/s10710-013-9195-8","journal-title":"Genet Program Evolvable Mach"},{"key":"10344_CR49","doi-asserted-by":"publisher","unstructured":"Smigielska M, Blot A, Petke J (2021) Uniform edit selection for genetic improvement: Empirical analysis of mutation operator efficacy. In: 10th IEEE\/ACM International Workshop on Genetic Improvement, GI@ICSE 2021, Madrid, Spain, May 30, 2021, IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/GI52543.2021.00009","DOI":"10.1109\/GI52543.2021.00009"},{"key":"10344_CR50","doi-asserted-by":"publisher","unstructured":"Soto M, Goues CL (2018) Using a probabilistic model to predict bug fixes. In: Oliveto R, Penta MD, Shepherd DC (eds) 25th International Conference on Software Analysis, Evolution and Reengineering, SANER 2018, Campobasso, Italy, March 20-23, 2018, IEEE Computer Society, pp 221\u2013231. https:\/\/doi.org\/10.1109\/SANER.2018.8330211","DOI":"10.1109\/SANER.2018.8330211"},{"key":"10344_CR51","doi-asserted-by":"publisher","unstructured":"Wen M, Chen J, Wu R, Hao D, Cheung S (2018) Context-aware patch generation for better automated program repair. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th International Conference on Software Engineering, ICSE 2018, Gothenburg, Sweden, May 27 - June 03, 2018, ACM, pp 1\u201311. https:\/\/doi.org\/10.1145\/3180155.3180233","DOI":"10.1145\/3180155.3180233"},{"key":"10344_CR52","doi-asserted-by":"publisher","unstructured":"Wong C, Meinicke J, Lazarek L, K\u00e4stner C (2018) Faster variational execution with transparent bytecode transformation. Proc ACM Program Lang 2(OOPSLA):117:1\u2013117:30. https:\/\/doi.org\/10.1145\/3276487","DOI":"10.1145\/3276487"},{"key":"10344_CR53","doi-asserted-by":"publisher","unstructured":"Wong C, Santiesteban P, K\u00e4stner C, Goues CL (2021) Varfix: balancing edit expressiveness and search effectiveness in automated program repair. In: Spinellis D, Gousios G, Chechik M, Penta MD (eds) ESEC\/FSE \u201921: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Athens, Greece, August 23-28, 2021, ACM, pp 354\u2013366. https:\/\/doi.org\/10.1145\/3468264.3468600","DOI":"10.1145\/3468264.3468600"},{"key":"10344_CR54","doi-asserted-by":"publisher","unstructured":"Xin Q, Reiss SP (2017) Identifying test-suite-overfitted patches through test case generation. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Association for Computing Machinery, New York, NY, USA, ISSTA 2017, p 226\u2013236. https:\/\/doi.org\/10.1145\/3092703.3092718","DOI":"10.1145\/3092703.3092718"},{"key":"10344_CR55","doi-asserted-by":"publisher","unstructured":"Xiong Y, Liu X, Zeng M, Zhang L, Huang G (2018) Identifying patch correctness in test-based program repair. In: Proceedings of the 40th International Conference on Software Engineering, Association for Computing Machinery, New York, NY, USA, ICSE \u201918, p 789\u2013799. https:\/\/doi.org\/10.1145\/3180155.3180182","DOI":"10.1145\/3180155.3180182"},{"key":"10344_CR56","doi-asserted-by":"publisher","unstructured":"Yang J, Zhikhartsev A, Liu Y, Tan L (2017) Better test cases for better automated program repair. In: Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering, Association for Computing Machinery, New York, NY, USA, ESEC\/FSE 2017, p 831\u2013841. https:\/\/doi.org\/10.1145\/3106237.3106274","DOI":"10.1145\/3106237.3106274"},{"issue":"2","key":"10344_CR57","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1007\/s10664-020-09920-w","volume":"26","author":"H Ye","year":"2021","unstructured":"Ye H, Martinez M, Monperrus M (2021) Automated patch assessment for program repair at scale. Empir Softw Eng 26(2):20. https:\/\/doi.org\/10.1007\/s10664-020-09920-w","journal-title":"Empir Softw Eng"},{"issue":"10","key":"10344_CR58","doi-asserted-by":"publisher","first-page":"1040","DOI":"10.1109\/TSE.2018.2874648","volume":"46","author":"Y Yuan","year":"2020","unstructured":"Yuan Y, Banzhaf W (2020) ARJA: automated repair of java programs via multi-objective genetic programming. IEEE Trans Software Eng 46(10):1040\u20131067. https:\/\/doi.org\/10.1109\/TSE.2018.2874648","journal-title":"IEEE Trans Software Eng"},{"key":"10344_CR59","doi-asserted-by":"publisher","unstructured":"Zhong H, Su Z (2015) An empirical study on real bug fixes. In: Bertolino A, Canfora G, Elbaum SG (eds) 37th IEEE\/ACM International Conference on Software Engineering, ICSE 2015, Florence, Italy, May 16-24, 2015, Volume 1, IEEE Computer Society, pp 913\u2013923. https:\/\/doi.org\/10.1109\/ICSE.2015.101","DOI":"10.1109\/ICSE.2015.101"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10344-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10344-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10344-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T09:22:56Z","timestamp":1695806576000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10344-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7]]},"references-count":59,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["10344"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10344-5","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7]]},"assertion":[{"value":"23 May 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 July 2023","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"104"}}