{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,20]],"date-time":"2026-01-20T00:43:10Z","timestamp":1768869790394,"version":"3.49.0"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,1,31]],"date-time":"2017-01-31T00:00:00Z","timestamp":1485820800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61572375"],"award-info":[{"award-number":["61572375"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Korean government","award":["2016R1C1B1011"],"award-info":[{"award-number":["2016R1C1B1011"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2017,1,31]]},"abstract":"<jats:p>We report on the application of Genetic Programming to Software Fault Localisation, a problem in the area of Search-Based Software Engineering (SBSE). We give both empirical and theoretical evidence for the human competitiveness of the evolved fault localisation formul\u00e6 under the single fault scenario, compared to those generated by human ingenuity and reported in many papers, published over more than a decade. Though there have been previous human competitive results claimed for SBSE problems, this is the first time that evolved solutions have been formally proved to be human competitive. We further prove that no future human investigation could outperform the evolved solutions. We complement these proofs with an empirical analysis of both human and evolved solutions, which indicates that the evolved solutions are not only theoretically human competitive, but also convey similar practical benefits to human-evolved counterparts.<\/jats:p>","DOI":"10.1145\/3078840","type":"journal-article","created":{"date-parts":[[2017,6,30]],"date-time":"2017-06-30T12:36:19Z","timestamp":1498826179000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":69,"title":["Human Competitiveness of Genetic Programming in Spectrum-Based Fault Localisation"],"prefix":"10.1145","volume":"26","author":[{"given":"Shin","family":"Yoo","sequence":"first","affiliation":[{"name":"Korea Advanced Institute of Science and Technology, Republic of Korea"}]},{"given":"Xiaoyuan","family":"Xie","sequence":"additional","affiliation":[{"name":"Wuhan University, Hubei, China"}]},{"given":"Fei-Ching","family":"Kuo","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Hawthorn, VIC, Austrailia"}]},{"given":"Tsong Yueh","family":"Chen","sequence":"additional","affiliation":[{"name":"Swinburne University of Technology, Hawthorn, VIC, Austrailia"}]},{"given":"Mark","family":"Harman","sequence":"additional","affiliation":[{"name":"University College London, UK"}]}],"member":"320","published-online":{"date-parts":[[2017,6,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2009.25"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC.2006.18"},{"key":"e_1_2_1_3_1","volume-title":"Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques\u2014MUTATION. IEEE Computer Society, 89--98","author":"Abreu Rui","unstructured":"Rui Abreu , Peter Zoeteweij , and Arjan J. C . van Gemund. 2007. On the accuracy of spectrum-based fault localization . In Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques\u2014MUTATION. IEEE Computer Society, 89--98 . Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Proceedings of the Testing: Academic and Industrial Conference Practice and Research Techniques\u2014MUTATION. IEEE Computer Society, 89--98."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2008.12.005"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985795"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1831708.1831715"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771796"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2768829"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the Conference of the Centre for Advanced Studies on Collaborative research (CASCON\u201902)","author":"Chen Yanping","unstructured":"Yanping Chen , Robert L. Probert , and D. Paul Sims . 2002. Specification-based regression test selection with risk analysis . In Proceedings of the Conference of the Centre for Advanced Studies on Collaborative research (CASCON\u201902) . IBM Press, 1--14. Yanping Chen, Robert L. Probert, and D. Paul Sims. 2002. Specification-based regression test selection with risk analysis. In Proceedings of the Conference of the Centre for Advanced Studies on Collaborative research (CASCON\u201902). IBM Press, 1--14."},{"key":"e_1_2_1_10_1","volume-title":"Foreman","author":"Corder Gregory W.","year":"2009","unstructured":"Gregory W. Corder and Dale I . Foreman . 2009 . Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. Wiley . Gregory W. Corder and Dale I. Foreman. 2009. Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. Wiley."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1085130.1085143"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001420.2001446"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-005-3861-2"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1569901.1570031"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483774"},{"key":"e_1_2_1_16_1","series-title":"Lecture Notes in Computer Science","volume-title":"Search-Based Software Engineering, MyraB. Cohen and Mel \u00d3 Cinn\u00e9ide (Eds.)","author":"Freitas Fabr\u00edcio Gomes","unstructured":"Fabr\u00edcio Gomes Freitas and Jerffeson Teixeira Souza . 2011. Ten years of search-based software engineering: A bibliometric analysis . In Search-Based Software Engineering, MyraB. Cohen and Mel \u00d3 Cinn\u00e9ide (Eds.) . Lecture Notes in Computer Science , Vol. 6956 . Springer , Berlin , 18--32. Fabr\u00edcio Gomes Freitas and Jerffeson Teixeira Souza. 2011. Ten years of search-based software engineering: A bibliometric analysis. In Search-Based Software Engineering, MyraB. Cohen and Mel \u00d3 Cinn\u00e9ide (Eds.). Lecture Notes in Computer Science, Vol. 6956. Springer, Berlin, 18--32."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2011.6100153"},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI\u201911)","author":"Gonzalez-Sanchez Alberto","unstructured":"Alberto Gonzalez-Sanchez , Rui Abreu , Hans-Gerhard Gross , and Arjan J. C . van Gemund. 2011a. Spectrum-based sequential diagnosis . In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI\u201911) . AAAI Press, 189--196. Alberto Gonzalez-Sanchez, Rui Abreu, Hans-Gerhard Gross, and Arjan J. C. van Gemund. 2011a. Spectrum-based sequential diagnosis. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI\u201911). AAAI Press, 189--196."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2010.28"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 1st IEEE Working Conference on Software Visualization (VISSOFT\u201913)","author":"Gouveia C.","unstructured":"C. Gouveia , J. Campos , and R. Abreu . 2013. Using HTML5 visualizations in software fault localization . In Proceedings of the 1st IEEE Working Conference on Software Visualization (VISSOFT\u201913) . 1--10. C. Gouveia, J. Campos, and R. Abreu. 2013. Using HTML5 visualizations in software fault localization. In Proceedings of the 1st IEEE Working Conference on Software Visualization (VISSOFT\u201913). 1--10."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-008-0025-9"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2011.263"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379787"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/277631.277647"},{"key":"e_1_2_1_25_1","volume-title":"\u00c9tude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 37","author":"Jaccard Paul","year":"1901","unstructured":"Paul Jaccard . 1901. \u00c9tude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 37 ( 1901 ), 547--579. Paul Jaccard. 1901. \u00c9tude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin del la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 37 (1901), 547--579."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337279"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483763"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101908.1101949"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/581339.581397"},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of ICSE Workshop on Software Visualization. 71--75","author":"Jones James A.","unstructured":"James A. Jones , Mary Jean Harrold , and John T. Stasko . 2001. Visualization for fault localization . In Proceedings of ICSE Workshop on Software Visualization. 71--75 . James A. Jones, Mary Jean Harrold, and John T. Stasko. 2001. Visualization for fault localization. In Proceedings of ICSE Workshop on Software Visualization. 71--75."},{"key":"e_1_2_1_31_1","volume-title":"Genetic Programming: On the Programming of Computers by Means of Natural Selection","author":"Koza J. R.","year":"1992","unstructured":"J. R. Koza . 1992 . Genetic Programming: On the Programming of Computers by Means of Natural Selection . MIT Press, Cambridge , MA. J. R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-013-9208-0"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2010.22"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.v14:2"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2014.28"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2000791.2000795"},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the 35th Australasian Computer Science Conference\u2014Volume 122 (ACSC\u201912)","author":"Naish Lee","year":"2012","unstructured":"Lee Naish , Hua Jie Lee , and Kotagiri Ramamohanarao . 2012 . Spectral debugging: How much better can we do? In Proceedings of the 35th Australasian Computer Science Conference\u2014Volume 122 (ACSC\u201912) . Australian Computer Society, Inc., Darlinghurst, Australia, 99--106. Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2012. Spectral debugging: How much better can we do? In Proceedings of the 35th Australasian Computer Science Conference\u2014Volume 122 (ACSC\u201912). Australian Computer Society, Inc., Darlinghurst, Australia, 99--106."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.2331\/suisan.22.526"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2486788.2486857"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806799.1806838"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001420.2001445"},{"key":"e_1_2_1_43_1","unstructured":"Riccardo Poli William B. Langdon and Nicholas Freitag McPhee. 2008. A Field Guide to Genetic Programming. Published via http:\/\/lulu.com and retrieved from http:\/\/www.gp-field-guide.org.uk (with contributions by J. R. Koza).   Riccardo Poli William B. Langdon and Nicholas Freitag McPhee. 2008. A Field Guide to Genetic Programming. Published via http:\/\/lulu.com and retrieved from http:\/\/www.gp-field-guide.org.uk (with contributions by J. R. Koza)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483785"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/3106091.3106097"},{"key":"e_1_2_1_46_1","first-page":"153","article-title":"On habitat and association of species of anopheline larvae in south-eastern Madras","volume":"3","author":"Russel P. F.","year":"1940","unstructured":"P. F. Russel and T. Ramachandra Rao . 1940 . On habitat and association of species of anopheline larvae in south-eastern Madras . J. Malar. Inst. India 3 , 1 (1940), 153 -- 178 . P. F. Russel and T. Ramachandra Rao. 1940. On habitat and association of species of anopheline larvae in south-eastern Madras. J. Malar. Inst. India 3, 1 (1940), 153--178.","journal-title":"J. Malar. Inst. India"},{"key":"e_1_2_1_47_1","unstructured":"SLOCCount. 2004.\n   Retrieved from http:\/\/www.dwheeler.com\/sloccount\/sloccount.html (\n  2004\n  ).  SLOCCount. 2004. Retrieved from http:\/\/www.dwheeler.com\/sloccount\/sloccount.html (2004)."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483767"},{"key":"e_1_2_1_49_1","first-page":"101","article-title":"A critique and improvement of the \u201cCL\u201d common language effect size statistics of McGraw and Wong","volume":"25","author":"Vargha Andr\u00e1s","year":"2000","unstructured":"Andr\u00e1s Vargha and Harold D. Delaney . 2000 . A critique and improvement of the \u201cCL\u201d common language effect size statistics of McGraw and Wong . J. Educat. Behav. Stat. 25 , 2 (2000), pp. 101 -- 132 . Andr\u00e1s Vargha and Harold D. Delaney. 2000. A critique and improvement of the \u201cCL\u201d common language effect size statistics of McGraw and Wong. J. Educat. Behav. Stat. 25, 2 (2000), pp. 101--132.","journal-title":"J. Educat. Behav. Stat."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070536"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2007.109"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2522920.2522924"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39742-4_17"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2011.32"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33119-0_18"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491509.2491513"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1368088.1368116"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3078840","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3078840","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:03:08Z","timestamp":1750215788000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3078840"}},"subtitle":["Theoretical and Empirical Analysis"],"short-title":[],"issued":{"date-parts":[[2017,1,31]]},"references-count":56,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,1,31]]}},"alternative-id":["10.1145\/3078840"],"URL":"https:\/\/doi.org\/10.1145\/3078840","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,1,31]]},"assertion":[{"value":"2015-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}