{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T17:23:07Z","timestamp":1777137787353,"version":"3.51.4"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA","license":[{"start":{"date-parts":[[2017,10,12]],"date-time":"2017-10-12T00:00:00Z","timestamp":1507766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1566589"],"award-info":[{"award-number":["1566589"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006785","name":"Google","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006785","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2017,10,12]]},"abstract":"<jats:p>Localizing failure-inducing code is essential for software debugging. Manual fault localization can be quite tedious, error-prone, and time-consuming. Therefore, a huge body of research e orts have been dedicated to automated fault localization. Spectrum-based fault localization, the most intensively studied fault localization approach based on test execution information, may have limited effectiveness, since a code element executed by a failed tests may not necessarily have impact on the test outcome and cause the test failure. To bridge the gap, mutation-based fault localization has been proposed to transform the programs under test to check the impact of each code element for better fault localization. However, there are limited studies on the effectiveness of mutation-based fault localization on sufficient number of real bugs. In this paper, we perform an extensive study to compare mutation-based fault localization techniques with various state-of-the-art spectrum-based fault localization techniques on 357 real bugs from the Defects4J benchmark suite. The study results firstly demonstrate the effectiveness of mutation-based fault localization, as well as revealing a number of guidelines for further improving mutation-based fault localization. Based on the learnt guidelines, we further transform test outputs\/messages and test code to obtain various mutation information. Then, we propose TraPT, an automated Learning-to-Rank technique to fully explore the obtained mutation information for effective fault localization. The experimental results show that TraPT localizes 65.12% and 94.52% more bugs within Top-1 than state-of-the-art mutation and spectrum based techniques when using the default setting of LIBSVM.<\/jats:p>","DOI":"10.1145\/3133916","type":"journal-article","created":{"date-parts":[[2017,10,13]],"date-time":"2017-10-13T15:15:45Z","timestamp":1507907745000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":124,"title":["Transforming programs and tests in tandem for fault localization"],"prefix":"10.1145","volume":"1","author":[{"given":"Xia","family":"Li","sequence":"first","affiliation":[{"name":"University of Texas at Dallas, USA"}]},{"given":"Lingming","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Texas at Dallas, USA"}]}],"member":"320","published-online":{"date-parts":[[2017,10,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC.2006.18"},{"key":"e_1_2_1_2_1","volume-title":"Testing: Academic and Industrial Conference Practice and Research Techniques-MUTATION, 2007","author":"Abreu Rui","year":"2007","unstructured":"Rui Abreu , Peter Zoeteweij , and Arjan JC Van Gemund . 2007 . On the accuracy of spectrum-based fault localization . In Testing: Academic and Industrial Conference Practice and Research Techniques-MUTATION, 2007 . TAICPART-MUTATION 2007. IEEE, 89\u201398. Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and Industrial Conference Practice and Research Techniques-MUTATION, 2007. TAICPART-MUTATION 2007. IEEE, 89\u201398."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1831708.1831715"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931049"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025136"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_2_1_7_1","first-page":"193","article-title":"Learning to rank with nonsmooth cost functions","volume":"6","author":"Burges Christopher JC","year":"2006","unstructured":"Christopher JC Burges , Robert Ragno , and Quoc Viet Le . 2006 . Learning to rank with nonsmooth cost functions . In NIPS , Vol. 6. 193 \u2013 200 . Christopher JC Burges, Robert Ragno, and Quoc Viet Le. 2006. Learning to rank with nonsmooth cost functions. In NIPS, Vol. 6. 193\u2013200.","journal-title":"NIPS"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.12988\/ces.2014.4674"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2017.29"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1978.218136"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 947\u2013958","author":"Denaro Giovanni","year":"2015","unstructured":"Giovanni Denaro , Alessandro Margara , Mauro Pezze , and Mattia Vivanti . 2015 . Dynamic data \ufffdow testing of object oriented systems . In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 947\u2013958 . Giovanni Denaro, Alessandro Margara, Mauro Pezze, and Mattia Vivanti. 2015. Dynamic data \ufffdow testing of object oriented systems. In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, 947\u2013958."},{"key":"e_1_2_1_13_1","first-page":"1138","article-title":"Automatic fault localization for property checking. Computer-Aided Design of Integrated Circuits and Systems","volume":"27","author":"Fey G\u00f6rschwin","year":"2008","unstructured":"G\u00f6rschwin Fey , Stefan Staber , Roderick Bloem , and Rolf Drechsler . 2008 . Automatic fault localization for property checking. Computer-Aided Design of Integrated Circuits and Systems , IEEE Transactions on 27 , 6 (2008), 1138 \u2013 1149 . G\u00f6rschwin Fey, Stefan Staber, Roderick Bloem, and Rolf Drechsler. 2008. Automatic fault localization for property checking. Computer-Aided Design of Integrated Circuits and Systems, IEEE Transactions on 27, 6 (2008), 1138\u20131149.","journal-title":"IEEE Transactions on"},{"key":"e_1_2_1_14_1","first-page":"933","article-title":"An e\ufffdcient boosting algorithm for combining preferences","author":"Freund Yoav","year":"2003","unstructured":"Yoav Freund , Raj Iyer , Robert E Schapire , and Yoram Singer . 2003 . An e\ufffdcient boosting algorithm for combining preferences . Journal of machine learning research 4 , Nov (2003), 933 \u2013 969 . Yoav Freund, Raj Iyer, Robert E Schapire, and Yoram Singer. 2003. An e\ufffdcient boosting algorithm for combining preferences. Journal of machine learning research 4, Nov (2003), 933\u2013969.","journal-title":"Journal of machine learning research 4"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.4135\/9781412983419"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351682"},{"key":"e_1_2_1_17_1","volume-title":"Automated Software Engineering (ASE), 2012 Proceedings of the 27th IEEE\/ACM International Conference on. IEEE, 40\u201349","author":"Gopinath Divya","year":"2012","unstructured":"Divya Gopinath , Razieh Nokhbeh Zaeem , and Sarfraz Khurshid . 2012 . Improving the e\ufffdectiveness of spectra-based fault localization using speci\ufffdcations . In Automated Software Engineering (ASE), 2012 Proceedings of the 27th IEEE\/ACM International Conference on. IEEE, 40\u201349 . Divya Gopinath, Razieh Nokhbeh Zaeem, and Sarfraz Khurshid. 2012. Improving the e\ufffdectiveness of spectra-based fault localization using speci\ufffdcations. In Automated Software Engineering (ASE), 2012 Proceedings of the 27th IEEE\/ACM International Conference on. IEEE, 40\u201349."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcs.2006.12.032"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1977.231145"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0076767"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.62"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1101908.1101949"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628055"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931051"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786880"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227211"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00571"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1064978.1065014"},{"key":"e_1_2_1_29_1","volume-title":"Learning to rank for information retrieval. Foundations and Trends\u00ae in Information Retrieval 3, 3","author":"Liu Tie-Yan","year":"2009","unstructured":"Tie-Yan Liu . 2009. Learning to rank for information retrieval. Foundations and Trends\u00ae in Information Retrieval 3, 3 ( 2009 ), 225\u2013331. Tie-Yan Liu. 2009. Learning to rank for information retrieval. Foundations and Trends\u00ae in Information Retrieval 3, 3 (2009), 225\u2013331."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786811"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884874"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1616"},{"key":"e_1_2_1_33_1","volume-title":"2014 IEEE Seventh International Conference on. IEEE, 153\u2013162","author":"Moon Seokhyeon","year":"2014","unstructured":"Seokhyeon Moon , Yunho Kim , Moonzoo Kim , and Shin Yoo . 2014 . Ask the mutants: Mutating faulty programs for fault localization. In Software Testing, Veri\ufffdcation and Validation (ICST) , 2014 IEEE Seventh International Conference on. IEEE, 153\u2013162 . Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In Software Testing, Veri\ufffdcation and Validation (ICST), 2014 IEEE Seventh International Conference on. IEEE, 153\u2013162."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-49094-6_14"},{"key":"e_1_2_1_35_1","volume-title":"A large-scale study of call graph-based impact prediction using mutation testing. Software Quality Journal","author":"Musco Vincenzo","year":"2016","unstructured":"Vincenzo Musco , Martin Monperrus , and Philippe Preux . 2016. A large-scale study of call graph-based impact prediction using mutation testing. Software Quality Journal ( 2016 ), 1\u201330. Vincenzo Musco, Martin Monperrus, and Philippe Preux. 2016. A large-scale study of call graph-based impact prediction using mutation testing. Software Quality Journal (2016), 1\u201330."},{"key":"e_1_2_1_36_1","volume-title":"Hua Jie Lee, and Kotagiri Ramamohanarao","author":"Naish Lee","year":"2011","unstructured":"Lee Naish , Hua Jie Lee, and Kotagiri Ramamohanarao . 2011 . A model for spectra-based software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3 (2011), 11. Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectra-based software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3 (2011), 11."},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the 2013 International Conference on Software Engineering. 772\u2013781","author":"Thien Nguyen Hoang Duong","year":"2013","unstructured":"Hoang Duong Thien Nguyen , Dawei Qi , Abhik Roychoudhury , and Satish Chandra . 2013 . Sem\ufffdx: Program repair via semantic analysis . In Proceedings of the 2013 International Conference on Software Engineering. 772\u2013781 . Hoang Duong Thien Nguyen, Dawei Qi, Abhik Roychoudhury, and Satish Chandra. 2013. Sem\ufffdx: Program repair via semantic analysis. In Proceedings of the 2013 International Conference on Software Engineering. 772\u2013781."},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the 15th international conference on Software Engineering. IEEE Computer Society Press, 100\u2013107","author":"O\ufffdutt A Je\ufffderson","year":"1993","unstructured":"A Je\ufffderson O\ufffdutt , Gregg Rothermel , and Christian Zapf . 1993 . An experimental evaluation of selective mutation . In Proceedings of the 15th international conference on Software Engineering. IEEE Computer Society Press, 100\u2013107 . A Je\ufffderson O\ufffdutt, Gregg Rothermel, and Christian Zapf. 1993. An experimental evaluation of selective mutation. In Proceedings of the 15th international conference on Software Engineering. IEEE Computer Society Press, 100\u2013107."},{"key":"e_1_2_1_39_1","volume-title":"2012 IEEE Fifth International Conference on. IEEE, 691\u2013700","author":"Papadakis Mike","year":"2012","unstructured":"Mike Papadakis and Yves Le Traon . 2012 . Using mutants to locate\" unknown\" faults. In Software Testing, Veri\ufffdcation and Validation (ICST) , 2012 IEEE Fifth International Conference on. IEEE, 691\u2013700 . Mike Papadakis and Yves Le Traon. 2012. Using mutants to locate\" unknown\" faults. In Software Testing, Veri\ufffdcation and Validation (ICST), 2012 IEEE Fifth International Conference on. IEEE, 691\u2013700."},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the 29th Annual ACM Symposium on Applied Computing. ACM, 1293\u20131300","author":"Papadakis Mike","year":"2014","unstructured":"Mike Papadakis and Yves Le Traon . 2014 . E\ufffdective fault localization via mutation analysis: A selective mutation approach . In Proceedings of the 29th Annual ACM Symposium on Applied Computing. ACM, 1293\u20131300 . Mike Papadakis and Yves Le Traon. 2014. E\ufffdective fault localization via mutation analysis: A selective mutation approach. In Proceedings of the 29th Annual ACM Symposium on Applied Computing. ACM, 1293\u20131300."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1509"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2001420.2001445"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.62"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693093"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2009.5070508"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635921"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277808"},{"key":"e_1_2_1_48_1","volume-title":"Repeated-measures analysis. Design and analysis of ecological experiments","author":"von Ende CARL N","year":"2001","unstructured":"CARL N von Ende . 2001. Repeated-measures analysis. Design and analysis of ecological experiments . Oxford University Press , Oxford ( 2001 ), 134\u2013157. CARL N von Ende. 2001. Repeated-measures analysis. Design and analysis of ecological experiments. Oxford University Press, Oxford (2001), 134\u2013157."},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis. To appear.","author":"Wang Qianqian","year":"2015","unstructured":"Qianqian Wang , Chris Parnin , and Alessandro Orso . 2015 . An In-Depth Study of IR-Based Fault Localization Techniques . In Proceedings of the International Symposium on Software Testing and Analysis. To appear. Qianqian Wang, Chris Parnin, and Alessandro Orso. 2015. An In-Depth Study of IR-Based Fault Localization Techniques. In Proceedings of the International Symposium on Software Testing and Analysis. To appear."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.2307\/3001968"},{"key":"e_1_2_1_51_1","volume-title":"Computer Software and Applications Conference, 2007. COMPSAC 2007. 31st Annual International","volume":"1","author":"Wong W Eric","year":"2007","unstructured":"W Eric Wong , Yu Qi , Lei Zhao , and Kai-Yuan Cai . 2007 . E\ufffdective fault localization using code coverage . In Computer Software and Applications Conference, 2007. COMPSAC 2007. 31st Annual International , Vol. 1 . IEEE, 449\u2013456. W Eric Wong, Yu Qi, Lei Zhao, and Kai-Yuan Cai. 2007. E\ufffdective fault localization using code coverage. In Computer Software and Applications Conference, 2007. COMPSAC 2007. 31st Annual International, Vol. 1. IEEE, 449\u2013456."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.45"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2014.41"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931038"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693070"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2509136.2509551"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3092703.3092731"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227210"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3133916","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3133916","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3133916","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:13:25Z","timestamp":1750212805000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3133916"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,10,12]]},"references-count":58,"journal-issue":{"issue":"OOPSLA","published-print":{"date-parts":[[2017,10,12]]}},"alternative-id":["10.1145\/3133916"],"URL":"https:\/\/doi.org\/10.1145\/3133916","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,10,12]]},"assertion":[{"value":"2017-10-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}