{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:02:24Z","timestamp":1750309344133,"version":"3.41.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,1,24]],"date-time":"2025-01-24T00:00:00Z","timestamp":1737676800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["NRF-2021R1A5A1021944, 2021R1I1A3048013"],"award-info":[{"award-number":["NRF-2021R1A5A1021944, 2021R1I1A3048013"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"name":"yungpook National University Research Fund"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>\n            We present\n            <jats:sc>Flex<\/jats:sc>\n            , a new approach to improve fault localization with external oracles. Spectrum-based fault localization techniques estimate suspicious statements based on the execution trace of the test suite. State-of-the-art techniques rely on test oracles that\n            <jats:italic>internally<\/jats:italic>\n            exist in the program. However, programs often have\n            <jats:italic>external<\/jats:italic>\n            oracles that observe their behavior from outside. This in turn hinders fine-grained and accurate estimation of suspicious statements in practice because the correctness of each execution can only be observed at termination. In this article, we aim to address this problem by observing counterfactual execution traces, which enable fine-grained estimation even without precise internal oracles. We observe two types of counterfactual scenarios related to different types of test cases: When the branch condition is set to a Boolean constant, (1) if most of the passing test cases still pass, we consider the newly executed statements in the branch statement as\n            <jats:italic>unrelated<\/jats:italic>\n            to the failure; (2) if failing test case still fails, we also consider the originally executed statements as\n            <jats:italic>unrelated<\/jats:italic>\n            to the failure. We evaluated the performance on widely used C and Java programs.\n            <jats:sc>Flex<\/jats:sc>\n            improves the accuracy of state-of-the-art SBFL techniques on C and Java programs by 24% and 22% on average, respectively.\n          <\/jats:p>","DOI":"10.1145\/3695997","type":"journal-article","created":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T15:43:55Z","timestamp":1727365435000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Improving Fault Localization with External Oracle by Using Counterfactual Execution"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8074-1768","authenticated-orcid":false,"given":"Jongchan","family":"Park","sequence":"first","affiliation":[{"name":"FuriosaAI, Seoul, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2442-8646","authenticated-orcid":false,"given":"Tae Eun","family":"Kim","sequence":"additional","affiliation":[{"name":"KAIST, Daejeon, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0272-6860","authenticated-orcid":false,"given":"Dongsun","family":"Kim","sequence":"additional","affiliation":[{"name":"Kyungpook National University, Daegu, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2671-0142","authenticated-orcid":false,"given":"Kihong","family":"Heo","sequence":"additional","affiliation":[{"name":"KAIST, Daejeon, Korea"}]}],"member":"320","published-online":{"date-parts":[[2025,1,24]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"39","volume-title":"Proceedings of the 12th IEEE Pacific Rim International Symposium on Dependable Computing (PRDC \u201906)","author":"Abreu Rui","year":"2006","unstructured":"Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2006. An evaluation of similarity coefficients for software fault localization. In Proceedings of the 12th IEEE Pacific Rim International Symposium on Dependable Computing (PRDC \u201906), 39\u201346."},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1145\/2610384.2628058","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis (ISSTA \u201914)","author":"B\u00f6hme Marcel","year":"2014","unstructured":"Marcel B\u00f6hme and Abhik Roychoudhury. 2014. CoREBench: Studying complexity of regression errors. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA \u201914), 105\u2013115."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","unstructured":"Thierry Titcheu Chekam Mike Papadakis and Yves Le Traon. 2016. Assessing and comparing mutation-based fault localization techniques. arXiv:1607.05512v1. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1607.05512","DOI":"10.48550\/arXiv.1607.05512"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2011.6100107"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2454513"},{"issue":"8","key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"e1802","DOI":"10.1002\/stvr.1802","article-title":"HOTFUZ: Cost-effective higher-order mutation-based fault localization","volume":"32","author":"Jang Jong-In","year":"2022","unstructured":"Jong-In Jang, Duksan Ryu, and Jongmoon Baik. 2022. HOTFUZ: Cost-effective higher-order mutation-based fault localization. Software Testing, Verification and Reliability 32, 8 (2022), e1802.","journal-title":"Software Testing, Verification and Reliability"},{"key":"e_1_3_2_8_2","volume-title":"Proceedings of the 25th IEEE International Conference on Software Maintenance (ICSM \u201909)","author":"Jeffrey Dennis","year":"2009","unstructured":"Dennis Jeffrey, Neelam Gupta, and Rajiv Gupta. 2009. Effective and efficient localization of multiple faults using value replacement. In Proceedings of the 25th IEEE International Conference on Software Maintenance (ICSM \u201909)."},{"key":"e_1_3_2_9_2","volume-title":"Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201905)","author":"Jones James A.","year":"2005","unstructured":"James A. Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201905)."},{"key":"e_1_3_2_10_2","volume-title":"Proceedings of the International Conference on Software Engineering (ICSE \u201902)","author":"Jones James A.","year":"2002","unstructured":"James A. Jones, Mary Jean Harrold, and John T. Stasko. 2002. Visualization of test information to assist fault localization. In Proceedings of the International Conference on Software Engineering (ICSE \u201902)."},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1145\/2610384.2628055","volume-title":"Proceedings of the International Symposium on Software Testing and Analysis (ISSTA \u201914)","author":"Just Ren\u00e9","year":"2014","unstructured":"Ren\u00e9 Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA \u201914), 437\u2013440."},{"issue":"4","key":"e_1_3_2_12_2","first-page":"23:1","article-title":"Precise learn-to-rank fault localization using dynamic and static features of target programs","volume":"28","author":"Kim Yunho","year":"2019","unstructured":"Yunho Kim, Seokhyeon Mun, Shin Yoo, and Moonzoo Kim. 2019. Precise learn-to-rank fault localization using dynamic and static features of target programs. ACM Transactions on Software Engineering and Methodology (TOSEM) 28, 4 (2019), 23:1\u201323:34.","journal-title":"ACM Transactions on Software Engineering and Methodology (TOSEM)"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00066"},{"key":"e_1_3_2_14_2","volume-title":"Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201919)","author":"Li Xia","year":"2019","unstructured":"Xia Li, Wei Li, Yuqun Zhang, and Lingming Zhang. 2019. DeepFL: Integrating multiple fault diagnosis dimensions for deep fault localization. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201919)."},{"key":"e_1_3_2_15_2","first-page":"1","volume-title":"Proceedings of the ACM on Programming Languages","volume":"1","author":"Li Xia","year":"2017","unstructured":"Xia Li and Lingming Zhang. 2017. Transforming programs and tests in tandem for fault localization. Proceedings of the ACM on Programming Languages 1, OOPSLA (2017), 1\u201330."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00067"},{"key":"e_1_3_2_17_2","first-page":"153","volume-title":"Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST \u201914)","author":"Moon Seokhyeon","year":"2014","unstructured":"Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST \u201914), 153\u2013162."},{"key":"e_1_3_2_18_2","first-page":"213","volume-title":"Proceedings of the Compiler Construction, 11th International Conference, CC 2002, Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS \u201902)","volume":"2304","author":"Necula George C.","year":"2002","unstructured":"George C. Necula, Scott McPeak, Shree Prakash Rahul, and Westley Weimer. 2002. CIL: Intermediate language and tools for analysis and transformation of C programs. In Proceedings of the Compiler Construction, 11th International Conference, CC 2002, Held as Part of the Joint European Conferences on Theory and Practice of Software (ETAPS \u201902). R. Nigel Horspool (Ed.), Vol. 2304. Springer, 213\u2013228."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1509"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.2346"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.62"},{"key":"e_1_3_2_22_2","first-page":"382","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201920)","author":"Podgurski Andy","year":"2020","unstructured":"Andy Podgurski and Yigit K\u00fc\u00e7\u00fck. 2020. CounterFault: Value-based fault localization by modeling and predicting counterfactual outcomes. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201920), 382\u2013393."},{"key":"e_1_3_2_23_2","volume-title":"Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201917)","author":"Sohn Jeongju","year":"2017","unstructured":"Jeongju Sohn and Shin Yoo. 2017. FLUCCS: Using code and change metrics to improve fault localization. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201917)."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09931-7"},{"key":"e_1_3_2_25_2","article-title":"The DStar method for effective software fault localization","author":"Wong W. Eric","year":"2014","unstructured":"W. Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2014. The DStar method for effective software fault localization. IEEE Transactions on Reliability (TRel) (2014).","journal-title":"IEEE Transactions on Reliability (TRel)"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.107"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194011005505"},{"key":"e_1_3_2_28_2","first-page":"196","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201920)","author":"Xu Xuezheng","year":"2020","unstructured":"Xuezheng Xu, Changwei Zou, and Jingling Xue. 2020. Every mutation should be rewarded: Boosting fault localization with mutated predicates. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME \u201920), 196\u2013207."},{"key":"e_1_3_2_29_2","first-page":"196","volume-title":"Proceedings of the 30th IEEE International Conference on Software Maintenance and Evolution (ICSME \u201914)","author":"Xuan Jifeng","year":"2014","unstructured":"Jifeng Xuan and Martin Monperrus. 2014. Learning to combine multiple ranking metrics for fault localization. In Proceedings of the 30th IEEE International Conference on Software Maintenance and Evolution (ICSME \u201914), 196\u2013207."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510073"},{"key":"e_1_3_2_31_2","first-page":"765","volume-title":"Proceedings of the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA \u201913)","author":"Zhang Lingming","year":"2013","unstructured":"Lingming Zhang, Lu Zhang, and Sarfraz Khurshid. 2013. Injecting mechanical faults to localize developer faults for evolving software. In Proceedings of the ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications (OOPSLA \u201913), 765\u2013784."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2911283"},{"key":"e_1_3_2_33_2","first-page":"272","volume-title":"Proceedings of the International Conference on Software Engineering (ICSE \u201906)","author":"Zhang Xiangyu","year":"2006","unstructured":"Xiangyu Zhang, Neelam Gupta, and Rajiv Gupta. 2006. Locating faults through automated predicate switching. In Proceedings of the International Conference on Software Engineering (ICSE \u201906), 272\u2013281."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2892102"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3695997","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3695997","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:30Z","timestamp":1750291470000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3695997"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,24]]},"references-count":33,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3695997"],"URL":"https:\/\/doi.org\/10.1145\/3695997","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"type":"print","value":"1049-331X"},{"type":"electronic","value":"1557-7392"}],"subject":[],"published":{"date-parts":[[2025,1,24]]},"assertion":[{"value":"2024-02-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}