{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T11:35:11Z","timestamp":1791459311196,"version":"4.3.4"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,5,30]],"date-time":"2022-05-30T00:00:00Z","timestamp":1653868800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,5,30]],"date-time":"2022-05-30T00:00:00Z","timestamp":1653868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Applying mutation testing to test subtle program changes, such as program patches or other small-scale code modifications, requires using mutants that capture the delta of the altered behaviours. To address this issue, we introduce the concept of commit-relevant mutants, which are the mutants that interact with the behaviours of the system affected by a particular commit. Therefore, commit-aware mutation testing, is a test assessment metric tailored to a specific commit. By analysing 83 commits from 25 projects involving 2,253,610 mutants in both C and Java, we identify the commit-relevant mutants and explore their relationship with other categories of mutants. Our results show that commit-relevant mutants represent a small subset of all mutants, which differs from the other classes of mutants (subsuming and hard-to-kill), and that the commit-relevant mutation score is weakly correlated with the traditional mutation score (Kendall\/Pearson 0.15-0.4). Moreover, commit-aware mutation analysis provides insights about the testing of a commit, which can be more efficient than the classical mutation analysis; in our experiments, by analysing the same number of mutants, commit-aware mutants have better fault-revelation potential (30% higher chances of revealing commit-introducing faults) than traditional mutants. We also illustrate a possible application of commit-aware mutation testing as a metric to evaluate test case prioritisation.<\/jats:p>","DOI":"10.1007\/s10664-022-10138-1","type":"journal-article","created":{"date-parts":[[2022,5,30]],"date-time":"2022-05-30T09:03:03Z","timestamp":1653901383000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["On the use of commit-relevant mutants"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8213-3381","authenticated-orcid":false,"given":"Milo\u0161","family":"Ojdani\u0107","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Laurent","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thierry Titcheu","family":"Chekam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anthony","family":"Ventresque","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mike","family":"Papadakis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,5,30]]},"reference":[{"key":"10138_CR1","doi-asserted-by":"crossref","unstructured":"Ammann P, Delamaro M E, Offutt J (2014) Establishing theoretical minimal sets of mutants. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation. IEEE, pp 21\u201330","DOI":"10.1109\/ICST.2014.13"},{"key":"10138_CR2","doi-asserted-by":"publisher","unstructured":"Ammann P, Offutt J (2008) Introduction to software testing. Cambridge University Press. https:\/\/doi.org\/10.1017\/CBO9780511809163","DOI":"10.1017\/CBO9780511809163"},{"issue":"8","key":"10138_CR3","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1109\/TSE.2006.83","volume":"32","author":"JH Andrews","year":"2006","unstructured":"Andrews J H, Briand L C, Labiche Y, Namin A S (2006) Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Trans Softw Eng 32(8):608\u2013624. https:\/\/doi.org\/10.1109\/TSE.2006.83","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR4","doi-asserted-by":"publisher","unstructured":"Apiwattanapong T, Santelices R A, Chittimalli P K, Orso A, Harrold M J (2006) MATRIX: maintenance-oriented testing requirements identifier and examiner. In: Testing: Academia and Industry Conference - Practice And Research Techniques (TAIC PART 2006), 29-31 August 2006, Windsor, United Kingdom. https:\/\/doi.org\/10.1109\/TAIC-PART.2006.18, pp 137\u2013146","DOI":"10.1109\/TAIC-PART.2006.18"},{"issue":"8","key":"10138_CR5","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1109\/32.624306","volume":"23","author":"DW Binkley","year":"1997","unstructured":"Binkley D W (1997) Semantics guided regression test cost reduction. IEEE Trans Softw Eng 23(8):498\u2013516. https:\/\/doi.org\/10.1109\/32.624306","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR6","doi-asserted-by":"publisher","unstructured":"Binkley D W, Gold N E, Harman M, Islam S S, Krinke J, Yoo S (2015) ORBS and the limits of static slicing. In: 15th IEEE International Working Conference on Source Code Analysis and Manipulation, SCAM 2015, Bremen, Germany, September 27-28, 2015. https:\/\/doi.org\/10.1109\/SCAM.2015.7335396, pp 1\u201310","DOI":"10.1109\/SCAM.2015.7335396"},{"key":"10138_CR7","doi-asserted-by":"publisher","unstructured":"B\u00f6hme M, d. S. Oliveira B C, Roychoudhury A (2013) Regression tests to expose change interaction errors. In: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC\/FSE\u201913, Saint Petersburg, Russian Federation. https:\/\/doi.org\/10.1145\/2491411.2491430, pp 334\u2013344","DOI":"10.1145\/2491411.2491430"},{"key":"10138_CR8","doi-asserted-by":"publisher","unstructured":"B\u00f6hme M, Roychoudhury A (2014) Corebench: studying complexity of regression errors. In: International Symposium on Software Testing and Analysis, ISSTA \u201914, San Jose, CA, USA - July 21 - 26, 2014. https:\/\/doi.org\/10.1145\/2610384.2628058, pp 105\u2013115","DOI":"10.1145\/2610384.2628058"},{"issue":"1","key":"10138_CR9","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1007\/BF00625279","volume":"18","author":"TA Budd","year":"1982","unstructured":"Budd T A, Angluin D (1982) Two Notions of Correctness and Their Relation to Testing. Acta Informatica 18(1):31\u201345","journal-title":"Acta Informatica"},{"key":"10138_CR10","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1016\/j.entcs.2013.02.012","volume":"294","author":"MA Cachia","year":"2013","unstructured":"Cachia M A, Micallef M, Colombo C (2013) Towards incremental mutation testing. Electr Notes Theor Comput Sci 294:2\u201311. https:\/\/doi.org\/10.1016\/j.entcs.2013.02.012","journal-title":"Electr Notes Theor Comput Sci"},{"key":"10138_CR11","unstructured":"Cadar C, Dunbar D, Engler D (2008) Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation, OSDI\u201908, USENIX Association, USA, pp 209\u2013224"},{"issue":"1","key":"10138_CR12","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1007\/s10664-019-09778-7","volume":"25","author":"TT Chekam","year":"2020","unstructured":"Chekam T T, Papadakis M, Bissyand\u00e9 T F, Traon Y L, Sen K (2020) Selecting fault revealing mutants. Empir Softw Eng 25(1):434\u2013487. https:\/\/doi.org\/10.1007\/s10664-019-09778-7","journal-title":"Empir Softw Eng"},{"key":"10138_CR13","doi-asserted-by":"crossref","unstructured":"Chekam T T, Papadakis M, Cordy M, Traon Y L (2020) Killing stubborn mutants with symbolic execution. arXiv:2001.02941","DOI":"10.1145\/3425497"},{"key":"10138_CR14","doi-asserted-by":"publisher","unstructured":"Chekam T T, Papadakis M, Traon Y L (2019) Mart: a mutant generation tool for LLVM. In: Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC\/SIGSOFT FSE 2019. https:\/\/doi.org\/10.1145\/3338906.3341180, Tallinn, pp 1080\u20131084","DOI":"10.1145\/3338906.3341180"},{"key":"10138_CR15","doi-asserted-by":"publisher","unstructured":"Chekam T T, Papadakis M, Traon Y L (2020) Muteria: An extensible and flexible multi-criteria software testing framework. In: AST \u201920: International Conference on Automation of Software Test (AST \u201920). https:\/\/doi.org\/10.1145\/3387903.3389316, ACM, New York, p 4","DOI":"10.1145\/3387903.3389316"},{"key":"10138_CR16","doi-asserted-by":"publisher","unstructured":"Chekam T T, Papadakis M, Traon Y L, Harman M (2017) An empirical study on mutation, statement and branch coverage fault revelation that avoids the unreliable clean program assumption. In: Proceedings of the 39th International Conference on Software Engineering, ICSE 2017, Buenos Aires, Argentina, May 20-28, 2017. https:\/\/doi.org\/10.1109\/ICSE.2017.61, pp 597\u2013608","DOI":"10.1109\/ICSE.2017.61"},{"key":"10138_CR17","doi-asserted-by":"publisher","unstructured":"Coles H, Laurent T, Henard C, Papadakis M, Ventresque A (2016) PIT: a practical mutation testing tool for java (demo). In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016, Saarbr\u00fccken, Germany, July 18-20, 2016. https:\/\/doi.org\/10.1145\/2931037.2948707, pp 449\u2013452","DOI":"10.1145\/2931037.2948707"},{"issue":"4","key":"10138_CR18","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1002\/stvr.1572","volume":"25","author":"D Di Nardo","year":"2015","unstructured":"Di Nardo D, Alshahwan N, Briand L, Labiche Y (2015) Coverage-based regression test case selection, minimization and prioritization: A case study on an industrial system. Softw Test Verif Reliab 25(4):371\u2013396","journal-title":"Softw Test Verif Reliab"},{"issue":"9","key":"10138_CR19","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1109\/TSE.2006.92","volume":"32","author":"H Do","year":"2006","unstructured":"Do H, Rothermel G (2006) On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Trans Softw Eng 32(9):733\u2013752","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR20","doi-asserted-by":"publisher","unstructured":"Evans R B, Savoia A (2007) Differential testing: a new approach to change detection. In: Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering. https:\/\/doi.org\/10.1145\/1287624.1287707, pp 549\u2013552","DOI":"10.1145\/1287624.1287707"},{"key":"10138_CR21","doi-asserted-by":"publisher","unstructured":"Falleri J-R, Morandat F, Blanc X, Martinez M, Monperrus M (2014) Fine-grained and accurate source code differencing. In: ACM\/IEEE international conference on automated software engineering, ASE \u201914. https:\/\/doi.org\/10.1145\/2642937.2642982, Vasteras, pp 313\u2013324","DOI":"10.1145\/2642937.2642982"},{"issue":"12","key":"10138_CR22","doi-asserted-by":"publisher","first-page":"2826","DOI":"10.1007\/s11432-012-4746-9","volume":"55","author":"C Fang","year":"2012","unstructured":"Fang C, Chen Z, Xu B (2012) Comparing logic coverage criteria on test case prioritization. Sci China Inf Sci 55(12):2826\u20132840","journal-title":"Sci China Inf Sci"},{"issue":"2","key":"10138_CR23","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/TSE.2011.93","volume":"38","author":"G Fraser","year":"2012","unstructured":"Fraser G, Zeller A (2012) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278\u2013292. https:\/\/doi.org\/10.1109\/TSE.2011.93","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR24","doi-asserted-by":"publisher","unstructured":"Garg A, Ojdanic M, Degiovanni R, Chekam T T, Papadakis M, Traon Y L (2022) Cerebro: Static subsuming mutant selection. IEEE Trans Softw Eng. https:\/\/doi.org\/10.1109\/TSE.2022.3140510","DOI":"10.1109\/TSE.2022.3140510"},{"key":"10138_CR25","doi-asserted-by":"publisher","first-page":"106496","DOI":"10.1016\/j.infsof.2020.106496","volume":"132","author":"R Gheyi","year":"2021","unstructured":"Gheyi R, Ribeiro M, Souza B, Guimar\u00e3es M A, Fernandes L, d\u2019Amorim M, Alves V, Teixeira L, Fonseca B (2021) Identifying method-level mutation subsumption relations using Z3. Inf Softw Technol 132:106496. https:\/\/doi.org\/10.1016\/j.infsof.2020.106496","journal-title":"Inf Softw Technol"},{"key":"10138_CR26","doi-asserted-by":"publisher","unstructured":"Hariri F, Shi A, Fernando V, Mahmood S, Marinov D (2019) Comparing mutation testing at the levels of source code and compiler intermediate representation. In: 12th IEEE Conference on Software Testing, Validation and Verification, ICST 2019, Xi\u2019an, China, April 22-27, 2019. https:\/\/doi.org\/10.1109\/ICST.2019.00021, pp 114\u2013124","DOI":"10.1109\/ICST.2019.00021"},{"key":"10138_CR27","doi-asserted-by":"crossref","unstructured":"Henard C, Papadakis M, Harman M, Jia Y, Le Traon Y (2016) Comparing white-box and black-box test prioritization. In: 2016 IEEE\/ACM 38th International Conference on Software Engineering (ICSE), pp 523\u2013534","DOI":"10.1145\/2884781.2884791"},{"issue":"10","key":"10138_CR28","doi-asserted-by":"publisher","first-page":"1379","DOI":"10.1016\/j.infsof.2009.04.016","volume":"51","author":"Y Jia","year":"2009","unstructured":"Jia Y, Harman M (2009) Higher order mutation testing. Inf Softw Technol 51(10):1379\u20131393. https:\/\/doi.org\/10.1016\/j.infsof.2009.04.016","journal-title":"Inf Softw Technol"},{"key":"10138_CR29","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/j.infsof.2017.08.014","volume":"93","author":"M Khatibsyarbini","year":"2018","unstructured":"Khatibsyarbini M, Isa M A, Jawawi DNA, Tumeng R (2018) Test case prioritization approaches in regression testing: A systematic literature review. Inf Softw Technol 93:74\u201393","journal-title":"Inf Softw Technol"},{"issue":"4","key":"10138_CR30","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.2017.2684805","volume":"44","author":"M Kintis","year":"2018","unstructured":"Kintis M, Papadakis M, Jia Y, Malevris N, Traon Y L, Harman M (2018) Detecting trivial mutant equivalences via compiler optimisations. IEEE Trans Softw Eng 44(4):308\u2013333. https:\/\/doi.org\/10.1109\/TSE.2017.2684805","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR31","doi-asserted-by":"publisher","unstructured":"Kintis M, Papadakis M, Malevris N (2010) Evaluating mutation testing alternatives: A collateral experiment. In: Han J, Thu T D (eds) 17th Asia Pacific Software Engineering Conference, APSEC 2010. https:\/\/doi.org\/10.1109\/APSEC.2010.42. IEEE Computer Society, Sydney, pp 300\u2013309","DOI":"10.1109\/APSEC.2010.42"},{"issue":"3","key":"10138_CR32","doi-asserted-by":"publisher","first-page":"10:1","DOI":"10.1145\/3208952","volume":"27","author":"T Kuchta","year":"2018","unstructured":"Kuchta T, Palikareva H, Cadar C (2018) Shadow symbolic execution for testing software patches. ACM Trans Softw Eng Methodol 27(3):10:1\u201310:32. https:\/\/doi.org\/10.1145\/3208952","journal-title":"ACM Trans Softw Eng Methodol"},{"key":"10138_CR33","doi-asserted-by":"crossref","unstructured":"Kurtz B, Ammann P, Delamaro M E, Offutt J, Deng L (2014) Mutant subsumption graphs. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation Workshops. IEEE, pp 176\u2013185","DOI":"10.1109\/ICSTW.2014.20"},{"key":"10138_CR34","doi-asserted-by":"publisher","unstructured":"Kurtz B, Ammann P, Offutt J, Delamaro M E, Kurtz M, G\u00f6k\u00e7e N (2016) Analyzing the validity of selective mutation with dominator mutants. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2016. https:\/\/doi.org\/10.1145\/2950290.2950322, Seattle, pp 571\u2013582","DOI":"10.1145\/2950290.2950322"},{"key":"10138_CR35","doi-asserted-by":"publisher","unstructured":"Lam W, Godefroid P, Nath S, Santhiar A, Thummalapenta S (2019) Root causing flaky tests in a large-scale industrial setting. ISSTA 2019. https:\/\/doi.org\/10.1145\/3293882.3330570. Association for Computing Machinery, New York, pp 101\u2013111","DOI":"10.1145\/3293882.3330570"},{"key":"10138_CR36","doi-asserted-by":"crossref","unstructured":"Laurent T, Papadakis M, Kintis M, Henard C, Le Traon Y, Ventresque A (2017) Assessing and improving the mutation testing practice of pit. In: 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, pp 430\u2013435","DOI":"10.1109\/ICST.2017.47"},{"key":"10138_CR37","doi-asserted-by":"publisher","unstructured":"Leong C, Singh A, Papadakis M, Traon Y L, Micco J (2019) Assessing transition-based test selection algorithms at google. In: Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP) 2019, Montreal, QC, Canada, May 25-31, 2019. https:\/\/doi.org\/10.1109\/ICSE-SEIP.2019.00019, pp 101\u2013110","DOI":"10.1109\/ICSE-SEIP.2019.00019"},{"key":"10138_CR38","unstructured":"Li L, Zhou Y, Yu Y, Zhao F, Wu S, Yang Z (2019) An empirical study of mutation-based test case clustering prioritization and reduction technique. ICSEA 2019:13"},{"key":"10138_CR39","doi-asserted-by":"crossref","unstructured":"Li N, Praphamontripong U, Offutt J (2009) An experimental comparison of four unit test criteria: Mutation, edge-pair, all-uses and prime path coverage. In: 2009 International Conference on Software Testing, Verification, and Validation Workshops. IEEE, pp 220\u2013229","DOI":"10.1109\/ICSTW.2009.30"},{"key":"10138_CR40","doi-asserted-by":"crossref","unstructured":"Lou Y, Hao D, Zhang L (2015) Mutation-based test-case prioritization in software evolution. In: 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pp 46\u201357","DOI":"10.1109\/ISSRE.2015.7381798"},{"key":"10138_CR41","doi-asserted-by":"crossref","unstructured":"Luo Q, Moran K, Poshyvanyk D, Di Penta M (2018) Assessing test case prioritization on real faults and mutants. In: 2018 IEEE international conference on software maintenance and evolution (ICSME). IEEE, pp 240\u2013251","DOI":"10.1109\/ICSME.2018.00033"},{"key":"10138_CR42","doi-asserted-by":"crossref","unstructured":"Ma W, Laurent T, Ojdani\u0107 M, Chekam T T, Ventresque A, Papadakis M (2020) Commit-aware mutation testing. In: 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, pp 394\u2013405","DOI":"10.1109\/ICSME46990.2020.00045"},{"key":"10138_CR43","doi-asserted-by":"publisher","unstructured":"Mao D, Chen L, Zhang L (2019) An extensive study on cross-project predictive mutation testing. In: 12th IEEE Conference on Software Testing, Validation and Verification, ICST 2019. https:\/\/doi.org\/10.1109\/ICST.2019.00025, Xi\u2019an, pp 160\u2013171","DOI":"10.1109\/ICST.2019.00025"},{"key":"10138_CR44","doi-asserted-by":"publisher","unstructured":"Marcozzi M, Bardin S, Kosmatov N, Papadakis M, Prevosto V, Correnson L (2018) Time to clean your test objectives. In: Chaudron M, Crnkovic I, Chechik M, Harman M (eds) Proceedings of the 40th International Conference on Software Engineering, ICSE 2018. https:\/\/doi.org\/10.1145\/3180155.3180191. ACM, Gothenburg, pp 456\u2013467","DOI":"10.1145\/3180155.3180191"},{"key":"10138_CR45","doi-asserted-by":"publisher","unstructured":"Marinescu P D, Cadar C (2013) KATCH: high-coverage testing of software patches. In: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, ESEC\/FSE\u201913. https:\/\/doi.org\/10.1145\/2491411.2491438, Saint Petersburg, pp 235\u2013245","DOI":"10.1145\/2491411.2491438"},{"key":"10138_CR46","doi-asserted-by":"crossref","unstructured":"Offutt A J, Rothermel G, Zapf C (1993) An experimental evaluation of selective mutation. In: Proceedings of the 15th International Conference on Software Engineering. http:\/\/portal.acm.org\/citation.cfm?id=257572.257597, Baltimore, pp 100\u2013107","DOI":"10.1109\/ICSE.1993.346062"},{"key":"10138_CR47","doi-asserted-by":"publisher","unstructured":"Papadakis M, Chekam T T, Traon Y L (2018) Mutant quality indicators. In: 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops, ICST Workshops. https:\/\/doi.org\/10.1109\/ICSTW.2018.00025. IEEE Computer Society, V\u00e4ster\u00e5s, pp 32\u201339","DOI":"10.1109\/ICSTW.2018.00025"},{"key":"10138_CR48","doi-asserted-by":"publisher","unstructured":"Papadakis M, Henard C, Harman M, Jia Y, Traon Y L (2016) Threats to the validity of mutation-based test assessment. In: Proceedings of the 25th International Symposium on Software Testing and Analysis, ISSTA 2016. https:\/\/doi.org\/10.1145\/2931037.2931040, Saarbr\u00fccken, pp 354\u2013365","DOI":"10.1145\/2931037.2931040"},{"key":"10138_CR49","doi-asserted-by":"publisher","unstructured":"Papadakis M, Jia Y, Harman M, Traon Y L (2015) Trivial compiler equivalence: A large scale empirical study of a simple, fast and effective equivalent mutant detection technique. In: Bertolino A, Canfora G, Elbaum S G (eds) 37th IEEE\/ACM International Conference on Software Engineering, ICSE 2015. https:\/\/doi.org\/10.1109\/ICSE.2015.103, vol 1. IEEE Computer Society, Florence, pp 936\u2013946","DOI":"10.1109\/ICSE.2015.103"},{"key":"10138_CR50","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/bs.adcom.2018.03.015","volume":"112","author":"M Papadakis","year":"2019","unstructured":"Papadakis M, Kintis M, Zhang J, Jia Y, Traon Y L, Harman M (2019) Chapter six - mutation testing advances: An analysis and survey. Adv Comput 112:275\u2013378. https:\/\/doi.org\/10.1016\/bs.adcom.2018.03.015","journal-title":"Adv Comput"},{"key":"10138_CR51","doi-asserted-by":"publisher","unstructured":"Papadakis M, Shin D, Yoo S, Bae D-H (2018) Are mutation scores correlated with real fault detection?: a large scale empirical study on the relationship between mutants and real faults. In: Proceedings of the 40th International Conference on Software Engineering, ICSE 2018. https:\/\/doi.org\/10.1145\/3180155.3180183, Gothenburg, pp 537\u2013548","DOI":"10.1145\/3180155.3180183"},{"key":"10138_CR52","doi-asserted-by":"publisher","unstructured":"Person S, Dwyer M B, Elbaum S G, Pasareanu C S (2008) Differential symbolic execution. In: Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering. https:\/\/doi.org\/10.1145\/1453101.1453131, Atlanta, pp 226\u2013237","DOI":"10.1145\/1453101.1453131"},{"key":"10138_CR53","doi-asserted-by":"publisher","unstructured":"Petrovic G, Ivankovic M (2018) State of mutation testing at google. In: Proceedings of the 40th International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP) 2018. https:\/\/doi.org\/10.1145\/3183519.3183521, Gothenburg, pp 163\u2013171","DOI":"10.1145\/3183519.3183521"},{"key":"10138_CR54","doi-asserted-by":"publisher","unstructured":"Qi D, Roychoudhury A, Liang Z (2010) Test generation to expose changes in evolving programs. In: ASE 2010, 25th IEEE\/ACM International Conference on Automated Software Engineering. https:\/\/doi.org\/10.1145\/1858996.1859083, Antwerp, pp 397\u2013406","DOI":"10.1145\/1858996.1859083"},{"key":"10138_CR55","doi-asserted-by":"publisher","unstructured":"Rothermel G, Harrold M J (1994) Selecting tests and identifying test coverage requirements for modified software. In: Proceedings of the 1994 International Symposium on Software Testing and Analysis, ISSTA 1994. https:\/\/doi.org\/10.1145\/186258.187171, Seattle, pp 169\u2013184","DOI":"10.1145\/186258.187171"},{"key":"10138_CR56","doi-asserted-by":"crossref","unstructured":"Rothermel G, Untch R H, Chu C, Harrold M J (1999) Test case prioritization: An empirical study. In: Proceedings IEEE International Conference on Software Maintenance-1999 (ICSM\u201999).\u2019 Software Maintenance for Business Change\u2019(Cat. No. 99CB36360). IEEE, pp 179\u2013188","DOI":"10.1109\/ICSM.1999.792604"},{"key":"10138_CR57","doi-asserted-by":"publisher","unstructured":"Santelices R A, Chittimalli P K, Apiwattanapong T, Orso A, Harrold M J (2008) Test-suite augmentation for evolving software. In: 23rd IEEE\/ACM International Conference on Automated Software Engineering (ASE 2008). https:\/\/doi.org\/10.1109\/ASE.2008.32, L\u2019Aquila, pp 218\u2013227","DOI":"10.1109\/ASE.2008.32"},{"key":"10138_CR58","doi-asserted-by":"publisher","unstructured":"Santelices R A, Harrold M J (2010) Exploiting program dependencies for scalable multiple-path symbolic execution. In: Proceedings of the Nineteenth International Symposium on Software Testing and Analysis, ISSTA 2010. https:\/\/doi.org\/10.1145\/1831708.1831733, Trento, pp 195\u2013206","DOI":"10.1145\/1831708.1831733"},{"key":"10138_CR59","doi-asserted-by":"publisher","unstructured":"Santelices R A, Harrold M J (2011) Applying aggressive propagation-based strategies for testing changes. In: Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST 2011. https:\/\/doi.org\/10.1109\/ICST.2011.46, Berlin, pp 11\u201320","DOI":"10.1109\/ICST.2011.46"},{"issue":"1-2","key":"10138_CR60","doi-asserted-by":"publisher","first-page":"e1695","DOI":"10.1002\/stvr.1695","volume":"29","author":"D Shin","year":"2019","unstructured":"Shin D, Yoo S, Papadakis M, Bae D-H (2019) Empirical evaluation of mutation-based test case prioritization techniques. Softw Test Verif Reliab 29(1-2):e1695","journal-title":"Softw Test Verif Reliab"},{"issue":"3","key":"10138_CR61","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1007\/s10664-008-9083-7","volume":"14","author":"BH Smith","year":"2009","unstructured":"Smith B H, Williams L (2009) On guiding the augmentation of an automated test suite via mutation analysis. Empir Softw Eng 14 (3):341\u2013369. https:\/\/doi.org\/10.1007\/s10664-008-9083-7","journal-title":"Empir Softw Eng"},{"issue":"11","key":"10138_CR62","doi-asserted-by":"publisher","first-page":"1819","DOI":"10.1016\/j.jss.2009.06.031","volume":"82","author":"BH Smith","year":"2009","unstructured":"Smith B H, Williams L (2009) Should software testers use mutation analysis to augment a test set?. J Syst Softw 82(11):1819\u20131832. https:\/\/doi.org\/10.1016\/j.jss.2009.06.031","journal-title":"J Syst Softw"},{"key":"10138_CR63","doi-asserted-by":"publisher","unstructured":"Xu Z, Kim Y, Kim M, Rothermel G, Cohen M B (2010) Directed test suite augmentation: techniques and tradeoffs. In: Proceedings of the 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering. https:\/\/doi.org\/10.1145\/1882291.1882330, Santa Fe, pp 257\u2013266","DOI":"10.1145\/1882291.1882330"},{"issue":"2","key":"10138_CR64","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1002\/stv.430","volume":"22","author":"S Yoo","year":"2012","unstructured":"Yoo S, Harman M (2012) Regression testing minimization, selection and prioritization: a survey. Softw Test Verif Reliab 22(2):67\u2013120. https:\/\/doi.org\/10.1002\/stv.430","journal-title":"Softw Test Verif Reliab"},{"issue":"9","key":"10138_CR65","doi-asserted-by":"publisher","first-page":"898","DOI":"10.1109\/TSE.2018.2809496","volume":"45","author":"J Zhang","year":"2019","unstructured":"Zhang J, Zhang L, Harman M, Hao D, Jia Y, Zhang L (2019) Predictive mutation testing. IEEE Trans Softw Eng 45(9):898\u2013918. https:\/\/doi.org\/10.1109\/TSE.2018.2809496","journal-title":"IEEE Trans Softw Eng"},{"key":"10138_CR66","doi-asserted-by":"crossref","unstructured":"Zhang L, Hao D, Zhang L, Rothermel G, Mei H (2013) Bridging the gap between the total and additional test-case prioritization strategies. In: 2013 35th International Conference on Software Engineering (ICSE). IEEE, pp 192\u2013201","DOI":"10.1109\/ICSE.2013.6606565"},{"key":"10138_CR67","doi-asserted-by":"crossref","unstructured":"Zhang L, Marinov D, Zhang L, Khurshid S (2012) Regression mutation testing. In: International Symposium on Software Testing and Analysis, ISSTA 2012, Minneapolis, pp 331\u2013341","DOI":"10.1145\/2338965.2336793"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10138-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10138-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10138-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T05:11:25Z","timestamp":1658812285000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10138-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,30]]},"references-count":67,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,9]]}},"alternative-id":["10138"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10138-1","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,30]]},"assertion":[{"value":"22 February 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 May 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}],"article-number":"114"}}