{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T17:47:51Z","timestamp":1781372871657,"version":"3.54.1"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CCF-2129388, CCF-1910067, CCF-2129446"],"award-info":[{"award-number":["CCF-2129388, CCF-1910067, CCF-2129446"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>\n                    While mutation testing has been a topic of academic interest for decades, it is only recently that \u201creal-world\u201d developers, including industry leaders such as Google and Meta, have adopted mutation testing. We propose a new approach to the development of mutation testing tools, and in particular the core challenge of\n                    <jats:italic toggle=\"yes\">generating mutants<\/jats:italic>\n                    . Current practice tends towards two limited approaches to mutation generation: mutants are either (1) generated at the bytecode\/IR level, and thus neither human readable nor adaptable to source-level features of languages or projects, or (2) generated at the source level by language-specific tools that are hard to write and maintain, and in fact are often abandoned by both developers and users. We propose instead that source-level mutation generation is a special case of\n                    <jats:italic toggle=\"yes\">program transformation<\/jats:italic>\n                    in general, and that adopting this approach allows for a single tool that can effectively generate source-level mutants for essentially\n                    <jats:italic toggle=\"yes\">any<\/jats:italic>\n                    programming language. Furthermore, by using\n                    <jats:italic toggle=\"yes\">parser parser combinators<\/jats:italic>\n                    many of the seeming limitations of an any-language approach can be overcome, without the need to parse specific languages. We compare this new approach to mutation to existing tools, and demonstrate the advantages of using parser parser combinators to improve on a regular-expression based approach to generation. Finally, we show that our approach can provide effective mutant generation even for a language for which it lacks any language-specific operators, and that is not very similar in syntax to any language it has been applied to previously.\n                  <\/jats:p>","DOI":"10.1145\/3643756","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"654-674","source":"Crossref","is-referenced-by-count":8,"title":["Syntax Is All You Need: A Universal-Language Approach to Mutant Generation"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4225-7453","authenticated-orcid":false,"given":"Sourav","family":"Deb","sequence":"first","affiliation":[{"name":"Northern Arizona University, Flagstaff, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7583-6713","authenticated-orcid":false,"given":"Kush","family":"Jain","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2912-191X","authenticated-orcid":false,"given":"Rijnard","family":"van Tonder","sequence":"additional","affiliation":[{"name":"Mysten Labs, Palo Alto, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3931-060X","authenticated-orcid":false,"given":"Claire","family":"Le Goues","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0273-4668","authenticated-orcid":false,"given":"Alex","family":"Groce","sequence":"additional","affiliation":[{"name":"Northern Arizona University, Flagstaff, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"[n. d.]. How fast (or slow) mutation testing really is? https:\/\/solidsoft.wordpress.com\/2017\/09\/19\/how-fast-or-slow-mutation-testing-really-is\/."},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3526099"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781316771273"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00036"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/2976002.2976003"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2462173"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/ICST.2013.20","article-title":"Empirical evaluation of the statement deletion mutation operator.","author":"Deng Lin","year":"2013","unstructured":"LinDengJeffOffuttNanLi. 2013. Empirical evaluation of the statement deletion mutation operator.In 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation. IEEE, 84-93.","journal-title":"2013 IEEE Sixth International Conference on Software Testing, Verification and Validation"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10270-020-00827-0"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-016-9317-7"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2017.2705662"},{"key":"e_1_3_1_12_2","article-title":"On The Limits of Mutation Reduction Strategies.","author":"Gopinath Rahul","year":"2016","unstructured":"RahulGopinathIftekharMohammad Amin AlipourCarlosAhmedAlexJensenGroce. 2016. On The Limits of Mutation Reduction Strategies.In 2016 IEEE\/ACM 38th International Conference on Software Engineering (ICSE). 511-522.","journal-title":"2016 IEEE\/ACM 38th International Conference on Software Engineering (ICSE)"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS54544.2021.00032"},{"issue":"4","key":"e_1_3_1_14_2","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1007\/s10515-018-0240-y","article-title":"How verified (or tested) is my code? falsification-driven verification and testing.","volume":"25","author":"Groce Alex","year":"2018","unstructured":"AlexGroceIftekharAhmedCarlosJensenPaul EMckenneyJosieHolmes. 2018. How verified (or tested) is my code? falsification-driven verification and testing. Automated Software Engineering Journal 25, 4 (2018), 917-960.","journal-title":"Automated Software Engineering Journal"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183440.3183485"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","DOI":"10.14722\/fuzzing.2022.23002","article-title":"Registered report: First, fuzz the mutants.","volume":"22","author":"Groce Alex","year":"2022","unstructured":"AlexGroceClaire LeGoutamkumar Tulajappa KalburgiKushGouesRahulJainGopinath. 2022. Registered report: First, fuzz the mutants.In International Fuzzing Workshop, ser. FUZZING, Vol. 22.","journal-title":"International Fuzzing Workshop, ser. FUZZING"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3497776.3517765"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00021"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.14"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","DOI":"10.1145\/3510003.3510187","article-title":"Prioritizing Mutants to Guide Mutation Testing.","author":"Kaufman Samuel J.","year":"2022","unstructured":"Samuel J.Kaufman, RyanFeatherman, JustinAlvin, BobKurtz, PaulAmmann, and Ren\u00e9Just. 2022. Prioritizing Mutants to Guide Mutation Testing.In Proceedings of the International Conference on Software Engineering (ICSE).","journal-title":"Proceedings of the International Conference on Software Engineering (ICSE)"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCAM.2016.28"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628052"},{"issue":"4","key":"e_1_3_1_23_2","first-page":"34","article-title":"Hints on test data selection: Help for the practicing programmer.","volume":"11","author":"Lipton Richard J.","year":"1978","unstructured":"Richard J.Lipton, Richard ADeMillo, and Frederick GSayward. 1978. Hints on test data selection: Help for the practicing programmer. Computer 11, 4 (1978), 34-41.","journal-title":"Computer"},{"key":"e_1_3_1_24_2","unstructured":"Aditya P Mathur. 2012. Foundations of Software Testing. Addison-Wesley."},{"key":"e_1_3_1_25_2","unstructured":"GiorgioNatili. [n. d.]. Mutation Testing at Scale. https:\/\/slides.com\/giorgionatili\/mutation-testing-at-scale."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/227607.227610"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/227607.227610"},{"issue":"3","key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1002\/(SICI)1099-1689(199709)7:3<165::AID-STVR143>3.0.CO;2-U","article-title":"Automatically detecting equivalent mutants and infeasible paths.","volume":"7","author":"Offutt A Jefferson","year":"1997","unstructured":"A JeffersonOffutt and JiePan. 1997. Automatically detecting equivalent mutants and infeasible paths. Software testing, verification and reliability 7, 3 (1997), 165-192.","journal-title":"Software testing, verification and reliability"},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1007\/978-1-4757-5939-6_7","article-title":"Mutation 2000: Uniting the orthogonal.","author":"Offutt A Jefferson","year":"2001","unstructured":"A JeffersonOffutt and Roland HUntch. 2001. Mutation 2000: Uniting the orthogonal.In Mutation testing for the new century. Springer, 34-44.","journal-title":"Mutation testing for the new century"},{"key":"e_1_3_1_30_2","article-title":"Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple Fast and Effective Equivalent Mutant Detection Technique.","author":"Papadakis Mike","year":"2015","unstructured":"MikePapadakis, YueJia, MarkHarman, and Yves LeTraon. 2015. Trivial Compiler Equivalence: A Large Scale Empirical Study of a Simple Fast and Effective Equivalent Mutant Detection Technique.In International Conference on Software Engineering.","journal-title":"International Conference on Software Engineering"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183521"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2018.00027"},{"key":"e_1_3_1_33_2","article-title":"A systematic literature review of techniques and metrics to reduce the cost of mutation testing.","volume":"157","author":"Pizzoleto Alessandro Viola","year":"2019","unstructured":"Alessandro ViolaPizzoleto, Fabiano CutigiFerrari, JeffOffutt, LeoFernandes, and MarcioRibeiro. 2019. A systematic literature review of techniques and metrics to reduce the cost of mutation testing. Journal of Systems and Software 157 (2019), 110388.","journal-title":"Journal of Systems and Software"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1473"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/174146.154265"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","DOI":"10.1145\/3314221.3314589","article-title":"Lightweight Multi-Language Syntax Transformation with Parser Parser Combinators.","author":"Tonder Rijnard van","year":"2019","unstructured":"Rijnard vanTonder and Claire LeGoues. 2019. Lightweight Multi-Language Syntax Transformation with Parser Parser Combinators.In Conference on Programming language Design and Implementation (PLDI \u201919).","journal-title":"Conference on Programming language Design and Implementation (PLDI \u201919)"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931038"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643756","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643756","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643756","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T08:07:44Z","timestamp":1770192464000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643756"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":37,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3643756"],"URL":"https:\/\/doi.org\/10.1145\/3643756","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}