{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T15:04:27Z","timestamp":1789743867906,"version":"4.0.1"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T00:00:00Z","timestamp":1789603200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"UKRI EPSRC","award":["EP\/W524360\/1, EP\/X024539\/1"],"award-info":[{"award-number":["EP\/W524360\/1, EP\/X024539\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,10,31]]},"abstract":"<jats:p>There is a lack of serious tooling for mutation analysis for Rust, a safety-focused systems programming language seeing increased adoption across the industry. As such, the testing technique has not been widely used on programs written in the language as of yet. Without robust mutation analysis, Rust developers cannot determine test thoroughness.<\/jats:p>\n                  <jats:p>In response to this challenge, we designed a mutation analysis pipeline for Rust, which overcomes the challenges of generating valid mutants caused by the strictness of the language.<\/jats:p>\n                  <jats:p>Our approach accounts for Rust\u2019s distinction between safe and unsafe operations, ensuring that safe mutations of valid Rust programs\u2014those with only valid unsafe code sections\u2014can be safely evaluated within the same process, without the potential for crashes or other undefined behavior invalidating the mutation analysis.<\/jats:p>\n                  <jats:p>We introduce mutation batching, our novel technique for efficiently evaluating multiple mutations simultaneously, while guaranteeing they do not interact. Batching maximizes thread usage, by executing significantly more test cases in parallel. As batching is NP-hard, we present multiple fast approximation algorithms for grouping mutations.<\/jats:p>\n                  <jats:p>\n                    We implemented our techniques into a mutation analysis tool,\n                    <jats:sans-serif>mutest-rs<\/jats:sans-serif>\n                    , which we used in our empirical evaluation on a diverse set of 22 Rust libraries and programs. We found that mutation batching reduces the overall runtime of mutation analysis by up to\n                    <jats:inline-formula content-type=\"math\/tex\">\n                      <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(52.3\\%\\)<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    and also saving 73.8 seconds in one case, and that unsafe mutations are detected at a lower rate, of up to\n                    <jats:inline-formula content-type=\"math\/tex\">\n                      <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(53.4\\%\\)<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    . Our mutation analysis tool is available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/mutest.rs\">https:\/\/mutest.rs<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3787851","type":"journal-article","created":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T14:49:24Z","timestamp":1768834164000},"page":"1-54","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A Comprehensive Empirical and Theoretical Analysis of Batching Algorithms for Efficient, Safe, Parallel Mutation Analysis in Rust"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4173-8562","authenticated-orcid":false,"given":"Zal\u00e1n","family":"L\u00e9vai","sequence":"first","affiliation":[{"name":"The University of Sheffield, Sheffield, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0840-6449","authenticated-orcid":false,"given":"Donghwan","family":"Shin","sequence":"additional","affiliation":[{"name":"The University of Sheffield, Sheffield, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9137-7433","authenticated-orcid":false,"given":"Phil","family":"McMinn","sequence":"additional","affiliation":[{"name":"The University of Sheffield, Sheffield, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,9,17]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.21236\/ADA076575"},{"key":"e_1_3_2_3_2","unstructured":"Abhishek Arya Caleb Brown Rob Pike and The Open Source Security Foundation. 2023. Open Source Project Criticality Score. The Open Source Security Foundation. Retrieved from https:\/\/github.com\/ossf\/criticality_score"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428204"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jda.2013.11.003"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/AST52587.2021.00021"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2450"},{"key":"e_1_3_2_8_2","unstructured":"Andre Bogus. 2022. Mutagen. Retrieved from https:\/\/github.com\/llogiq\/mutagen"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/362342.362367"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3341180"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2948707"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1978.218136"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2018.00024"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2006.92"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338945"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380413"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/1831708.1831728"},{"key":"e_1_3_2_18_2","volume-title":"Computers and Intractibility: A Guide to the Theory of NP-Completeness","author":"Garey Michael","year":"1979","unstructured":"Michael Garey and David Johnson. 1979. Computers and Intractibility: A Guide to the Theory of NP-Completeness. W. H. Freeman, New York, NY, USA."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2889160.2892655"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1412228.1412236"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSCE.2012.6487127"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1977.231145"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3240482"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00021"},{"key":"e_1_3_2_25_2","unstructured":"Falk H\u00fcffner. 2021. Falk-Hueffner\/Clique-Cover. Retrieved from https:\/\/github.com\/falk-hueffner\/clique-cover"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCAM.2008.36"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2009.04.016"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.62"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628053"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610388"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1982595.1982606"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4684-2001-2_9"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.4380210704"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2014.20"},{"key":"e_1_3_2_35_2","unstructured":"Zal\u00e1n L\u00e9vai. 2025. mutest-rs. Retrieved from https:\/\/github.com\/zalanlevai\/mutest-rs"},{"key":"e_1_3_2_36_2","unstructured":"Zal\u00e1n L\u00e9vai. 2025. Replication Package. Retrieved from https:\/\/github.com\/rust-mutation-testing\/publications#mutation-batching-article-rep-pak"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST57152.2023.00014"},{"key":"e_1_3_2_38_2","volume-title":"Fault Diagnosis of Computer Programs","author":"Lipton Richard J.","year":"1971","unstructured":"Richard J. Lipton. 1971. Fault Diagnosis of Computer Programs. Student Report. Carnegie Mellon University."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2015.7381798"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1534"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2014.28"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-5939-6_7"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.103"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1509"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3476105"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.07.100"},{"key":"e_1_3_2_47_2","unstructured":"Martin Pool. 2025. cargo-mutants. Retrieved from https:\/\/github.com\/sourcefrog\/cargo-mutants"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/1595696.1595750"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3604919"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1695"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.00023"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453483.3454084"},{"key":"e_1_3_2_53_2","unstructured":"The Open Source Security Foundation. 2022. Criticality Score. Retrieved from https:\/\/commondatastorage.googleapis.com\/ossf-criticality-score\/index.html"},{"key":"e_1_3_2_54_2","unstructured":"The Rust Project Developers. 2024. Rust: Production Users. Retrieved from https:\/\/www.rust-lang.org\/production\/users"},{"key":"e_1_3_2_55_2","unstructured":"The Rust Project Developers. 2025. Guide to Rustc Development: Monomorphization. Retrieved from https:\/\/rustc-dev-guide.rust-lang.org\/backend\/monomorph.html"},{"key":"e_1_3_2_56_2","unstructured":"The Rust Project Developers. 2025. Miri. Retrieved from https:\/\/github.com\/rust-lang\/miri"},{"key":"e_1_3_2_57_2","unstructured":"The Rust Project Developers. 2025. The Rustonomicon: How Safe and Unsafe Interact. Retrieved from https:\/\/doc.rust-lang.org\/nomicon\/safe-unsafe-meaning.html"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2016.18"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/503720.503749"},{"key":"e_1_3_2_60_2","unstructured":"Roland H. Untch. 1995. Schema-Based Mutation Analysis: A New Test Data Adequacy Assessment Method. Ph.D. Dissertation. Clemson University Clemson SC. Retrieved from https:\/\/www.proquest.com\/docview\/304173339\/abstract\/F975B52F62924781PQ\/1"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/154183.154265"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1002\/stvr.1865"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/2483760.2483782"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3787851","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T14:16:29Z","timestamp":1789740989000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3787851"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,17]]},"references-count":62,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2026,10,31]]}},"alternative-id":["10.1145\/3787851"],"URL":"https:\/\/doi.org\/10.1145\/3787851","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,17]]},"assertion":[{"value":"2024-10-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-09-17","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}