{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T09:15:44Z","timestamp":1783242944405,"version":"3.54.6"},"publisher-location":"New York, NY, USA","reference-count":39,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T00:00:00Z","timestamp":1783209600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"No Funding","award":["none"],"award-info":[{"award-number":["none"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,7,5]]},"DOI":"10.1145\/3805760.3814933","type":"proceedings-article","created":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T08:46:13Z","timestamp":1783241173000},"page":"397-405","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["SecMutBench: Evaluating LLM-Generated Security Tests via Mutation-Based Vulnerability Detection"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-3468-6091","authenticated-orcid":false,"given":"Mariam","family":"ALMutairi","sequence":"first","affiliation":[{"name":"Virginia Polytechnic Institute and State University, alexandria, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3675-0199","authenticated-orcid":false,"given":"Chang-Tien","family":"Lu","sequence":"additional","affiliation":[{"name":"Virginia Polytechnic Institute and State University, Alexandria, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,5]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","unstructured":"Mariam ALMutairi and Chang-Tien Lu. 2026. SecMutBench: Mutation-Based Benchmark for LLM Security Test Generation. doi:10.5281\/zenodo.20045643 10.5281\/zenodo.20045643","DOI":"10.5281\/zenodo.20045643"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3613099"},{"key":"e_1_3_2_1_3_1","volume-title":"2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 53-56","author":"Ami Amit Seal","year":"2021","unstructured":"Amit Seal Ami, Kaushal Kafle, Adwait Nadkarni, Denys Poshyvanyk, and Kevin Moran. 2021. \ud835\udf07se: Mutation-based evaluation of security-focused static analysis tools for android. In 2021 IEEE\/ACM 43rd International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 53-56."},{"key":"e_1_3_2_1_4_1","unstructured":"Manish Bhatt Sahana Chennabasappa Cyrus Nikolaidis Shengye Wan Ivan Evtimov Dominik Gabi Daniel Song Faizan Ahmad Cornelius Aschermann Lorenzo Fontana et al. 2023. Purple llama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.04724 (2023)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Elena Bruches Vadim Alperovich Dari Baturova Roman Derunets Daniil Grebenkin Georgy Mkrtchyan Oleg Sedukhin Mikhail Klementev Ivan Bondarenko Nikolay Bushkov et al. 2026. TAM-Eval: Evaluating LLMs for Automated Unit Test Maintenance. arXiv preprint arXiv:2601.18241 (2026).","DOI":"10.1109\/SANER-C67878.2026.00057"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/AIware69974.2025.00021"},{"key":"e_1_3_2_1_7_1","volume-title":"Cybercrime to Cost the World $10.5 Trillion Annually by","author":"Ventures Cybersecurity","year":"2025","unstructured":"Cybersecurity Ventures. 2025. Cybercrime to Cost the World $10.5 Trillion Annually by 2025. https:\/\/cybersecurityventures.com\/."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2024.107468"},{"key":"e_1_3_2_1_9_1","volume-title":"2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 160-169","author":"Degiovanni Renzo","year":"2022","unstructured":"Renzo Degiovanni and Mike Papadakis. 2022. \ud835\udf07Bert: Mutation Testing using Pre-Trained Language Models. In 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 160-169."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1978.218136"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598067"},{"key":"e_1_3_2_1_12_1","first-page":"156","article-title":"Experimental evaluation of mutation testing approaches to python programs. In 2014 IEEE seventh international conference on software testing, verification and validation workshops","author":"Derezinska Anna","year":"2014","unstructured":"Anna Derezinska and Konrad Halas. 2014. Experimental evaluation of mutation testing approaches to python programs. In 2014 IEEE seventh international conference on software testing, verification and validation workshops. IEEE, 156-164.","journal-title":"IEEE"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1002\/qre.480"},{"key":"e_1_3_2_1_14_1","volume-title":"32nd USENIX Security Symposium (USENIX Security 23)","author":"G\u00f6rz Philipp","year":"2023","unstructured":"Philipp G\u00f6rz, Bj\u00f6rn Mathis, Keno Hassler, Emre G\u00fcler, Thorsten Holz, Andreas Zeller, and Rahul Gopinath. 2023. Systematic assessment of fuzzers using mutation analysis. In 32nd USENIX Security Symposium (USENIX Security 23). 4535- 4552."},{"key":"e_1_3_2_1_15_1","volume-title":"Dwyer","author":"Hossain Soneya Binta","year":"2022","unstructured":"Soneya Binta Hossain and Matthew B. Dwyer. 2022. A Brief Survey on Oraclebased Test Adequacy Metrics. arXiv preprint arXiv:2212.06118 (2022)."},{"key":"e_1_3_2_1_16_1","volume-title":"Breach Report","author":"Security IBM","year":"2024","unstructured":"IBM Security. 2024. Cost of a Data Breach Report 2024. https:\/\/www.ibm.com\/re ports\/data-breach."},{"key":"e_1_3_2_1_17_1","volume-title":"An analysis and survey of the development of mutation testing","author":"Jia Yue","year":"2010","unstructured":"Yue Jia and Mark Harman. 2010. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2010), 649-678."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00085"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW.2017.24"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695318"},{"key":"e_1_3_2_1_21_1","unstructured":"Menlo Ventures. 2025. 2025 Mid-Year LLM Market Update: Foundation Model Landscape and Economics. https:\/\/menlovc.com\/perspective\/2025-mid-yearllm-market-update\/."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2009.49"},{"key":"e_1_3_2_1_23_1","volume-title":"SECCODEPLT: A Unified Evaluation Platform for Code GenAI Security Risks. In International Conference on Emerging Trends and Technologies on Intelligent Systems. Springer, 92-104","author":"Nabith Muhammed Abeed","year":"2025","unstructured":"Muhammed Abeed Nabith. 2025. SECCODEPLT: A Unified Evaluation Platform for Code GenAI Security Risks. In International Conference on Emerging Trends and Technologies on Intelligent Systems. Springer, 92-104."},{"key":"e_1_3_2_1_24_1","volume-title":"Yves Le Traon, and Mark Harman","author":"Papadakis Mike","year":"2019","unstructured":"Mike Papadakis, Marinos Kintis, Jie Zhang, Yue Jia, Yves Le Traon, and Mark Harman. 2019. Mutation testing advances: an analysis and survey. In Advances in computers. Vol. 112. Elsevier, 275-378."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610721"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/LLM4Code66737.2025.00009"},{"key":"e_1_3_2_1_27_1","volume-title":"Bandit: A Tool Designed to Find Common Security Issues in Python Code. https:\/\/bandit.readthedocs.io\/.","author":"CQA.","year":"2024","unstructured":"PyCQA. 2024. Bandit: A Tool Designed to Find Common Security Issues in Python Code. https:\/\/bandit.readthedocs.io\/."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3334955"},{"key":"e_1_3_2_1_29_1","volume-title":"Semgrep: Lightweight Static Analysis for Many Languages. https:\/\/semgrep.dev\/.","author":"Semgrep Inc.","year":"2024","unstructured":"Semgrep Inc. 2024. Semgrep: Lightweight Static Analysis for Many Languages. https:\/\/semgrep.dev\/."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2008.33"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/HASE.2008.8"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWSESS.2009.5068458"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3549035.3561184"},{"key":"e_1_3_2_1_34_1","unstructured":"Stack Overflow. 2025. 2025 Developer Survey. https:\/\/survey.stackoverflow.co\/2 025\/."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00084"},{"key":"e_1_3_2_1_36_1","volume-title":"Mutation- Guided Unit Test Generation with a Large Language Model. arXiv preprint arXiv:2506.02954","author":"Wang Guancheng","year":"2025","unstructured":"Guancheng Wang, Qinghua Xu, Lionel C Briand, and Kui Liu. 2025. Mutation- Guided Unit Test Generation with a Large Language Model. arXiv preprint arXiv:2506.02954 (2025)."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-36103-0_48"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3660783"},{"key":"e_1_3_2_1_39_1","volume-title":"Test suite effectiveness metric evaluation: what do we know and what should we do? arXiv preprint arXiv:2204.09165","author":"Zhang Peng","year":"2022","unstructured":"Peng Zhang, Yang Wang, Xutong Liu, Yibiao Yang, Yanhui Li, Lin Chen, Ziyuan Wang, Chang-ai Sun, and Yuming Zhou. 2022. Test suite effectiveness metric evaluation: what do we know and what should we do? arXiv preprint arXiv:2204.09165 (2022)."}],"event":{"name":"AIware '26: 3rd ACM International Conference on AI-Powered Software","location":"Montreal QC Canada","acronym":"AIware '26","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 3rd ACM International Conference on AI-Powered Software"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3805760.3814933","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T08:46:37Z","timestamp":1783241197000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3805760.3814933"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,5]]},"references-count":39,"alternative-id":["10.1145\/3805760.3814933","10.1145\/3805760"],"URL":"https:\/\/doi.org\/10.1145\/3805760.3814933","relation":{},"subject":[],"published":{"date-parts":[[2026,7,5]]},"assertion":[{"value":"2026-07-05","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}