{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T17:44:27Z","timestamp":1772041467609,"version":"3.50.1"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,6,4]],"date-time":"2024-06-04T00:00:00Z","timestamp":1717459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Union Horizon 2020 research and innovation program","award":["101021727"],"award-info":[{"award-number":["101021727"]}]},{"name":"France FUI CAESAR project and from the Chateaubriand Fellowship Program of the Embassy of France in the United States"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to guide the fuzzing process. The fuzzer retains inputs for further mutation only if branch coverage is increased. However, branch coverage only provides a shallow sampling of program behaviors and hence may discard interesting inputs to mutate. This work aims to take advantage of the large body of research in defining finer-grained code coverage metrics (such as control-flow, data-flow, or mutation coverage) and to evaluate how fuzzing performance is impacted when using these metrics to select interesting inputs for mutation. We propose to make branch coverage-based fuzzers support most fine-grained coverage metrics out of the box (i.e., without changing fuzzer internals). We achieve this by making the test objectives defined by these metrics (such as conditions to activate or mutants to kill) explicit as new branches in the target program. Fuzzing such a modified target is then equivalent to fuzzing the original target, but the fuzzer will also retain inputs covering the additional metric objectives for mutation. In addition, all the fuzzer mechanisms to penetrate hard-to-cover branches will help in covering the additional metric objectives. We use this approach to evaluate the impact of supporting two fine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of two state-of-the-art fuzzers (AFL++ and QSYM) with the standard LAVA-M and MAGMA benchmarks. This evaluation suggests that our mechanism for runtime fuzzer guidance, where the fuzzed code is instrumented with additional branches, is effective and could be leveraged to encode guidance from human users or static analyzers. Our results also show that the impact of fine-grained metrics over fuzzing performance is hard to predict before fuzzing and most of the time either neutral or negative. As a consequence, we do not recommend using them to guide fuzzers, except maybe in some possibly favorable circumstances yet to be investigated, like for limited parts of the code or to complement classical fuzzing campaigns.<\/jats:p>","DOI":"10.1145\/3587158","type":"journal-article","created":{"date-parts":[[2023,3,14]],"date-time":"2023-03-14T12:15:44Z","timestamp":1678796144000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Fine-grained Coverage-based Fuzzing"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9211-6224","authenticated-orcid":false,"given":"Wei-Cheng","family":"Wu","sequence":"first","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France and University of Southern California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2309-9096","authenticated-orcid":false,"given":"Bernard","family":"Nongpoh","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4709-959X","authenticated-orcid":false,"given":"Marwan","family":"Nour","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8087-0537","authenticated-orcid":false,"given":"Micha\u00ebl","family":"Marcozzi","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6509-3506","authenticated-orcid":false,"given":"S\u00e9bastien","family":"Bardin","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7698-8041","authenticated-orcid":false,"given":"Christophe","family":"Hauser","sequence":"additional","affiliation":[{"name":"Universit\u00e9 Paris-Saclay, CEA, List, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,6,4]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/96267.96279"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106295"},{"key":"e_1_3_3_4_2","doi-asserted-by":"crossref","unstructured":"Sanjay Rawat Vivek Jain Ashish Kumar Lucian Cojocar Cristiano Giuffrida and Herbert Bos. 2017. VUzzer: Application-aware evolutionary fuzzing. In Proceedings of the Network and Distributed System Security Symposium (NDSS\u201917).","DOI":"10.14722\/ndss.2017.23404"},{"key":"e_1_3_3_5_2","unstructured":"American fuzzy lop - a security-oriented fuzzer. n.d. Received December 12 2021 from https:\/\/github.com\/google\/AFL."},{"key":"e_1_3_3_6_2","volume-title":"14th USENIX Workshop on Offensive Technologies (WOOT\u201920)","author":"Fioraldi Andrea","year":"2020","unstructured":"Andrea Fioraldi, Dominik Maier, Heiko Ei\u00dffeldt, and Marc Heuse. 2020. AFL++: Combining incremental steps of fuzzing research. In 14th USENIX Workshop on Offensive Technologies (WOOT\u201920). USENIX Association."},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23371"},{"key":"e_1_3_3_8_2","volume-title":"USENIX Conference on Operating Systems Design and Implementation (OSDI\u201908)","author":"Cadar Cristian","unstructured":"Cristian Cadar, Daniel Dunbar, and Dawson Engler. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In USENIX Conference on Operating Systems Design and Implementation (OSDI\u201908)."},{"key":"e_1_3_3_9_2","volume-title":"Proceedings of the 27th USENIX Conference on Security Symposium (SEC\u201918)","author":"Yun Insu","year":"2018","unstructured":"Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM: A practical concolic execution engine tailored for hybrid fuzzing. In Proceedings of the 27th USENIX Conference on Security Symposium (SEC\u201918). USENIX Association."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809163"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360600"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2019.00015"},{"key":"e_1_3_3_13_2","volume-title":"Proceedings of the 29th USENIX Conference on Security Symposium (SEC\u201920)","author":"\u00d6sterlund Sebastian","year":"2020","unstructured":"Sebastian \u00d6sterlund, Kaveh Razavi, Herbert Bos, and Cristiano Giuffrida. 2020. ParmeSan: Sanitizer-guided greybox fuzzing. In Proceedings of the 29th USENIX Conference on Security Symposium (SEC\u201920). USENIX Association, Article 129, 18 pages."},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2021.102641"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-020-00567-y"},{"key":"e_1_3_3_16_2","article-title":"Subsumption of condition coverage techniques by mutation testing","author":"Offutt A. Jefferson","year":"1996","unstructured":"A. Jefferson Offutt and Jeffrey M. Voas. 1996. Subsumption of condition coverage techniques by mutation testing. Department of Information and Software Systems Engineering, George Mason University, Tech. Rep. ISSE-TR-96-100.","journal-title":"Department of Information and Software Systems Engineering, George Mason University, Tech. Rep. ISSE-TR-96-100"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.5555\/2161638"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","unstructured":"Brendan Dolan-Gavitt Patrick Hulin Engin Kirda Tim Leek Andrea Mambretti Wil Robertson Frederick Ulrich and Ryan Whelan. 2016. LAVA: Large-scale automated vulnerability addition. DOI:10.1109\/SP.2016.15","DOI":"10.1109\/SP.2016.15"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428334"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3545258.3545285"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP53844.2022.00026"},{"key":"e_1_3_3_22_2","unstructured":"Adrian Herrera Mathias Payer and Antony L. Hosking. n.d. Registered report: DATAFLOW. In Proceedings of the 1st International Fuzzing Workshop (FUZZING\u201922)."},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2946563"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1186\/s42400-018-0002-y"},{"key":"e_1_3_3_25_2","doi-asserted-by":"crossref","unstructured":"Nick Stephens John Grosen Christopher Salls Andrew Dutcher Ruoyu Wang Jacopo Corbetta Yan Shoshitaishvili Christopher Kruegel and Giovanni Vigna. 2016. Driller: Augmenting fuzzing through selective symbolic execution. In Proceedings of the Network and Distributed System Security Symposium (NDSS\u201916).","DOI":"10.14722\/ndss.2016.23368"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2003.1251034"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1017\/9781316771273"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2010.5609672"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/C-M.1978.218136"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.286422"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180191"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.103"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-09099-3_4"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2014.30"},{"key":"e_1_3_3_35_2","volume-title":"2017 IEEE International Conference on Software Testing, Verification and Validation (ICST\u201917)","author":"Marcozzi Micha\u00ebl","unstructured":"Micha\u00ebl Marcozzi, Micka\u00ebl Delahaye, S\u00e9bastien Bardin, Nikolai Kosmatov, and Virgile Prevosto. Generic and effective specification of structural test objectives. In 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST\u201917)."},{"key":"e_1_3_3_36_2","volume-title":"Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO\u201904)","author":"Lattner Chris","unstructured":"Chris Lattner and Vikram Adve. LLVM: A compilation framework for lifelong program analysis & transformation. In Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO\u201904)."},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985795"},{"key":"e_1_3_3_38_2","first-page":"1683","volume-title":"30th USENIX Security Symposium (USENIX Security\u201921)","author":"Nagy Stefan","year":"2021","unstructured":"Stefan Nagy, Anh Nguyen-Tuong, Jason D. Hiser, Jack W. Davidson, and Matthew Hicks. 2021. Breaking through binaries: Compiler-quality instrumentation for better binary-only fuzzing. In 30th USENIX Security Symposium (USENIX Security\u201921). USENIX Association, 1683\u20131700. https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/nagy."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP40000.2020.00117"},{"key":"e_1_3_3_40_2","volume-title":"Proceedings of the 22nd International Symposium on Research in Attacks, Intrusions and Defenses (RAID\u201919)","author":"Wang Jinghan","unstructured":"Jinghan Wang, Yue Duan, Wei Song, Heng Yin, and Chengyu Song. Be sensitive and collaborative: Analyzing impact of coverage metrics in greybox fuzzing. In Proceedings of the 22nd International Symposium on Research in Attacks, Intrusions and Defenses (RAID\u201919)."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.24486"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380421"},{"key":"e_1_3_3_43_2","volume-title":"30th USENIX Security Symposium (USENIX Security\u201921)","author":"Fioraldi Andrea","year":"2021","unstructured":"Andrea Fioraldi, Daniele Cono D\u2019Elia, and Davide Balzarotti. 2021. The use of likely invariants as feedback for fuzzers. In 30th USENIX Security Symposium (USENIX Security\u201921)."},{"key":"e_1_3_3_44_2","unstructured":"laf-intel. n.d. Retrieved August 16 2016 from https:\/\/lafintel.wordpress.com\/."},{"key":"e_1_3_3_45_2","unstructured":"Isabella Laybourn Vasudev Vikram Rafaello Sanna Ao Li and Rohan Padhye. 2022. Guiding Greybox Fuzzing with Mutation Testing. Program Analysis Software Testing and Applications Laboratory Carnegie Mellon University Technical Report. (2022)."},{"key":"e_1_3_3_46_2","unstructured":"Alex Groce Goutamkumar Tulajappa Kalburgi Claire Le Goues Kush Jain and Rahul Gopinath. n.d. Registered report: First fuzz the mutants. In Proceedings of the 1st International Fuzzing Workshop (FUZZING\u201922)."},{"key":"e_1_3_3_47_2","unstructured":"Rahul Gopinath Philipp G\u00f6rz and Alex Groce. 2022. Mutation Analysis: Answering the Fuzzing Challenge. arXiv preprint arXiv:2201.11303 (2022)."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587158","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3587158","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:47:15Z","timestamp":1750178835000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3587158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,4]]},"references-count":46,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3587158"],"URL":"https:\/\/doi.org\/10.1145\/3587158","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,4]]},"assertion":[{"value":"2022-06-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}