{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T22:15:36Z","timestamp":1766182536466,"version":"3.48.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"National Science Foundation","award":["CCF-2120955"],"award-info":[{"award-number":["CCF-2120955"]}]},{"name":"Defense Advanced Research Projects Agency (DARPA), and Naval Information Warfare Center Pacific","award":["NN66001-22-C-4027"],"award-info":[{"award-number":["NN66001-22-C-4027"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,1,31]]},"abstract":"<jats:p>\n                    Parametric generators combine coverage-guided and generator-based fuzzing for testing programs requiring structured inputs. They function as decoders that transform arbitrary byte sequences into structured inputs, allowing mutations on byte sequences to map directly to mutations on structured inputs, without requiring specialized mutators. However, this technique is prone to the\n                    <jats:italic toggle=\"yes\">havoc effect<\/jats:italic>\n                    , where small mutations on the byte sequence cause large, destructive mutations to the structured input. This article investigates the paradoxical nature of the havoc effect for generator-based fuzzing in Java. In particular, we measure mutation characteristics and confirm the existence of the havoc effect, as well as scenarios where it may be more detrimental. Our evaluation across seven real-world Java applications compares various techniques that perform context-aware, finer-grained mutations on parametric byte sequences, such as JQF-EI, BeDivFuzz, and Zeugma. We find that these techniques exhibit better control over input mutations and consistently reduce the havoc effect compared to our coverage-guided fuzzer baseline Zest. While we find that context-aware mutation approaches can achieve significantly higher code coverage, we see that destructive mutations still play a valuable role in discovering inputs that increase code coverage. Specialized mutation strategies, while effective, impose substantial computational overhead\u2014revealing practical tradeoffs in mitigating the havoc effect.\n                  <\/jats:p>","DOI":"10.1145\/3742894","type":"journal-article","created":{"date-parts":[[2025,6,6]],"date-time":"2025-06-06T12:50:16Z","timestamp":1749214216000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["The Havoc Paradox in Generator-Based Fuzzing"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3189-7079","authenticated-orcid":false,"given":"Ao","family":"Li","sequence":"first","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2533-139X","authenticated-orcid":false,"given":"Madonna","family":"Huang","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, British Columbia, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7093-910X","authenticated-orcid":false,"given":"Vasudev","family":"Vikram","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9610-8520","authenticated-orcid":false,"given":"Caroline","family":"Lemieux","sequence":"additional","affiliation":[{"name":"The University of British Columbia, Vancouver, British Columbia, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4939-033X","authenticated-orcid":false,"given":"Rohan","family":"Padhye","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, Pennsylvania, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,12,19]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"American Fuzzy Lop. 2024. Retrieved December 15 2024 from https:\/\/github.com\/google\/AFL"},{"key":"e_1_3_3_3_2","unstructured":"Apache Ant Is a Java-based Build Tool. 2024. Retrieved from https:\/\/github.com\/apache\/ant"},{"key":"e_1_3_3_4_2","unstructured":"Cargo-fuzz. 2024. Retrieved December 15 2024 from https:\/\/github.com\/rust-fuzz\/cargo-fuzz"},{"key":"e_1_3_3_5_2","unstructured":"Closure Compiler. 2024. Retrieved December 15 2024 from https:\/\/developers.google.com\/closure\/compiler"},{"key":"e_1_3_3_6_2","unstructured":"GSON: A Java Serialization\/Deserialization Library to Convert Java Objects into JSON and Back. 2024. Retrieved from https:\/\/github.com\/google\/gson"},{"key":"e_1_3_3_7_2","unstructured":"Jackson Project Home @github. 2024. Retrieved from https:\/\/github.com\/FasterXML\/jackson"},{"key":"e_1_3_3_8_2","unstructured":"libFuzzer \u2013 A Library for Coverage-Guided Fuzz Testing. 2024. Retrieved December 15 2024 from https:\/\/llvm.org\/docs\/LibFuzzer.html"},{"key":"e_1_3_3_9_2","unstructured":"libFuzzer \u2013 How to Split a Fuzzer-Generated Input into Several Parts. 2024. Retrieved December 15 2024 from https:\/\/github.com\/google\/fuzzing\/blob\/41d7725\/docs\/split-inputs.md"},{"key":"e_1_3_3_10_2","unstructured":"libprotobuf-Mutator. 2024. Retrieved December 15 2024 from https:\/\/github.com\/google\/libprotobuf-mutator"},{"key":"e_1_3_3_11_2","unstructured":"Rhino: JavaScript in Java. 2024. Retrieved from https:\/\/github.com\/mozilla\/rhino"},{"key":"e_1_3_3_12_2","unstructured":"Structure-Aware Fuzzing with libFuzzer. 2024. Retrieved December 15 2024 from https:\/\/github.com\/google\/fuzzing\/blob\/master\/docs\/structure-aware-fuzzing.md"},{"key":"e_1_3_3_13_2","unstructured":"Thinking about Fuzzer Evaluation. 2024. Retrieved December 15 2024 from https:\/\/addisoncrump.info\/research\/thinking-about-fuzzer-evaluation\/"},{"key":"e_1_3_3_14_2","article-title":"NAUTILUS: Fishing for deep bugs with grammars","author":"Aschermann Cornelius","year":"2019","unstructured":"Cornelius Aschermann, Tommaso Frassetto, Thorsten Holz, Patrick Jauernig, Ahmad-Reza Sadeghi, and Daniel Teuchert. 2019. NAUTILUS: Fishing for deep bugs with grammars. In Proceedings of the Conference on Network and Distributed System Security (NDSS).","journal-title":"Proceedings of the Conference on Network and Distributed System Security (NDSS)"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2785841"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510230"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/1315245.1315286"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00046"},{"key":"e_1_3_3_19_2","first-page":"237","volume-title":"In Proceedings of the 2020 35th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Chen Yiqun T.","year":"2020","unstructured":"Yiqun T. Chen, Rahul Gopinath, Anita Tadakamalla, Michael D. Ernst, Reid Holmes, Gordon Fraser, Paul Ammann, and Ren\u00e9 Just. 2020. Revisiting the relationship between fault detection, test adequacy criteria, and test set size. In Proceedings of the 2020 35th IEEE\/ACM International Conference on Automated Software Engineering (ASE), 237\u2013249."},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/351240.351266"},{"key":"e_1_3_3_21_2","volume-title":"Proceedings of the 14th USENIX Workshop on Offensive Technologies (WOOT \u201920)","author":"Fioraldi Andrea","year":"2020","unstructured":"Andrea Fioraldi, Dominik Maier, Heiko Ei\u00dffeldt, and Marc Heuse. 2020. AFL++: Combining incremental steps of fuzzing research. In Proceedings of the 14th USENIX Workshop on Offensive Technologies (WOOT \u201920). USENIX Association."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3548606.3560602"},{"key":"e_1_3_3_23_2","first-page":"2577","volume-title":"Proceedings of the 29th USENIX Security Symposium (USENIX Security 20)","author":"Gan Shuitao","year":"2020","unstructured":"Shuitao Gan, Chao Zhang, Peng Chen, Bodong Zhao, Xiaojun Qin, Dong Wu, and Zuoning Chen. 2020. GREYONE: Data flow sensitive fuzzing. In Proceedings of the 29th USENIX Security Symposium (USENIX Security 20), 2577\u20132594."},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510228"},{"key":"e_1_3_3_25_2","first-page":"445","volume-title":"Proceedings of the 21st USENIX Security Symposium (USENIX Security 12)","author":"Holler Christian","year":"2012","unstructured":"Christian Holler, Kim Herzig, and Andreas Zeller. 2012. Fuzzing with code fragments. In Proceedings of the 21st USENIX Security Symposium (USENIX Security 12). USENIX Association, 445\u2013458. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity12\/technical-sessions\/presentation\/holler"},{"key":"e_1_3_3_26_2","unstructured":"Paul Holser. n.d. junit-quickcheck: Property-based testing JUnit-style. Retrieved December 15 2024 from https:\/\/github.com\/pholser\/junit-quickcheck"},{"key":"e_1_3_3_27_2","first-page":"1","volume-title":"Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering","author":"Hough Katherine","year":"2024","unstructured":"Katherine Hough and Jonathan Bell. 2024. Crossover in parametric fuzzing. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering, 1\u201312."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/1542476.1542489"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243804"},{"key":"e_1_3_3_30_2","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1145\/3510003.3510628","volume-title":"Proceedings of the 44th International Conference on Software Engineering","author":"Kukucka James","year":"2022","unstructured":"James Kukucka, Lu\u00eds Pina, Paul Ammann, and Jonathan Bell. 2022. Confetti: Amplifying concolic guidance for fuzzers. In Proceedings of the 44th International Conference on Software Engineering, 438\u2013450."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680387"},{"key":"e_1_3_3_32_2","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1145\/3650212.3680387","volume-title":"Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Kukucka James","year":"2024","unstructured":"James Kukucka, Lu\u00eds Pina, Paul Ammann, and Jonathan Bell. 2024. An empirical examination of fuzzer mutator performance. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, 1631\u20131642."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360607"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238176"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527317"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473932"},{"issue":"2007","key":"e_1_3_3_37_2","first-page":"1","article-title":"Analysis of mutation and generation-based fuzzing. Independent security evaluators","volume":"4","author":"Miller Charlie","year":"2007","unstructured":"Charlie Miller and Zachary N. J. Peterson. 2007. Analysis of mutation and generation-based fuzzing. Independent security evaluators. Technology Reports 4, 2007 (2007), 1\u20137.","journal-title":"Technology Reports"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.5555\/1942838"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510182"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3339002"},{"key":"e_1_3_3_41_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1145\/3293882.3330576","volume-title":"Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis","author":"Rohan Padhye","year":"2019","unstructured":"Rohan Padhye, Caroline Lemieux, Koushik Sen, Mike Papadakis, and Yves Le Traon. 2019. Semantic fuzzing with zest. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis, 329\u2013340."},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.50"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3358711.3361627"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP40000.2020.00067"},{"issue":"9","key":"e_1_3_3_45_2","first-page":"1980","article-title":"Smart greybox fuzzing","volume":"7","author":"Pham Van-Thuan","year":"2019","unstructured":"Van-Thuan Pham, Marcel B\u00f6hme, and Andrew Edward Santosa, Alexandru Razvan Caciulescu, and Abhik Roychoudhury. 2019. Smart greybox fuzzing. IEEE Transactions on Software Engineering 7, 9 (2019), 1980\u20131997. DOI: https:\/\/dx.doi.org\/10.1109\/TSE.2019.2941681","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380399"},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","first-page":"1974","DOI":"10.1109\/SP54263.2024.00137","volume-title":"Proceedings of the 2024 IEEE Symposium on Security and Privacy (SP)","author":"Schloegel Moritz","year":"2024","unstructured":"Moritz Schloegel, Nils Bars, Nico Schiller, Lukas Bernhard, Tobias Scharnowski, Addison Crump, Arash Ale-Ebrahim, Nicolai Bissantz, Arius Uench, and Horsten Olz. 2024. Sok: Prudent evaluation practices for fuzzing. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 1974\u20131993."},{"key":"e_1_3_3_48_2","first-page":"2597","volume-title":"Proceedings of the 30th USENIX Security Symposium (USENIX Security 21)","author":"Schumilo Sergej","year":"2021","unstructured":"Sergej Schumilo, Cornelius Aschermann, Ali Abbasi, Simon W\u00f6r-Ner, and Thorsten Holz. 2021. Nyx: Greybox hypervisor fuzzing using fast snapshots and affine types. In Proceedings of the 30th USENIX Security Symposium (USENIX Security 21). USENIX Association, 2597\u20132614. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/schumilo"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464814"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598107"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00072"},{"key":"e_1_3_3_52_2","first-page":"724","volume-title":"Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE)","author":"Wang Junjie","year":"2019","unstructured":"Junjie Wang, Bihuan Chen, Lei Wei, and Yang Liu. 2019. Superion: Grammar-aware greybox fuzzing. In Proceedings of the 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 724\u2013735."},{"key":"e_1_3_3_53_2","unstructured":"Dylan Wolff Marcel B\u00f6hme and Abhik Roychoudhury. 2022. Explainable fuzzer evaluation. arXiv:2212.09519. Retrieved from https:\/\/arxiv.org\/abs\/2212.09519"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510174"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/1375581.1375611"},{"key":"e_1_3_3_56_2","first-page":"745","volume-title":"Proceedings of the 27th USENIX Security Symposium (USENIX Security 18)","author":"Yun Insu","year":"2018","unstructured":"Insu Yun, Sangho Lee, Meng Xu, Yeongjin Jang, and Taesoo Kim. 2018. QSYM: A practical concolic execution engine tailored for hybrid fuzzing. In Proceedings of the 27th USENIX Security Symposium (USENIX Security 18), 745\u2013761."},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3417260"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3767167"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3742894","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T15:46:07Z","timestamp":1766159167000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3742894"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,19]]},"references-count":57,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,31]]}},"alternative-id":["10.1145\/3742894"],"URL":"https:\/\/doi.org\/10.1145\/3742894","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"type":"print","value":"1049-331X"},{"type":"electronic","value":"1557-7392"}],"subject":[],"published":{"date-parts":[[2025,12,19]]},"assertion":[{"value":"2024-10-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-12-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}