{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:38:12Z","timestamp":1784392692529,"version":"3.55.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T00:00:00Z","timestamp":1737331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"JSPS for the KAKENHI","award":["JP21H04877, JP22K18630"],"award-info":[{"award-number":["JP21H04877, JP22K18630"]}]},{"name":"JSPS for the KAKENHI","award":["JPJSBP12023992"],"award-info":[{"award-number":["JPJSBP12023992"]}]},{"name":"Bilateral Program","award":["JPJSBP12023992"],"award-info":[{"award-number":["JPJSBP12023992"]}]},{"name":"JST for the PRESTO","award":["JPMJPR22P3"],"award-info":[{"award-number":["JPMJPR22P3"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>Fuzzing is an automated software testing technique used to find software vulnerabilities that works by sending large amounts of inputs to a software system to trigger bad behaviors. In recent years, the open source software ecosystem has seen a significant increase in the adoption of fuzzing to avoid spreading vulnerabilities throughout the ecosystem. While fuzzing can uncover vulnerabilities, there is currently a lack of knowledge regarding the challenges of conducting fuzzing activities over time. Specifically, fuzzers are very complex tools to set up and build before they can be used.<\/jats:p>\n          <jats:p>We set out to empirically find out how challenging is build maintenance in the context of fuzzing. We mine over 1.2 million build logs from Google\u2019s OSS-Fuzz service to investigate fuzzing build failures. We first conduct a quantitative analysis to quantify the prevalence of fuzzing build failures. We then manually investigate 677 failing fuzzing builds logs and establish a taxonomy of 25 root causes of build failures. We finally train a machine learning model to recognize common failure patterns in failing build logs. Our taxonomy can serve as a reference for practitioners conducting fuzzing build maintenance. Our modeling experiment shows the potential of using automation to simplify the process of fuzzing.<\/jats:p>","DOI":"10.1145\/3688842","type":"journal-article","created":{"date-parts":[[2024,8,21]],"date-time":"2024-08-21T22:24:52Z","timestamp":1724279092000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["My Fuzzers Won\u2019t Build: An Empirical Study of Fuzzing Build Failures"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7100-9227","authenticated-orcid":false,"given":"Olivier","family":"Nourry","sequence":"first","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9633-7577","authenticated-orcid":false,"given":"Yutaro","family":"Kashiwa","sequence":"additional","affiliation":[{"name":"Nara Institute of Science and Technology, Ikoma, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6222-7444","authenticated-orcid":false,"given":"Weiyi","family":"Shang","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7311-7060","authenticated-orcid":false,"given":"Honglin","family":"Shu","sequence":"additional","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7058-1045","authenticated-orcid":false,"given":"Yasutaka","family":"Kamei","sequence":"additional","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,1,20]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106267"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00083"},{"key":"e_1_3_2_4_2","article-title":"Fuzzing Vulnerability Discovery Techniques: Survey, Challenges and Future Directions","author":"Beaman Craig","year":"2022","unstructured":"Craig Beaman, Michael Redbourne, J. Darren Mummery, and Saqib Hakak. 2022. Fuzzing Vulnerability Discovery Techniques: Survey, Challenges and Future Directions. Computers and Security (2022).","journal-title":"Computers and Security"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/MS.2020.3016773","article-title":"Fuzzing: Challenges and Reflections","author":"Boehme Marcel","year":"2021","unstructured":"Marcel Boehme, Cristian Cadar, and Abhik Roychoudhury. 2021. Fuzzing: Challenges and Reflections. IEEE Software (2021), 79\u201386.","journal-title":"IEEE Software"},{"key":"e_1_3_2_6_2","first-page":"713","volume-title":"Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE\u2019 20)","author":"B\u00f6hme Marcel","year":"2020","unstructured":"Marcel B\u00f6hme and Brandon Falk. 2020. Fuzzing: On the Exponential Cost of Vulnerability Discovery. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE\u2019 20). ACM, 713\u2013724."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_2_8_2","unstructured":"Circle CI. 2011. Circle CI. Retrieved from https:\/\/circleci.com\/"},{"key":"e_1_3_2_9_2","unstructured":"Travis CI. 2011. Travis CI. Retrieved from https:\/\/www.travis-ci.com\/"},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Jacob Cohen. 1960. A Coefficient of Agreement for Nominal Scales. Educational and Psychological Measurement (1960) 37\u201346. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:15926286","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_11_2","unstructured":"National Vulnerability Database. [n.d.]. Log4Shell Vulnerability. Retrieved from https:\/\/nvd.nist.gov\/vuln\/detail\/ CVE-2021-44228"},{"key":"e_1_3_2_12_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171\u20134186."},{"key":"e_1_3_2_13_2","first-page":"131","volume-title":"Proceedings of the 18th International Conference on Mining Software Repositories","author":"Ding Zhen Yu","year":"2021","unstructured":"Zhen Yu Ding and Claire Le Goues. 2021. An Empirical Study of OSS-Fuzz Bugs. In Proceedings of the 18th International Conference on Mining Software Repositories. ACM, 131\u2013142."},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387460"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397388"},{"key":"e_1_3_2_16_2","volume-title":"Proceedings of the 14th USENIX Conference on Offensive Technologies","author":"Fioraldi Andrea","year":"2020","unstructured":"Andrea Fioraldi, Dominik Maier, Heiko Ei\u00dffeldt, and Marc Heuse. 2020. AFL++: Combining Incremental Steps of Fuzzing Research. In Proceedings of the 14th USENIX Conference on Offensive Technologies. USENIX Association."},{"key":"e_1_3_2_17_2","unstructured":"Go Fuzzing. 2022. Go Fuzzing Documentation. Retrieved from https:\/\/go.dev\/doc\/security\/fuzz\/"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510211"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2015.91"},{"key":"e_1_3_2_20_2","unstructured":"GitLab. 2019. GitLab Acquisition of Fuzzit. Retrieved from https:\/\/about.gitlab.com\/blog\/2020\/10\/22\/fuzzit-acquisition-journey\/"},{"key":"e_1_3_2_21_2","unstructured":"Oss-Fuzz Google. 2023. Chrome Monorail Bug Tracker. Retrieved from https:\/\/bugs.chromium.org\/p\/oss-fuzz\/issues\/list"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"1078","DOI":"10.1145\/3180155.3180181","volume-title":"Proceedings of the 40th International Conference on Software Engineering (ICSE\u201918)","author":"Hassan Foyzul","year":"2018","unstructured":"Foyzul Hassan and Xiaoyin Wang. 2018. HireBuild: An Automatic Approach to History-Driven Repair of Build Scripts. In Proceedings of the 40th International Conference on Software Engineering (ICSE\u201918). IEEE\/ACM, 1078\u20131089."},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","first-page":"3248","DOI":"10.1007\/s10664-018-9603-z","article-title":"Studying and Detecting Log-Related Issues","author":"Hassani Mehran","year":"2018","unstructured":"Mehran Hassani, Weiyi Shang, Emad Shihab, and Nikolaos Tsantalis. 2018. Studying and Detecting Log-Related Issues. Empirical Software Engineering (2018), 3248\u20133280.","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1031"},{"key":"e_1_3_2_25_2","first-page":"1592","volume-title":"Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923)","author":"Humayun Ahmad","year":"2023","unstructured":"Ahmad Humayun, Yaoxuan Wu, Miryung Kim, and Muhammad Ali Gulzar. 2023. NaturalFuzz: Natural Input Generation for Big Data Analytics. In Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923). ACM, 1592\u20131603."},{"key":"e_1_3_2_26_2","unstructured":"Google Inc. [n.d.]. OSS-Fuzz Architecture. Retrieved from https:\/\/google.github.io\/oss-fuzz\/architecture\/"},{"key":"e_1_3_2_27_2","unstructured":"Synopsys Inc. 2014. Heartbleed Vulnerability. Retrieved from https:\/\/heartbleed.com\/"},{"key":"e_1_3_2_28_2","unstructured":"Jenkins. 2011. Jenkins.io. Retrieved from https:\/\/www.jenkins.io\/"},{"key":"e_1_3_2_29_2","first-page":"207","volume-title":"Proceedings of the 20th International Conference on Mining Software Repositories","author":"Keller Brandon","year":"2023","unstructured":"Brandon Keller, Andrew Meneely, and Benjamin Meyers. 2023. What Happens When We Fuzz? Investigating OSS-Fuzz Bug History. In Proceedings of the 20th International Conference on Mining Software Repositories. ACM, 207\u2013217."},{"key":"e_1_3_2_30_2","first-page":"41","volume-title":"Proceedings of the International Conference on Software Maintenance and Evolution","author":"Kerzazi Noureddine","year":"2014","unstructured":"Noureddine Kerzazi, Foutse Khomh, and Bram Adams. 2014. Why Do Automated Builds Break? An Empirical Study. In Proceedings of the International Conference on Software Maintenance and Evolution. IEEE, 41\u201350."},{"key":"e_1_3_2_31_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238176"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.2970422"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9467-z"},{"key":"e_1_3_2_35_2","first-page":"1","article-title":"A Survey on Text Classification: From Traditional to Deep Learning","author":"Li Qian","year":"2022","unstructured":"Qian Li, Hao Peng, Jianxin Li, Congying Xia, Renyu Yang, Lichao Sun, Philip S. Yu, and Lifang He. 2022. A Survey on Text Classification: From Traditional to Deep Learning. ACM Transactions on Intelligent Systems and Technology (2022), 1\u201341.","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1109\/TR.2018.2834476","article-title":"Fuzzing: State of the Art","author":"Liang Hongliang","year":"2018","unstructured":"Hongliang Liang, Xiaoxiao Pei, Xiaodong Jia, Wuwei Shen, and Jian Zhang. 2018. Fuzzing: State of the Art. IEEE Transactions on Reliability (2018), 1199\u20131218.","journal-title":"IEEE Transactions on Reliability"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330578"},{"key":"e_1_3_2_38_2","first-page":"617","volume-title":"Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE \u201920)","author":"Lou Yiling","year":"2020","unstructured":"Yiling Lou, Zhenpeng Chen, Yanbin Cao, Dan Hao, and Lu Zhang. 2020. Understanding Build Issue Resolution in Practice: Symptoms and Fix Patterns. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE \u201920). ACM, 617\u2013628."},{"key":"e_1_3_2_39_2","first-page":"106","volume-title":"Proceedings of the 25th International Conference on Software Analysis, Evolution and Reengineering (SANER \u201918)","author":"Macho Christian","year":"2018","unstructured":"Christian Macho, Shane McIntosh, and Martin Pinzger. 2018. Automatically Repairing Dependency-Related Build Breakage. In Proceedings of the 25th International Conference on Software Analysis, Evolution and Reengineering (SANER \u201918). ACM, 106\u2013117."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2946563"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359174"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985813"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623097"},{"key":"e_1_3_2_44_2","article-title":"Evaluating Seed Selection for Fuzzing JavaScript Engines","author":"Ming Wen","year":"2023","unstructured":"Wen Ming, Wang Yongcong, Xia Yifan, and Jin Hai. 2023. Evaluating Seed Selection for Fuzzing JavaScript Engines. Empirical Software Engineering (2023).","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464797"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3611668","article-title":"The Human Side of Fuzzing: Challenges Faced by Developers During Fuzzing Activities","volume":"33","author":"Nourry Olivier","year":"2023","unstructured":"Olivier Nourry, Yutaro Kashiwa, Bin Lin, Gabriele Bavota, Michele Lanza, and Yasutaka Kamei. 2023. The Human Side of Fuzzing: Challenges Faced by Developers During Fuzzing Activities. ACM Transactions on Software Engineering and Methodology 33, (2023), 1\u201326.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Olivier Nourry Yutaro Kashiwa Weiyi Shang and Yasutaka Kamei. 2024. Replication Package for the Paper: My Fuzzers Won\u2019t Build: An Empirical Study of Fuzzing Build Failures. Retrieved from https:\/\/github.com\/posl\/FuzzingBuildLogs-ReplicationPackage","DOI":"10.1145\/3688842"},{"key":"e_1_3_2_48_2","unstructured":"OpenAI. 2022. GPT text-embedding-ada-002. Retrieved from https:\/\/openai.com\/blog\/new-and-improved-embedding-model"},{"key":"e_1_3_2_49_2","unstructured":"Google OSS-Fuzz. 2016. OSS-Fuzz. Retrieved from https:\/\/google.github.io\/oss-fuzz\/"},{"key":"e_1_3_2_50_2","unstructured":"Google OSS-Fuzz. 2023. Oss-Fuzz build logs storage api. Retrieved from https:\/\/oss-fuzz-build-logs.storage.googleapis.com\/index.html"},{"key":"e_1_3_2_51_2","first-page":"1","article-title":"NSFuzz: Towards Efficient and State-Aware Network Service Fuzzing","author":"Qin Shisong","year":"2023","unstructured":"Shisong Qin, Fan Hu, Zheyu Ma, Bodong Zhao, Tingting Yin, and Chao Zhang. 2023. NSFuzz: Towards Efficient and State-Aware Network Service Fuzzing. ACM Transactions on Software Engineering and Methodology (2023), 1\u201326.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_52_2","article-title":"Evaluating the Impact of Falsely Detected Performance Bug-Inducing Changes in JIT Models","author":"Quach Sophia","year":"2021","unstructured":"Sophia Quach, Maxime Lamothe, Bram Adams, Yasutaka Kamei, and Weiyi Shang. 2021. Evaluating the Impact of Falsely Detected Performance Bug-Inducing Changes in JIT Models. Empirical Software Engineering (2021).","journal-title":"Empirical Software Engineering"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.54"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568255"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340544"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","first-page":"259","DOI":"10.4236\/jbise.2016.95019","article-title":"Detection of Ventricular Fibrillation Using Random Forest Classifier","author":"Verma Anurag","year":"2016","unstructured":"Anurag Verma and Xiaodai Dong. 2016. Detection of Ventricular Fibrillation Using Random Forest Classifier. Journal of Biomedical Science and Engineering (2016), 259\u2013268.","journal-title":"Journal of Biomedical Science and Engineering"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616264"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616294"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510174"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387483"},{"key":"e_1_3_2_61_2","first-page":"1","volume-title":"Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering","author":"Zhang Chen","year":"2023","unstructured":"Chen Zhang, Bihuan Chen, Junhao Hu, Xin Peng, and Wenyun Zhao. 2023. BuildSonic: Detecting and Repairing Performance-Related Configuration Smells for Continuous Integration Builds. In Proceedings of the 37th IEEE\/ACM International Conference on Automated Software Engineering. ACM, 1\u201313."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549107"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3512345"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3688842","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3688842","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:10Z","timestamp":1750291450000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3688842"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,20]]},"references-count":62,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3688842"],"URL":"https:\/\/doi.org\/10.1145\/3688842","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,20]]},"assertion":[{"value":"2024-02-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-28","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}