{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T11:03:37Z","timestamp":1790679817254,"version":"4.1.0"},"reference-count":41,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R011605\/1 and EP\/R006865\/1"],"award-info":[{"award-number":["EP\/R011605\/1 and EP\/R006865\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2019,10,10]]},"abstract":"<jats:p>Despite much recent interest in randomised testing (fuzzing) of compilers, the practical impact of fuzzer-found compiler bugs on real-world applications has barely been assessed. We present the first quantitative and qualitative study of the tangible impact of miscompilation bugs in a mature compiler. We follow a rigorous methodology where the bug impact over the compiled application is evaluated based on (1) whether the bug appears to trigger during compilation; (2) the extent to which generated assembly code changes syntactically due to triggering of the bug; and (3) whether such changes cause regression test suite failures, or whether we can manually find application inputs that trigger execution divergence due to such changes. The study is conducted with respect to the compilation of more than 10 million lines of C\/C++ code from 309 Debian packages, using 12% of the historical and now fixed miscompilation bugs found by four state-of-the-art fuzzers in the Clang\/LLVM compiler, as well as 18 bugs found by human users compiling real code or as a by-product of formal verification efforts. The results show that almost half of the fuzzer-found bugs propagate to the generated binaries for at least one package, in which case only a very small part of the binary is typically affected, yet causing two failures when running the test suites of all the impacted packages. User-reported and formal verification bugs do not exhibit a higher impact, with a lower rate of triggered bugs and one test failure. The manual analysis of a selection of the syntactic changes caused by some of our bugs (fuzzer-found and non fuzzer-found) in package assembly code, shows that either these changes have no semantic impact or that they would require very specific runtime circumstances to trigger execution divergence.<\/jats:p>","DOI":"10.1145\/3360581","type":"journal-article","created":{"date-parts":[[2019,10,11]],"date-time":"2019-10-11T14:53:33Z","timestamp":1570805613000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":43,"title":["Compiler fuzzing: how much does it matter?"],"prefix":"10.1145","volume":"3","author":[{"given":"Micha\u00ebl","family":"Marcozzi","sequence":"first","affiliation":[{"name":"Imperial College London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiyi","family":"Tang","sequence":"additional","affiliation":[{"name":"Imperial College London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alastair F.","family":"Donaldson","sequence":"additional","affiliation":[{"name":"Imperial College London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cristian","family":"Cadar","sequence":"additional","affiliation":[{"name":"Imperial College London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Deniable Backdoors using Compiler Bugs. PoC GTFO","author":"Bauer Scott","year":"2015","unstructured":"Scott Bauer , Pascal Cuoq , and John Regehr . 2015. Deniable Backdoors using Compiler Bugs. PoC GTFO ( 2015 ), 7\u20139. Scott Bauer, Pascal Cuoq, and John Regehr. 2015. Deniable Backdoors using Compiler Bugs. PoC GTFO (2015), 7\u20139."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0950-5849(97)00017-7"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/0950-5849(95)01055-6"},{"key":"e_1_2_2_4_1","unstructured":"Cristian Cadar Lu\u00eds Pina and John Regehr. 2015. Multi-Version Execution Defeats a Compiler-Bug-Based Backdoor. http:\/\/ccadar.blogspot.co.uk\/2015\/11\/multi-version-execution-defeats.html .  Cristian Cadar Lu\u00eds Pina and John Regehr. 2015. Multi-Version Execution Defeats a Compiler-Bug-Based Backdoor. http:\/\/ccadar.blogspot.co.uk\/2015\/11\/multi-version-execution-defeats.html ."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2408776.2408795"},{"key":"e_1_2_2_6_1","volume-title":"Technical Report HKUST-CS98-01. Hong Kong University of Science and Technology.","author":"Chen T.Y.","year":"1998","unstructured":"T.Y. Chen , S.C. Cheung , and S.M. Yiu . 1998 . Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01. Hong Kong University of Science and Technology. T.Y. Chen, S.C. Cheung, and S.M. Yiu. 1998. Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01. Hong Kong University of Science and Technology."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2462173"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908080.2908095"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28891-3_12"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1287624.1287651"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3133917"},{"key":"e_1_2_2_12_1","volume-title":"Proc. of the International Workshop on Metamorphic Testing (MET\u201916)","author":"Alastair","unstructured":"Alastair F. Donaldson and Andrei Lascu. 2016. Metamorphic testing for (graphics) compilers . In Proc. of the International Workshop on Metamorphic Testing (MET\u201916) . Alastair F. Donaldson and Andrei Lascu. 2016. Metamorphic testing for (graphics) compilers. In Proc. of the International Workshop on Metamorphic Testing (MET\u201916)."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1147\/sj.94.0242"},{"key":"e_1_2_2_14_1","volume-title":"Proc. of the 21st USENIX Security Symposium (USENIX Security\u201912)","author":"Holler Christian","year":"2012","unstructured":"Christian Holler , Kim Herzig , and Andreas Zeller . 2012 . Fuzzing with Code Fragments . In Proc. of the 21st USENIX Security Symposium (USENIX Security\u201912) . Christian Holler, Kim Herzig, and Andreas Zeller. 2012. Fuzzing with Code Fragments. In Proc. of the 21st USENIX Security Symposium (USENIX Security\u201912)."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2694344.2694390"},{"key":"e_1_2_2_16_1","unstructured":"Derek Jones. 2015. So you found a bug in my compiler: Whoopee do. http:\/\/shape-of-code.coding-guidelines.com\/2015\/12\/ 07\/so-you-found-a-bug-in-my-compiler-whoopee-do\/ .  Derek Jones. 2015. So you found a bug in my compiler: Whoopee do. http:\/\/shape-of-code.coding-guidelines.com\/2015\/12\/ 07\/so-you-found-a-bug-in-my-compiler-whoopee-do\/ ."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155636"},{"key":"e_1_2_2_18_1","volume-title":"Survey of Compiler Testing Methods. Programming and Computing Software 31 (Jan","author":"Kossatchev Alexander","year":"2005","unstructured":"Alexander Kossatchev and Mikhail Posypkin . 2005. Survey of Compiler Testing Methods. Programming and Computing Software 31 (Jan . 2005 ), 10\u201319. Issue 1. Alexander Kossatchev and Mikhail Posypkin. 2005. Survey of Compiler Testing Methods. Programming and Computing Software 31 (Jan. 2005), 10\u201319. Issue 1."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.5555\/977395.977673"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2666356.2594334"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2814270.2814319"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2771783.2771785"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1538788.1538814"},{"key":"e_1_2_2_24_1","volume-title":"Proc. of the Conference on Programing Language Design and Implementation (PLDI\u201915)","author":"Lidbury Christopher","unstructured":"Christopher Lidbury , Andrei Lascu , Nathan Chong , and Alastair F. Donaldson . 2015. Many-core compiler fuzzing . In Proc. of the Conference on Programing Language Design and Implementation (PLDI\u201915) . Christopher Lidbury, Andrei Lascu, Nathan Chong, and Alastair F. Donaldson. 2015. Many-core compiler fuzzing. In Proc. of the Conference on Programing Language Design and Implementation (PLDI\u201915)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2737965"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635920"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2610419"},{"key":"e_1_2_2_28_1","first-page":"100","article-title":"Differential testing for software","volume":"10","author":"McKeeman W. M.","year":"1998","unstructured":"W. M. McKeeman . 1998 . Differential testing for software . Digital Technical Journal 10 (1998), 100 \u2013 107 . Issue 1. W. M. McKeeman. 1998. Differential testing for software. Digital Technical Journal 10 (1998), 100\u2013107. Issue 1.","journal-title":"Digital Technical Journal"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.2197\/ipsjtsldm.7.91"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/APCCAS.2016.7804063"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01932308"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254104"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460833.1460869"},{"key":"e_1_2_2_34_1","unstructured":"Sergio Segura Gordon Fraser Ana Sanchez and Antonio Ruiz-Cort\u00e9s. 2016. A Survey on Metamorphic Testing. (2016).  Sergio Segura Gordon Fraser Ana Sanchez and Antonio Ruiz-Cort\u00e9s. 2016. A Survey on Metamorphic Testing. (2016)."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983990.2984038"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931074"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2010.39"},{"key":"e_1_2_2_38_1","unstructured":"B.A. Wichmann. 1998. Some Remarks about Random Testing. http:\/\/www.npl.co.uk\/upload\/pdf\/random_testing.pdf .  B.A. Wichmann. 1998. Some Remarks about Random Testing. http:\/\/www.npl.co.uk\/upload\/pdf\/random_testing.pdf ."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1993498.1993532"},{"key":"e_1_2_2_40_1","unstructured":"Yarpgen. 2018. https:\/\/github.com\/intel\/yarpgen .  Yarpgen. 2018. https:\/\/github.com\/intel\/yarpgen ."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062379"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360581","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3360581","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:22:59Z","timestamp":1750202579000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360581"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":41,"journal-issue":{"issue":"OOPSLA","published-print":{"date-parts":[[2019,10,10]]}},"alternative-id":["10.1145\/3360581"],"URL":"https:\/\/doi.org\/10.1145\/3360581","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}