{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:38:25Z","timestamp":1785537505907,"version":"3.56.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"PLDI","license":[{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2024,6,20]]},"abstract":"<jats:p>We introduce a novel approach for testing optimizing compilers with code from real-world applications The main idea is to construct well-formed programs by fusing multiple code snippets from various realworld projects. The key insight is backed by the fact that the large volume of real-world code exercises rich syntactical and semantic language features, which current engineering-intensive approaches like random program generators are hard to fully support. To construct well-formed programs from real-world code our approach works by (1) extracting real-world code at the granularity of function, (2) injecting function calls into seed programs, and (3) leveraging dynamic execution information to maintain the semantics and build complex data dependencies between injected functions and the seed program. With this idea, our approach complements the existing generators by boosting their expressiveness via fusing real-world code in a semantics-preserving way.<\/jats:p>\n          <jats:p>We implement our idea in a tool, Creal, to test C compilers. In a nine-month testing period, we have reported 132 bugs to GCC and LLVM, two of the most popular and well-tested C compilers. At the time of writing, 121 of them have been confirmed as unknown bugs, and 101 of them have been fixed. Most of these bugs were miscompilations, and many were recognized as long-latent and critical. Our evaluation results evidently demonstrate the significant advantage of using real-world code to stress-test compilers. We believe this idea will benefit the general compiler testing direction and will be directly applicable to other compilers.<\/jats:p>","DOI":"10.1145\/3656386","type":"journal-article","created":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T16:27:20Z","timestamp":1718900840000},"page":"223-245","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["Boosting Compiler Testing by Injecting Real-World Code"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7556-3615","authenticated-orcid":false,"given":"Shaohua","family":"Li","sequence":"first","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0896-9997","authenticated-orcid":false,"given":"Theodoros","family":"Theodoridis","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2970-1391","authenticated-orcid":false,"given":"Zhendong","family":"Su","sequence":"additional","affiliation":[{"name":"ETH Zurich, Zurich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,20]]},"reference":[{"key":"e_1_3_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/MobileSoft.2015.11"},{"key":"e_1_3_1_3_1","doi-asserted-by":"publisher","unstructured":"Junjie Chen Jibesh Patra Michael Pradel Yingfei Xiong Hongyu Zhang Dan Hao and Lu Zhang. 2020. A Survey of Compiler Testing. Comput. Surveys 53 1 Article 4 (2020) 36 pages. https:\/\/doi.org\/10.1145\/3363562 10.1145\/3363562","DOI":"10.1145\/3363562"},{"key":"e_1_3_1_4_1","unstructured":"Clang. 2023. AddressSanitizer. https:\/\/clang.llvm.org\/docs\/AddressSanitizer.html."},{"key":"e_1_3_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213848"},{"key":"e_1_3_1_6_1","doi-asserted-by":"publisher","unstructured":"Anderson Faustino da Silva Bruno Conde Kind Jos\u00e9 Wesley de Souza Magalh\u00e3es Jer\u00f4nimo Nunes Rocha Breno Campos Ferreira Guimar\u00e3es and Fernando Magno Quint\u00e3o Pereira. 2021. AnghaBench: A Suite with One Million Compilable C Benchmarks for Code-Size Reduction. In Proceedings of the 2021 IEEE\/ACM International Symposium on Code Generation and Optimization (CGO \u201921). 378-390. https:\/\/doi.org\/10.1109\/CGO51591.2021.9370322 10.1109\/CGO51591.2021.9370322","DOI":"10.1109\/CGO51591.2021.9370322"},{"key":"e_1_3_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3418933"},{"key":"e_1_3_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598130"},{"key":"e_1_3_1_9_1","unstructured":"Project QMK. Firmware. 2023. Quantum Mechanical Keyboard Firmware. https:\/\/github.com\/qmk\/qmk_firmware\/blob\/gb_port\/keyboards\/bfake\/matrix.c."},{"key":"e_1_3_1_10_1","unstructured":"Project FreeBSD. 2023. FreeBSD. https:\/\/github.com\/freebsd\/freebsd-src\/blob\/stable\/10\/contrib\/binutils\/libiberty\/memset.c."},{"key":"e_1_3_1_11_1","unstructured":"Project FreeType. 2023. FreeType. https:\/\/github.com\/freetype\/freetype\/blob\/master\/src\/tools\/ftrandom\/ftrandom.c."},{"key":"e_1_3_1_12_1","unstructured":"GNU GCC. 2023. Passes and Files of the Compiler. https:\/\/gcc.gnu.org\/onlinedocs\/gccint\/Passes.html."},{"key":"e_1_3_1_13_1","unstructured":"Zhendong Su\u2019s Group. 2023. EMI Compiler Testing. https:\/\/people.inf.ethz.ch\/suz\/emi\/index.html."},{"key":"e_1_3_1_14_1","doi-asserted-by":"crossref","unstructured":"Sumit Gulwani Oleksandr Polozov Rishabh Singh et al. 2017. Program Synthesis. Foundations and Trends in Programming Languages 4 1-2 (2017) 1-119.","DOI":"10.1561\/2500000010"},{"key":"e_1_3_1_15_1","unstructured":"Christian Holler Kim Herzig and Andreas Zeller. 2012. Fuzzing with Code Fragments. In Proceedings of the 21st USENIX Conference on Security Symposium (USENIX Security \u201912). USENIX Association USA 445-458."},{"key":"e_1_3_1_16_1","doi-asserted-by":"publisher","unstructured":"Raphael Isemann Cristiano Giuffrida Herbert Bos Erik van der Kouwe and Klaus von Gleissenthall. 2023. Don\u2019t Look UB: Exposing Sanitizer-Eliding Compiler Optimizations. Proc. ACM Program. Lang. 7 PLDI Article 143 (2023) 21 pages. https:\/\/doi.org\/10.1145\/3591257 10.1145\/3591257","DOI":"10.1145\/3591257"},{"key":"e_1_3_1_17_1","doi-asserted-by":"publisher","unstructured":"Vu Le Mehrdad Afshari and Zhendong Su. 2014. Compiler Validation via Equivalence modulo Inputs. In Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI \u201914). 216-226. https:\/\/doi.org\/10.1145\/2594291.2594334 10.1145\/2594291.2594334","DOI":"10.1145\/2594291.2594334"},{"key":"e_1_3_1_18_1","doi-asserted-by":"publisher","unstructured":"Vu Le Chengnian Sun and Zhendong Su. 2015. Finding Deep Compiler Bugs via Guided Stochastic Program Mutation. In Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming Systems Languages and Applications (OOPSLA \u201915). 386-399. https:\/\/doi.org\/10.1145\/2814270.2814319 10.1145\/2814270.2814319","DOI":"10.1145\/2814270.2814319"},{"key":"e_1_3_1_19_1","doi-asserted-by":"publisher","unstructured":"Xavier Leroy. 2009. Formal Verification of a Realistic Compiler. Commun. ACM 52 7 (2009) 107-115. https:\/\/doi.org\/10.1145\/1538788.1538814 10.1145\/1538788.1538814","DOI":"10.1145\/1538788.1538814"},{"key":"e_1_3_1_20_1","unstructured":"Xavier Leroy. 2023. CompCert. https:\/\/compcert.org\/index.html."},{"key":"e_1_3_1_21_1","unstructured":"Shaohua Li and Zhendong Su. 2024. UBFuzz: Finding Bugs in Sanitizer Implementations.. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '24)."},{"key":"e_1_3_1_22_1","unstructured":"Shaohua Li Theodoros Theodoridis and Zhendong Su. 2024. Artifact for Creal - Boosting Compiler Testing by Injecting real-world Code. https:\/\/zenodo.org\/doi\/10.5281\/zenodo.10802521."},{"key":"e_1_3_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2813885.2737986"},{"key":"e_1_3_1_24_1","doi-asserted-by":"publisher","unstructured":"Vsevolod Livinskii Dmitry Babokin and John Regehr. 2020. Random Testing for C and C++ Compilers with YARPGen. Proceedings of the ACM Programming Languages 4 OOPSLA Article 196 (2020). https:\/\/doi.org\/10.1145\/3428264 10.1145\/3428264","DOI":"10.1145\/3428264"},{"key":"e_1_3_1_25_1","doi-asserted-by":"publisher","unstructured":"Vsevolod Livinskii Dmitry Babokin and John Regehr. 2023. Fuzzing Loop Optimizations in Compilers for C++ and DataParallel Languages. Proceedings of the ACM Programming Languages 7 PLDI Article 181 (2023). https:\/\/doi.org\/10.1145\/3591295 10.1145\/3591295","DOI":"10.1145\/3591295"},{"key":"e_1_3_1_26_1","unstructured":"LLVM. 2023a. LibTooling. https:\/\/clang.llvm.org\/docs\/LibTooling.html."},{"key":"e_1_3_1_27_1","unstructured":"LLVM. 2023b. LLVM\u2019s Analysis and Transform Passes. https:\/\/llvm.org\/docs\/Passes.html."},{"key":"e_1_3_1_28_1","doi-asserted-by":"publisher","unstructured":"Zohar Manna and Richard Waldinger. 1980. A Deductive Approach to Program Synthesis. ACM Trans. Program. Lang. Syst 2 1 (1980) 90-121. https:\/\/doi.org\/10.1145\/357084.357090 10.1145\/357084.357090","DOI":"10.1145\/357084.357090"},{"key":"e_1_3_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360581"},{"key":"e_1_3_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2737924.2738007"},{"key":"e_1_3_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254104"},{"key":"e_1_3_1_32_1","unstructured":"Project GB Studio. 2023. GBStudio. https:\/\/github.com\/chrismaltby\/gb-studio\/blob\/develop\/buildTools\/linux-x64\/mod2gbt\/mod2gbt.c."},{"key":"e_1_3_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983990.2984038"},{"key":"e_1_3_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522728"},{"key":"e_1_3_1_35_1","doi-asserted-by":"publisher","unstructured":"Anjiang Wei Yinlin Deng Chenyuan Yang and Lingming Zhang. 2022. Free Lunch for Testing: Fuzzing Deep-Learning Libraries from Open Source. In Proceedings of the 44th International Conference on Software Engineering (ICSE \u201922). 995-1007. https:\/\/doi.org\/10.1145\/3510003.3510041 10.1145\/3510003.3510041","DOI":"10.1145\/3510003.3510041"},{"key":"e_1_3_1_36_1","first-page":"1","article-title":"Universal fuzzing via large Language models","author":"Xia Chunqiu Steven","year":"2024","unstructured":"Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. 2024. Universal fuzzing via large Language models. In Proceedings of the 46th International Conference on Software Engineering (ICSE \u201924). 1-13.","journal-title":"In Proceedings of the 46th International Conference on Software Engineering (ICSE \u201924)"},{"key":"e_1_3_1_37_1","doi-asserted-by":"publisher","unstructured":"Xuejun Yang Yang Chen Eric Eide and John Regehr. 2011. Finding and Understanding Bugs in C Compilers. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI \u201911). 283-294. https:\/\/doi.org\/10.1145\/1993498.1993532 10.1145\/1993498.1993532","DOI":"10.1145\/1993498.1993532"},{"key":"e_1_3_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110884"},{"key":"e_1_3_1_39_1","doi-asserted-by":"publisher","unstructured":"Xiaogang Zhu Sheng Wen Seyit Camtepe and Yang Xiang. 2022. Fuzzing: A Survey for Roadmap. Comput. Surveys 54 11s Article 230 (2022) 36 pages. https:\/\/doi.org\/10.1145\/3512345 10.1145\/3512345","DOI":"10.1145\/3512345"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656386","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3656386","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,4]],"date-time":"2025-07-04T20:44:11Z","timestamp":1751661851000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656386"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,20]]},"references-count":38,"journal-issue":{"issue":"PLDI","published-print":{"date-parts":[[2024,6,20]]}},"alternative-id":["10.1145\/3656386"],"URL":"https:\/\/doi.org\/10.1145\/3656386","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,20]]},"assertion":[{"value":"2024-06-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}