{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T04:32:33Z","timestamp":1768451553400,"version":"3.49.0"},"reference-count":56,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T00:00:00Z","timestamp":1657238400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T00:00:00Z","timestamp":1657238400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R011605\/1"],"award-info":[{"award-number":["EP\/R011605\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R006865\/1"],"award-info":[{"award-number":["EP\/R006865\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/R011605\/1"],"award-info":[{"award-number":["EP\/R011605\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Compiler fuzzing techniques require a means of generating programs that are free from undefined behaviour (UB) to reliably reveal miscompilation bugs. Existing program generators such as <jats:sc>Csmith<\/jats:sc> achieve UB-freedom by heavily restricting the form of generated programs. The idiomatic nature of the resulting programs risks limiting the test coverage they can offer, and thus the compiler bugs they can discover. We investigate the idea of adapting existing fuzzers to be less restrictive concerning UB, in the practical setting of C compiler testing via a new tool, <jats:sc>CsmithEdge<\/jats:sc>, which extends <jats:sc>Csmith<\/jats:sc>. <jats:sc>CsmithEdge<\/jats:sc> probabilistically weakens the constraints used to enforce UB-freedom, thus generated programs are no longer guaranteed to be UB-free. It then employs several off-the-shelf UB detection tools and a novel dynamic analysis to (a) detect cases where the generated program exhibits UB and (b) determine where <jats:sc>Csmith<\/jats:sc> has been too conservative in its use of <jats:italic>safe math<\/jats:italic> wrappers that guarantee UB-freedom for arithmetic operations, removing the use of redundant ones. The resulting UB-free programs can be used to test for miscompilation bugs via differential testing. The non-UB-free programs can still be used to check that the compiler under test does not crash or hang. Our experiments on recent versions of GCC, LLVM and the Microsoft Visual Studio Compiler show that <jats:sc>CsmithEdge<\/jats:sc> was able to discover 7 previously unknown miscompilation bugs (5 already fixed in response to our reports) that could not be found via intensive testing using <jats:sc>Csmith<\/jats:sc>, and 2 compiler-hang bugs that were fixed independently shortly before we considered reporting them.<\/jats:p>","DOI":"10.1007\/s10664-022-10146-1","type":"journal-article","created":{"date-parts":[[2022,7,8]],"date-time":"2022-07-08T08:13:19Z","timestamp":1657267999000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["CsmithEdge: more effective compiler testing by handling undefined behaviour less conservatively"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3099-1189","authenticated-orcid":false,"given":"Karine","family":"Even-Mendoza","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3599-7264","authenticated-orcid":false,"given":"Cristian","family":"Cadar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7448-7961","authenticated-orcid":false,"given":"Alastair F.","family":"Donaldson","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,7,8]]},"reference":[{"key":"10146_CR1","unstructured":"Address Sanitizer (2012) https:\/\/clang.llvm.org\/docs\/AddressSanitizer.html"},{"key":"10146_CR2","unstructured":"Babokin D (2019) Comment on running one million Yarpgen programs https:\/\/twitter.com\/DmitryBabokin\/status\/1134907976085516290 [Online; accessed 24-February-2022]"},{"key":"10146_CR3","doi-asserted-by":"crossref","unstructured":"Barr ET, Harman M, McMinn P, Shahbaz M, Yoo S (2015) The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering (TSE) 41(5)","DOI":"10.1109\/TSE.2014.2372785"},{"key":"10146_CR4","unstructured":"Bugzilla GCC (2020a) Bug 93744 https:\/\/gcc.gnu.org\/bugzilla\/show_bug.cgi?id=93744 [Online; accessed 24-February-2022]"},{"key":"10146_CR5","unstructured":"Bugzilla GCC (2020b) Bug 94809 https:\/\/gcc.gnu.org\/bugzilla\/show_bug.cgi?id=94809 [Online; accessed 24-February-2022]"},{"key":"10146_CR6","unstructured":"Bugzilla GCC (2020c) Bug 96369 https:\/\/gcc.gnu.org\/bugzilla\/show_bug.cgi?id=96369 [Online; accessed 24-February-2022]"},{"key":"10146_CR7","unstructured":"Bugzilla GCC (2020d) Bug 96549 https:\/\/gcc.gnu.org\/bugzilla\/show_bug.cgi?id=96549 [Online; accessed 24-February-2022]"},{"key":"10146_CR8","unstructured":"Bugzilla GCC (2020e) Bug 96760 https:\/\/gcc.gnu.org\/bugzilla\/show_bug.cgi?id=96760 [Online; accessed 24-February-2022]"},{"key":"10146_CR9","unstructured":"Bugzilla LLVM (2020f) Bug 47578 https:\/\/bugs.llvm.org\/show_bug.cgi?id=47578 [Online; accessed 24-February-2022]"},{"key":"10146_CR10","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1016\/0950-5849(95)01055-6","volume":"38","author":"C Burgess","year":"1996","unstructured":"Burgess C, Saidi M (1996) The automatic generation of test cases for optimizing Fortran compilers. Information and Software Technology (IST) 38:111\u2013119","journal-title":"Information and Software Technology (IST)"},{"key":"10146_CR11","doi-asserted-by":"crossref","unstructured":"Chen J, Hu W, Hao D, Xiong Y, Zhang H, Zhang L, Xie B (2016) An empirical comparison of compiler testing techniques. In: Proc. of the 38th international conference on software engineering (ICSE\u201916)","DOI":"10.1145\/2884781.2884878"},{"issue":"1","key":"10146_CR12","first-page":"4:1","volume":"53","author":"J Chen","year":"2020","unstructured":"Chen J, Patra J, Pradel M, Xiong Y, Zhang H, Hao D, Zhang L (2020) A survey of compiler testing. ACM Computing Surveys 53(1):4:1\u20134:36","journal-title":"ACM Computing Surveys"},{"key":"10146_CR13","doi-asserted-by":"crossref","unstructured":"Chen J, Wang G, Hao D, Xiong Y, Zhang H, Zhang L (2019) History-guided configuration diversification for compiler test-program generation. In: Proc. of the 34th IEEE international conference on automated software engineering (ASE\u201919), pp 305\u2013316","DOI":"10.1109\/ASE.2019.00037"},{"key":"10146_CR14","unstructured":"Chen T, Cheung S, Yiu S (1998) Metamorphic testing: A new approach for generating next test cases. Tech. Rep. HKUST-CS98-01 Hong Kong University of Science and Technology"},{"key":"10146_CR15","unstructured":"Csmith Homepage (2021) https:\/\/srg.doc.ic.ac.uk\/projects\/CsmithEdge\/ [Online; accessed 24-February-2022]"},{"key":"10146_CR16","doi-asserted-by":"crossref","unstructured":"Cuoq P, Kirchner F, Kosmatov N, Prevosto V, Signoles J, Yakobowski B (2012) Frama-C: A software analysis perspective. In: Proc. of the 10th international conference on software engineering and formal methods (SEFM\u201912)","DOI":"10.1007\/978-3-642-33826-7_16"},{"key":"10146_CR17","doi-asserted-by":"crossref","unstructured":"Cuoq P, Monate B, Pacalet A, Prevosto V, Regehr J, Yakobowski B, Yang X (2012) Testing static analyzers with randomly generated programs. In: Proc. of the 4th international conference on NASA formal methods (NFM\u201912)","DOI":"10.1007\/978-3-642-28891-3_12"},{"key":"10146_CR18","doi-asserted-by":"crossref","unstructured":"Donaldson AF, Evrard H, Lascu A, Thomson P (2017) Automated testing of graphics shader compilers. In: Proc. of the ACM on programming languages (OOPSLA\u201917)","DOI":"10.1145\/3133917"},{"key":"10146_CR19","doi-asserted-by":"crossref","unstructured":"Donaldson AF, Thomson P, Teliman V, Milizia S, Maselco AP, Karpinski A (2021) Test-case reduction and deduplication almost for free with transformation-based compiler testing. In: Proc. of the conference on programing language design and implementation (PLDI\u201921)","DOI":"10.1145\/3453483.3454092"},{"key":"10146_CR20","unstructured":"CsmithEdge - Homepage (2022) https:\/\/srg.doc.ic.ac.uk\/projects\/CsmithEdge\/ (Date Accessed February 24)"},{"key":"10146_CR21","doi-asserted-by":"crossref","unstructured":"Even-Mendoza K, Cadar C, Donaldson A (2020) Closer to the edge: Testing compilers more thoroughly by being less conservative about undefined behaviour. In: Proc. of the 35th IEEE international conference on automated software engineering, new ideas and emerging results (ASE NIER\u201920)","DOI":"10.1145\/3324884.3418933"},{"key":"10146_CR22","unstructured":"Frama-C EVA plugin (2007) https:\/\/frama-c.com\/fc-plugins\/eva.html"},{"key":"10146_CR23","unstructured":"GitHub Y (2018) Git repository of Yarpgen https:\/\/github.com\/intel\/yarpgen [Online; accessed 24-February-2022]"},{"key":"10146_CR24","unstructured":"GitHub (2020a) Csmith pull request 86 https:\/\/github.com\/csmith-project\/csmith\/pull\/86 [Online; accessed 24-February-2022]"},{"key":"10146_CR25","unstructured":"GitHub (2020b) Csmith pull request 88 https:\/\/github.com\/csmith-project\/csmith\/pull\/88 [Online; accessed 24-February-2022]"},{"key":"10146_CR26","unstructured":"GitHub (2021a) CsmithEdge bugs details. https:\/\/github.com\/karineek\/CsmithEdge\/tree\/master\/results\/bugs [Online; accessed 24-February-2022]"},{"key":"10146_CR27","unstructured":"GitHub (2021b) CsmithEdge repository. https:\/\/github.com\/karineek\/CsmithEdge.git [Online; accessed 24-February-2022]"},{"key":"10146_CR28","unstructured":"GitHub (2011a) Git repository of Csmith https:\/\/github.com\/csmith-project\/csmith.git [Online; accessed 24-February-2022]"},{"key":"10146_CR29","unstructured":"GitHub (2011b) Git repository of Csmith, commit 7e33250, Csmith\u2019s options for testing static analyzers. https:\/\/github.com\/csmith-project\/csmith\/commit\/7e3325060b56cc5813b8701087b5206fb394c047, [Online; accessed 24-February-2022]"},{"key":"10146_CR30","unstructured":"GitHub (2018) Git repository of gfauto https:\/\/github.com\/google\/graphicsfuzz.git [Online; accessed 24-February-2022]"},{"key":"10146_CR31","doi-asserted-by":"crossref","unstructured":"Groce A, Holzmann GJ, Joshi R (2007) Randomized differential testing as a prelude to formal verification. In: Proc. of the 29th international conference on software engineering (ICSE\u201907). IEEE Computer Society, pp 621\u2013631","DOI":"10.1109\/ICSE.2007.68"},{"key":"10146_CR32","doi-asserted-by":"crossref","unstructured":"Groce A, Zhang C, Eide E, Chen Y, Regehr J (2012) Swarm testing. In: Proc. of the international symposium on software testing and analysis (ISSTA\u201912), pp 78\u201388","DOI":"10.1145\/2338965.2336763"},{"key":"10146_CR33","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1147\/sj.94.0242","volume":"9","author":"K Hanford","year":"1970","unstructured":"Hanford K (1970) Automatic generation of test cases. IBM Syst J 9:242\u2013257","journal-title":"IBM Syst J"},{"key":"10146_CR34","unstructured":"International Organization for Standardization (2018) ISO\/IEC 9899:2018: Programming Languages\u2014C"},{"key":"10146_CR35","doi-asserted-by":"crossref","unstructured":"Klees G, Ruef A, Cooper B, Wei S, Hicks M (2018) Evaluating fuzz testing. In: Proc. of the 24th ACM conference on computer and communications security (CCS\u201918), p 2123\u20132138","DOI":"10.1145\/3243734.3243804"},{"key":"10146_CR36","doi-asserted-by":"crossref","unstructured":"Le V, Afshari M, Su Z (2014) Compiler validation via equivalence modulo inputs. In: Proc. of the conference on programing language design and implementation (PLDI\u201914)","DOI":"10.1145\/2594291.2594334"},{"key":"10146_CR37","doi-asserted-by":"crossref","unstructured":"Le V, Sun C, Su Z (2015) Finding deep compiler bugs via guided stochastic program mutation. In: Proc. of the 30th annual conference on object-oriented programming systems, languages and applications (OOPSLA\u201915)","DOI":"10.1145\/2814270.2814319"},{"issue":"7","key":"10146_CR38","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1145\/1538788.1538814","volume":"52","author":"X Leroy","year":"2009","unstructured":"Leroy X (2009) Formal verification of a realistic compiler. Communications of the Association for Computing Machinery (CACM) 52 (7):107\u2013115","journal-title":"Communications of the Association for Computing Machinery (CACM)"},{"key":"10146_CR39","doi-asserted-by":"crossref","unstructured":"Lidbury C, Lascu A, Chong N, Donaldson AF (2015) Many-core compiler fuzzing. In: Proc. of the conference on programing language design and implementation (PLDI\u201915)","DOI":"10.1145\/2737924.2737986"},{"key":"10146_CR40","doi-asserted-by":"crossref","unstructured":"Livinskii V, Babokin D, Regehr J (2020) Random testing for C and C++ compilers with YARPGen. In: Proc. of the ACM on programming languages (OOPSLA\u201920), vol 4, pp 196:1\u2013196:25","DOI":"10.1145\/3428264"},{"key":"10146_CR41","doi-asserted-by":"crossref","unstructured":"Marcozzi M, Tang Q, Donaldson A, Cadar C (2019) Compiler fuzzing: How much does it matter?. In: Proc. of the ACM on programming languages (OOPSLA\u201919)","DOI":"10.1145\/3360581"},{"key":"10146_CR42","first-page":"100","volume":"10","author":"WM McKeeman","year":"1998","unstructured":"McKeeman WM (1998) Differential testing for software. Digit Tech J 10:100\u2013107","journal-title":"Digit Tech J"},{"key":"10146_CR43","unstructured":"Memory Sanitizer (2015) https:\/\/clang.llvm.org\/docs\/MemorySanitizer.html"},{"key":"10146_CR44","doi-asserted-by":"publisher","first-page":"91","DOI":"10.2197\/ipsjtsldm.7.91","volume":"7","author":"E Nagai","year":"2014","unstructured":"Nagai E, Hashimoto A, Ishiura N (2014) Reinforcing random testing of arithmetic optimization of C compilers by scaling up size and number of expressions. IPSJ Transactions on System LSI Design Methodology 7:91\u2013100","journal-title":"IPSJ Transactions on System LSI Design Methodology"},{"key":"10146_CR45","doi-asserted-by":"crossref","unstructured":"Nakamura K, Ishiura N (2016) Random testing of C compilers based on test program generation by equivalence transformation. In: 2016 IEEE Asia pacific conference on circuits and systems (APCCAS)","DOI":"10.1109\/APCCAS.2016.7804063"},{"key":"10146_CR46","unstructured":"Regehr J (2019) Comment on running one million Csmith programs https:\/\/twitter.com\/johnregehr\/status\/1134866965028196352 [Online; accessed 24-February-2022]"},{"key":"10146_CR47","doi-asserted-by":"crossref","unstructured":"Regehr J, Chen Y, Cuoq P, Eide E, Ellison C, Yang X (2012) Test-case reduction for C compiler bugs. In: Proc. of the conference on programing language design and implementation (PLDI\u201912)","DOI":"10.1145\/2254064.2254104"},{"key":"10146_CR48","doi-asserted-by":"crossref","unstructured":"Sauder RL (1962) A general test data generator for COBOL. In: Proc. of the 1962 spring joint computer conference (AIEE-IRE\u201962 spring)","DOI":"10.1145\/1460833.1460869"},{"key":"10146_CR49","doi-asserted-by":"crossref","unstructured":"Segura S, Fraser G, Sanchez A, Ruiz-cort\u00e9s A (2016) A survey on metamorphic testing","DOI":"10.1109\/TSE.2016.2532875"},{"key":"10146_CR50","unstructured":"Serebryany K, Bruening D, Potapenko A, Vyukov D (2012) Addresssanitizer: A fast address sanity checker. In: Proc. of the 2012 USENIX annual technical conference (USENIX ATC\u201912)"},{"key":"10146_CR51","doi-asserted-by":"crossref","unstructured":"Stepanov E, Serebryany K (2015) Memorysanitizer: fast detector of uninitialized memory use in C++. In: Proc. of the international symposium on code generation and optimization (CGO\u201915)","DOI":"10.1109\/CGO.2015.7054186"},{"key":"10146_CR52","doi-asserted-by":"crossref","unstructured":"Sun C, Le V, Su Z (2016) Finding compiler bugs via live code mutation. In: Proc. of the 31st annual conference on object-oriented programming systems, languages and applications (OOPSLA\u201916)","DOI":"10.1145\/2983990.2984038"},{"key":"10146_CR53","unstructured":"Undefined Behavior Sanitizer (2017) https:\/\/clang.llvm.org\/docs\/UndefinedBehaviorSanitizer.html"},{"key":"10146_CR54","unstructured":"Visual Studio Developer Community (2021) Bug 1485361: Msvc miscompiles program with unreachable out of bounds access at \/o2 https:\/\/developercommunity.visualstudio.com\/t\/msvc-miscompiles-program-with-unreachable-out-of-b\/1485361 [Online; accessed 24-February-2022]"},{"key":"10146_CR55","doi-asserted-by":"crossref","unstructured":"Wang X, Zeldovich N, Kaashoek F, Solar-Lezama A (2013) Towards optimization-safe systems: Analyzing the impact of undefined behavior. In: Proc. of the 24th ACM symposium on operating systems principles (SOSP\u201913)","DOI":"10.1145\/2517349.2522728"},{"key":"10146_CR56","doi-asserted-by":"crossref","unstructured":"Yang X, Chen Y, Eide E, Regehr J (2011) Finding and understanding bugs in C compilers. In: Proc. of the conference on programing language design and implementation (PLDI\u201911)","DOI":"10.1145\/1993498.1993532"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10146-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-022-10146-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-022-10146-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,9,26]],"date-time":"2022-09-26T08:19:35Z","timestamp":1664180375000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-022-10146-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,8]]},"references-count":56,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,11]]}},"alternative-id":["10146"],"URL":"https:\/\/doi.org\/10.1007\/s10664-022-10146-1","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,8]]},"assertion":[{"value":"14 March 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"129"}}