{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T02:32:11Z","timestamp":1769740331719,"version":"3.49.0"},"reference-count":73,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T00:00:00Z","timestamp":1705276800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T00:00:00Z","timestamp":1705276800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"crossref","award":["200020-188613"],"award-info":[{"award-number":["200020-188613"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>As research in automatically detecting bugs grows and produces new techniques, having suitable collections of programs with known bugs becomes crucial to reliably and meaningfully compare the effectiveness of these techniques. Most of the existing approaches rely on <jats:italic>benchmarks<\/jats:italic> collecting manually curated real-world bugs, or synthetic bugs seeded into real-world programs. Using real-world programs entails that extending the existing benchmarks or creating new ones remains a complex time-consuming task. In this paper, we propose a complementary approach that automatically generates programs with seeded bugs. Our technique, called HyperPUT, builds C\u00a0programs from a \u201cseed\u201d bug by incrementally applying program transformations (introducing programming constructs such as conditionals, loops, etc.) until a program of the desired size is generated. In our experimental evaluation, we demonstrate how HyperPUT can generate buggy programs that can challenge in different ways the capabilities of modern bug-finding tools, and some of whose characteristics are comparable to those of bugs in existing benchmarks. These results suggest that HyperPUT can be a useful tool to support further research in bug-finding techniques\u2014in particular their empirical evaluation.<\/jats:p>","DOI":"10.1007\/s10664-023-10430-8","type":"journal-article","created":{"date-parts":[[2024,1,15]],"date-time":"2024-01-15T11:02:50Z","timestamp":1705316570000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["HyperPUT: generating synthetic faulty programs to challenge bug-finding tools"],"prefix":"10.1007","volume":"29","author":[{"given":"Riccardo","family":"Felici","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Laura","family":"Pozzi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carlo A.","family":"Furia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,1,15]]},"reference":[{"key":"10430_CR1","volume-title":"Introduction to software testing","author":"P Ammann","year":"2007","unstructured":"Ammann P, Offutt J (2007) Introduction to software testing, 2nd edn. Cambridge University Press","edition":"2"},{"key":"10430_CR2","doi-asserted-by":"crossref","unstructured":"Aschermann C, Schumilo S, Blazytko T, Gawlik R, Holz T (2019) REDQUEEN: fuzzing with input-to-state correspondence. In: 26th annual network and distributed system security symposium, NDSS 2019. The Internet Society, San Diego. https:\/\/www.ndss-symposium.org\/ndss-paper\/redqueen-fuzzing-with-input-to-state-correspondence\/","DOI":"10.14722\/ndss.2019.23371"},{"key":"10430_CR3","doi-asserted-by":"publisher","unstructured":"Babic D, Bucur S, Chen Y, Ivancic F, King T, Kusano M, Lemieux C, Szekeres L, Wang W (2019) FUDGE: fuzz driver generation at scale. In: Dumas M, Pfahl D, Apel S, Russo A (eds) Proceedings of the ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/SIGSOFT FSE 2019. ACM, Tallinn, pp 975\u2013985. https:\/\/doi.org\/10.1145\/3338906.3340456","DOI":"10.1145\/3338906.3340456"},{"key":"10430_CR4","doi-asserted-by":"publisher","unstructured":"Baldoni R, Coppa E, D\u2019Elia DC, Demetrescu C, Finocchi I (2018) A survey of symbolic execution techniques. ACM Comput Surv 51(3):50:1\u201350:39. https:\/\/doi.org\/10.1145\/3182657","DOI":"10.1145\/3182657"},{"key":"10430_CR5","unstructured":"Bame P (2023) McCabe cyclomatic complexity for C and Cpp. https:\/\/manpages.ubuntu.com\/manpages\/jammy\/en\/man1\/pmccabe.1.html. Accessed 08 Aug 2023"},{"key":"10430_CR6","doi-asserted-by":"crossref","unstructured":"Banerjee S, Clapp L, Sridharan M (2019) NullAway: practical type-based null safety for Java. In: Proceedings of the ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/SIGSOFT FSE 2019. ACM, Tallinn, pp 740\u2013750","DOI":"10.1145\/3338906.3338919"},{"key":"10430_CR7","unstructured":"Bellard F (2005) QEMU, a fast and portable dynamic translator. In: Proceedings of the FREENIX track: 2005 USENIX annual technical conference. USENIX, Anaheim, pp 41\u201346. http:\/\/www.usenix.org\/events\/usenix05\/tech\/freenix\/bellard.html"},{"key":"10430_CR8","doi-asserted-by":"publisher","unstructured":"Beyer D (2021) Software verification: 10th comparative evaluation (SV-COMP 2021). In: Groote JF, Larsen KG (eds) Tools and algorithms for the construction and analysis of systems - 27th international conference, TACAS 2021, held as part of the European joint conferences on theory and practice of software, ETAS 2021, proceedings, part II, lecture notes in computer science, vol 12652. Springer, Luxembourg, pp 401\u2013422. https:\/\/doi.org\/10.1007\/978-3-030-72013-1_24","DOI":"10.1007\/978-3-030-72013-1_24"},{"key":"10430_CR9","doi-asserted-by":"publisher","unstructured":"Beyer D (2021a) Status report on software testing: test-comp 2021. In: Guerra E, Stoelinga M (eds) Fundamental approaches to software engineering - 24th international conference, FASE 2021, held as part of the European joint conferences on theory and practice of software, ETAPS 2021, proceedings, lecture notes in computer science, vol 12649. Springer, Luxembourg, pp 341\u2013357. https:\/\/doi.org\/10.1007\/978-3-030-71500-7_17","DOI":"10.1007\/978-3-030-71500-7_17"},{"key":"10430_CR10","doi-asserted-by":"publisher","unstructured":"Blackburn SM, Garner R, Hoffmann C, Khan AM, McKinley KS, Bentzur R, Diwan A, Feinberg D, Frampton D, Guyer SZ, Hirzel M, Hosking AL, Jump M, Lee HB, Moss JEB, Phansalkar A, Stefanovic D, VanDrunen T, von Dincklage D, Wiedermann B (2006) The DaCapo benchmarks: Java benchmarking development and analysis. In: Tarr PL, Cook WR (eds) Proceedings of the 21th Annual ACM SIGPLAN conference on object-oriented programming, systems, languages, and applications, OOPSLA 2006. ACM, Portland, pp 169\u2013190. https:\/\/doi.org\/10.1145\/1167473.1167488","DOI":"10.1145\/1167473.1167488"},{"issue":"3","key":"10430_CR11","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1109\/MS.2020.3016773","volume":"38","author":"M B\u00f6hme","year":"2021","unstructured":"B\u00f6hme M, Cadar C, Roychoudhury A (2021) Fuzzing: challenges and reflections. IEEE Softw 38(3):79\u201386. https:\/\/doi.org\/10.1109\/MS.2020.3016773","journal-title":"IEEE Softw"},{"key":"10430_CR12","doi-asserted-by":"publisher","unstructured":"B\u00f6hme M, Pham V, Roychoudhury A (2016) Coverage-based grey box fuzzing as Markov chain. In: Weippl ER, Katzenbeisser S, Kruegel C, Myers AC, Halevi S (eds) Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. ACM, Vienna, pp 1032\u20131043. https:\/\/doi.org\/10.1145\/2976749.2978428","DOI":"10.1145\/2976749.2978428"},{"key":"10430_CR13","doi-asserted-by":"publisher","unstructured":"Bornholt J, Torlak E (2018) Finding code that explodes under symbolic evaluation. Proc ACM Program Lang 2(OOPSLA):149:1\u2013149:26. https:\/\/doi.org\/10.1145\/3276519","DOI":"10.1145\/3276519"},{"key":"10430_CR14","doi-asserted-by":"publisher","unstructured":"Bowes D, Hall T, Harman M, Jia Y, Sarro F, Wu F (2016) Mutation-aware fault prediction. In: Proceedings of the 25th international symposium on software testing and analysis, ISSTA 2016. ACM, Saarbr\u00fccken, pp 330\u2013341. https:\/\/doi.org\/10.1145\/2931037.2931039","DOI":"10.1145\/2931037.2931039"},{"key":"10430_CR15","unstructured":"Busybox (2023). https:\/\/www.busybox.net\/. Accessed 08 Aug 2023"},{"issue":"2","key":"10430_CR16","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1145\/2408776.2408795","volume":"56","author":"C Cadar","year":"2013","unstructured":"Cadar C, Sen K (2013) Symbolic execution for software testing: three decades later. Commun ACM 56(2):82\u201390. https:\/\/doi.org\/10.1145\/2408776.2408795","journal-title":"Commun ACM"},{"key":"10430_CR17","unstructured":"Cadar C, Dunbar D, Engler DR (2008) KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs. In: Draves R, van Renesse R (eds) Proceedings 8th USENIX symposium on operating systems design and implementation, OSDI 2008. USENIX Association, San Diego, pp 209\u2013224. http:\/\/www.usenix.org\/events\/osdi08\/tech\/full_papers\/cadar\/cadar.pdf"},{"key":"10430_CR18","doi-asserted-by":"publisher","unstructured":"Cadar C, Ganesh V, Pawlowski PM, Dill DL, Engler DR (2008) EXE: automatically generating inputs of death. ACM Trans Inf Syst Secur 12(2):10:1\u201310:38. https:\/\/doi.org\/10.1145\/1455518.1455522","DOI":"10.1145\/1455518.1455522"},{"key":"10430_CR19","doi-asserted-by":"publisher","unstructured":"Candea G, Godefroid P (2019) Automated software test generation: some challenges, solutions, and recent advances. In: Steffen B, Woeginger GJ (eds) Computing and software science - state of the art and perspectives, lecture notes in computer science, vol 10000. Springer, pp 505\u2013531. https:\/\/doi.org\/10.1007\/978-3-319-91908-9_24","DOI":"10.1007\/978-3-319-91908-9_24"},{"key":"10430_CR20","doi-asserted-by":"publisher","unstructured":"Chipounov V, Kuznetsov V, Candea G (2012) The S2E platform: design, implementation, and applications. ACM Trans Comput Syst 30(1):2:1\u20132:49. https:\/\/doi.org\/10.1145\/2110356.2110358","DOI":"10.1145\/2110356.2110358"},{"key":"10430_CR21","unstructured":"Circumventing fuzzing roadblocks with compiler transformations. https:\/\/lafintel.wordpress.com\/ (2016). Accessed 08 Aug 2023"},{"key":"10430_CR22","doi-asserted-by":"publisher","unstructured":"Clarke EM, Kroening D, Lerda F (2004) A tool for checking ANSI-C programs. In: Jensen K, Podelski A (eds) Tools and algorithms for the construction and analysis of systems, 10th international conference, TACAS 2004, held as part of the joint European conferences on theory and practice of software, ETAPS 2004, proceedings, lecture notes in computer science, vol 2988. Springer, Barcelona, pp 168\u2013176. https:\/\/doi.org\/10.1007\/978-3-540-24730-2_15","DOI":"10.1007\/978-3-540-24730-2_15"},{"issue":"2","key":"10430_CR23","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1145\/5397.5399","volume":"8","author":"EM Clarke","year":"1986","unstructured":"Clarke EM, Emerson EA, Sistla AP (1986) Automatic verification of finite-state concurrent systems using temporal logic specifications. ACM Trans Program Lang Syst 8(2):244\u2013263. https:\/\/doi.org\/10.1145\/5397.5399","journal-title":"ACM Trans Program Lang Syst"},{"key":"10430_CR24","unstructured":"Cmplog instrumentation (2023). https:\/\/github.com\/AFLplusplus\/AFLplusplus\/blob\/stable\/instrumentation\/README.cmplog.md. Accessed 08 Aug 2023"},{"issue":"5","key":"10430_CR25","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1145\/1941487.1941509","volume":"54","author":"B Cook","year":"2011","unstructured":"Cook B, Podelski A, Rybalchenko A (2011) Proving program termination. Commun ACM 54(5):88\u201398. https:\/\/doi.org\/10.1145\/1941487.1941509","journal-title":"Commun ACM"},{"key":"10430_CR26","unstructured":"DARPA CGC (2018). https:\/\/github.com\/CyberGrandChallenge\/. Accessed 08 Aug 2023"},{"issue":"4","key":"10430_CR27","doi-asserted-by":"publisher","first-page":"405","DOI":"10.1007\/s10664-005-3861-2","volume":"10","author":"H Do","year":"2005","unstructured":"Do H, Elbaum SG, Rothermel G (2005) Supporting controlled experimentation with testing techniques: an infrastructure and its potential impact. Empir Softw Eng 10(4):405\u2013435. https:\/\/doi.org\/10.1007\/s10664-005-3861-2","journal-title":"Empir Softw Eng"},{"key":"10430_CR28","doi-asserted-by":"publisher","unstructured":"Dolan-Gavitt B, Hodosh J, Hulin P, Leek T, Whelan R (2015) Repeatable reverse engineering with PANDA. In: McDonald JT, Preda MD, Stakhanova N (eds) Proceedings of the 5th program protection and reverse engineering workshop, PPREW@ACSAC. ACM, Los Angeles, pp 4:1\u20134:11. https:\/\/doi.org\/10.1145\/2843859.2843867","DOI":"10.1145\/2843859.2843867"},{"key":"10430_CR29","doi-asserted-by":"publisher","unstructured":"Dolan-Gavitt B, Hulin P, Kirda E, Leek T, Mambretti A, Robertson WK, Ulrich F, Whelan R (2016) LAVA: large-scale automated vulnerability addition. In: IEEE symposium on security and privacy, SP 2016. IEEE Computer Society, San Jose, pp 110\u2013121. https:\/\/doi.org\/10.1109\/SP.2016.15","DOI":"10.1109\/SP.2016.15"},{"key":"10430_CR30","doi-asserted-by":"publisher","unstructured":"Even-Mendoza K, Cadar C, Donaldson AF (2020) Closer to the edge: testing compilers more thoroughly by being less conservative about undefined behaviour. In: 35th IEEE\/ACM international conference on automated software engineering, ASE 2020. IEEE, Melbourne, pp 1219\u20131223. https:\/\/doi.org\/10.1145\/3324884.3418933","DOI":"10.1145\/3324884.3418933"},{"key":"10430_CR31","doi-asserted-by":"publisher","unstructured":"Ferrer J, Chicano F, Alba E (2011) Benchmark generator for software testers. In: Iliadis LS, Maglogiannis I, Papadopoulos H (eds) Artificial intelligence applications and innovations - 12th INNS EANN-SIG international conference, EANN 2011 and 7th IFIP WG 12.5 international conference, AIAI 2011, proceedings, part II, IFIP advances in information and communication technology, vol 364. Springer, Corfu, pp 378\u2013388. https:\/\/doi.org\/10.1007\/978-3-642-23960-1_45","DOI":"10.1007\/978-3-642-23960-1_45"},{"key":"10430_CR32","unstructured":"Fioraldi A, Maier D, Ei\u00dffeldt H, Heuse M (2020) AFL++ : combining incremental steps of fuzzing research. In: Yarom Y, Zennou S (eds) 14th USENIX workshop on offensive technologies, WOOT 2020. USENIX Association. https:\/\/www.usenix.org\/conference\/woot20\/presentation\/fioraldi"},{"issue":"2","key":"10430_CR33","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1109\/TSE.2011.93","volume":"38","author":"G Fraser","year":"2012","unstructured":"Fraser G, Zeller A (2012) Mutation-driven generation of unit tests and oracles. IEEE Trans Softw Eng 38(2):278\u2013292. https:\/\/doi.org\/10.1109\/TSE.2011.93","journal-title":"IEEE Trans Softw Eng"},{"issue":"3","key":"10430_CR34","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1145\/2093548.2093564","volume":"55","author":"P Godefroid","year":"2012","unstructured":"Godefroid P, Levin MY, Molnar DA (2012) SAGE: whitebox fuzzing for security testing. Commun ACM 55(3):40\u201344. https:\/\/doi.org\/10.1145\/2093548.2093564","journal-title":"Commun ACM"},{"key":"10430_CR35","doi-asserted-by":"publisher","unstructured":"Godefroid P, Klarlund N, Sen K (2005) DART: directed automated random testing. In: Sarkar V, Hall MW (eds) Proceedings of the ACM SIGPLAN 2005 conference on programming language design and implementation. ACM, Chicago, pp 213\u2013223. https:\/\/doi.org\/10.1145\/1065010.1065036","DOI":"10.1145\/1065010.1065036"},{"key":"10430_CR36","doi-asserted-by":"publisher","unstructured":"Hazimeh A, Herrera A, Payer M (2021) Magma: a ground-truth fuzzing benchmark. In: Huang L, Gandhi A, Kiyavash N, Wang J (eds) SIGMETRICS \u201921: ACM SIGMETRICS\/international conference on measurement and modeling of computer systems, virtual event. ACM, China, pp 81\u201382. https:\/\/doi.org\/10.1145\/3410220.3456276","DOI":"10.1145\/3410220.3456276"},{"key":"10430_CR37","unstructured":"HyperPUT (2022). github. https:\/\/github.com\/user28134zx2734\/HyperPUT. Accessed 08 Aug 2023"},{"key":"10430_CR38","doi-asserted-by":"publisher","unstructured":"Just R, Jalali D, Ernst M (2014) Defects4J: a database of existing faults to enable controlled testing studies for Java programs. https:\/\/doi.org\/10.1145\/2610384.2628055","DOI":"10.1145\/2610384.2628055"},{"key":"10430_CR39","doi-asserted-by":"publisher","unstructured":"Kapus T, Cadar C (2017) Automatic testing of symbolic execution engines via program generation and differential testing. In: Rosu G, Penta MD, Nguyen TN (eds) Proceedings of the 32nd IEEE\/ACM international conference on automated software engineering, ASE 2017. IEEE Computer Society, Urbana, pp 590\u2013600. https:\/\/doi.org\/10.1109\/ASE.2017.8115669","DOI":"10.1109\/ASE.2017.8115669"},{"key":"10430_CR40","doi-asserted-by":"publisher","unstructured":"Klees G, Ruef A, Cooper B, Wei S, Hicks M (2018) Evaluating fuzz testing. In: Lie D, Mannan M, Backes M, Wang X (eds) Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, CCS 2018. ACM, Toronto, pp 2123\u20132138. https:\/\/doi.org\/10.1145\/3243734.3243804","DOI":"10.1145\/3243734.3243804"},{"key":"10430_CR41","doi-asserted-by":"publisher","unstructured":"Kochhar PS, Thung F, Lo D, Lawall J (2014) An empirical study on the adequacy of testing in open source projects. In: Cha SS, Gu\u00e9h\u00e9neuc Y, Kwon G (eds) 21st Asia-pacific software engineering conference, APSEC 2014, research papers, vol 1. IEEE Computer Society, Jeju, pp 215\u2013222. https:\/\/doi.org\/10.1109\/APSEC.2014.42","DOI":"10.1109\/APSEC.2014.42"},{"key":"10430_CR42","doi-asserted-by":"publisher","unstructured":"Kontar KA, Naji F, Demiane S, Haraty R (2019) A survey on mutation testing approaches. In: 2019 IEEE CHILEAN conference on electrical, electronics engineering, information and communication technologies (CHILECON), pp 1\u20137. https:\/\/doi.org\/10.1109\/CHILECON47746.2019.8987448","DOI":"10.1109\/CHILECON47746.2019.8987448"},{"key":"10430_CR43","doi-asserted-by":"publisher","unstructured":"Kusano M, Wang C (2013) CCmutator: a mutation generator for concurrency constructs in multithreaded C\/C++ applications. In: Denney E, Bultan T, Zeller A (eds) 2013 28th IEEE\/ACM international conference on automated software engineering, ASE 2013. IEEE, Silicon Valley, pp 722\u2013725. https:\/\/doi.org\/10.1109\/ASE.2013.6693142","DOI":"10.1109\/ASE.2013.6693142"},{"key":"10430_CR44","doi-asserted-by":"publisher","unstructured":"Lamport L (1977) Proving the correctness of multiprocess programs. IEEE Trans Softw Eng SE-3(2):125\u2013143. https:\/\/doi.org\/10.1109\/TSE.1977.229904","DOI":"10.1109\/TSE.1977.229904"},{"key":"10430_CR45","doi-asserted-by":"publisher","unstructured":"Lanza M, Marinescu R (2006) Object-oriented metrics in practice - using software metrics to characterize, evaluate, and improve the design of object-oriented systems. Springer. https:\/\/doi.org\/10.1007\/3-540-39538-5","DOI":"10.1007\/3-540-39538-5"},{"key":"10430_CR46","doi-asserted-by":"publisher","unstructured":"Lattner C, Adve VS (2004) LLVM: a compilation framework for lifelong program analysis & transformation. In: 2nd IEEE\/ACM international symposium on code generation and optimization (CGO 2004). IEEE Computer Society, San Jose, pp 75\u201388. https:\/\/doi.org\/10.1109\/CGO.2004.1281665","DOI":"10.1109\/CGO.2004.1281665"},{"key":"10430_CR47","unstructured":"Littlefair T (2005) C and Cpp code counter. http:\/\/cccc.sourceforge.net\/. Accessed Aug 08 2023"},{"key":"10430_CR48","unstructured":"Lyu C, Ji S, Zhang C, Li Y, Lee W, Song Y, Beyah R (2019) MOPT: optimized mutation scheduling for fuzzers. In: Heninger N, Traynor P (eds) 28th USENIX security symposium, USENIX security 2019. USENIX Association, Santa Clara, pp. 1949\u20131966. https:\/\/www.usenix.org\/conference\/usenixsecurity19\/presentation\/lyu"},{"issue":"4","key":"10430_CR49","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1007\/s10664-023-10330-x","volume":"28","author":"J Malik","year":"2023","unstructured":"Malik J, Pastore F (2023) An empirical study of vulnerabilities in edge frameworks to support security testing improvement. Empir Softw Eng 28(4):99. https:\/\/doi.org\/10.1007\/s10664-023-10330-x","journal-title":"Empir Softw Eng"},{"issue":"11","key":"10430_CR50","doi-asserted-by":"publisher","first-page":"2312","DOI":"10.1109\/TSE.2019.2946563","volume":"47","author":"VJM Man\u00e8s","year":"2021","unstructured":"Man\u00e8s VJM, Han H, Han C, Cha SK, Egele M, Schwartz EJ, Woo M (2021) The art, science, and engineering of fuzzing: a survey. IEEE Trans Softw Eng 47(11):2312\u20132331. https:\/\/doi.org\/10.1109\/TSE.2019.2946563","journal-title":"IEEE Trans Softw Eng"},{"key":"10430_CR51","doi-asserted-by":"publisher","unstructured":"Marcozzi M, Tang Q, Donaldson AF, Cadar C (2019) Compiler fuzzing: how much does it matter? Proc ACM Program Lang 3(OOPSLA):155:1\u2013155:29. https:\/\/doi.org\/10.1145\/3360581","DOI":"10.1145\/3360581"},{"issue":"4","key":"10430_CR52","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.1976.233837","volume":"2","author":"TJ McCabe","year":"1976","unstructured":"McCabe TJ (1976) A complexity measure. IEEE Trans Softw Eng 2(4):308\u2013320. https:\/\/doi.org\/10.1109\/TSE.1976.233837","journal-title":"IEEE Trans Softw Eng"},{"key":"10430_CR53","unstructured":"McKeeman WM (1998) Differential testing for software. Digit Tech J 10(1):100\u2013107. http:\/\/www.hpl.hp.com\/hpjournal\/dtj\/vol10num1\/vol10num1art9.pdf"},{"key":"10430_CR54","doi-asserted-by":"publisher","unstructured":"Metzman J, Szekeres L, Simon L, Sprabery R, Arya A (2021) FuzzBench: an open fuzzer benchmarking platform and service. In: Spinellis D, Gousios G, Chechik M, Penta MD (eds) ESEC\/FSE \u201921: 29th ACM joint European software engineering conference and symposium on the foundations of software engineering. ACM, Athens, pp 1393\u20131403. https:\/\/doi.org\/10.1145\/3468264.3473932","DOI":"10.1145\/3468264.3473932"},{"key":"10430_CR55","doi-asserted-by":"publisher","unstructured":"Moura LMD, Bj\u00f8rner N (2008) Z3: An Efficient SMT Solver. In: Ramakrishnan CR, Rehof J (eds) Tools and algorithms for the construction and analysis of systems, 14th international conference, TACAS 2008, held as part of the joint European conferences on theory and practice of software, ETAPS 2008, proceedings, lecture notes in computer science, vol 4963. Springer, Budapest, pp 337\u2013340. https:\/\/doi.org\/10.1007\/978-3-540-78800-3_24","DOI":"10.1007\/978-3-540-78800-3_24"},{"key":"10430_CR56","unstructured":"Nethercote N (2004) Dynamic binary analysis and instrumentation: or building tools is easy. Ph.D. thesis, University of Cambridge, UK"},{"issue":"5\u20137","key":"10430_CR57","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1002\/stvr.1509","volume":"25","author":"M Papadakis","year":"2015","unstructured":"Papadakis M, Le Traon Y (2015) Metallaxis-FL: mutation-based fault localization. Softw Test Verification Reliab 25(5\u20137):605\u2013628. https:\/\/doi.org\/10.1002\/stvr.1509","journal-title":"Softw Test Verification Reliab"},{"key":"10430_CR58","doi-asserted-by":"publisher","unstructured":"Papadakis M, Jia Y, Harman M, Traon YL (2015) Trivial compiler equivalence: a large scale empirical study of a simple, fast and effective equivalent mutant detection technique. In: 37th IEEE\/ACM international conference on software engineering, ICSE 2015, vol 1. IEEE Computer Society, Florence, pp 936\u2013946. https:\/\/doi.org\/10.1109\/ICSE.2015.103","DOI":"10.1109\/ICSE.2015.103"},{"issue":"1","key":"10430_CR59","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/MSEC.2018.2889892","volume":"17","author":"M Payer","year":"2019","unstructured":"Payer M (2019) The fuzzing hype-train: how random testing triggers thousands of crashes. IEEE Secur Priv 17(1):78\u201382. https:\/\/doi.org\/10.1109\/MSEC.2018.2889892","journal-title":"IEEE Secur Priv"},{"key":"10430_CR60","doi-asserted-by":"publisher","unstructured":"Peng H, Shoshitaishvili Y, Payer M (2018) T-Fuzz: fuzzing by program transformation. In: (2018) IEEE symposium on security and privacy, SP 2018, proceedings. IEEE Computer Society, San Francisco, pp 697\u2013710. https:\/\/doi.org\/10.1109\/SP.2018.00056","DOI":"10.1109\/SP.2018.00056"},{"key":"10430_CR61","unstructured":"Pezz\u00e8 M, Young M (2007) Software testing and analysis. Process, principles and techniques. Wiley"},{"key":"10430_CR62","doi-asserted-by":"publisher","unstructured":"Roy S, Pandey A, Dolan-Gavitt B, Hu Y (2018) Bug synthesis: challenging bug-finding tools with deep faults. In: Leavens GT, Garcia A, Pasareanu CS (eds) Proceedings of the 2018 ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering, ESEC\/SIGSOFT FSE 2018. ACM, Lake Buena Vista, pp 224\u2013234. https:\/\/doi.org\/10.1145\/3236024.3236084","DOI":"10.1145\/3236024.3236084"},{"issue":"5","key":"10430_CR63","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1002\/stvr.1473","volume":"23","author":"D Schuler","year":"2013","unstructured":"Schuler D, Zeller A (2013) Covering and uncovering equivalent mutants. Softw Test Verification Reliab 23(5):353\u2013374. https:\/\/doi.org\/10.1002\/stvr.1473","journal-title":"Softw Test Verification Reliab"},{"key":"10430_CR64","doi-asserted-by":"publisher","unstructured":"Shoshitaishvili Y, Wang R, Salls C, Stephens N, Polino M, Dutcher A, Grosen J, Feng S, Hauser C, Kr\u00fcgel C, Vigna G (2016) SOK: (state of) the art of war: offensive techniques in binary analysis. In: IEEE symposium on security and privacy, SP 2016. IEEE Computer Society, San Jose, pp 138\u2013157. https:\/\/doi.org\/10.1109\/SP.2016.17","DOI":"10.1109\/SP.2016.17"},{"key":"10430_CR65","unstructured":"Stallman R (2023) The GCC developer\u00a0community: using the GNU compiler collection (GCC). GCC version 10.2.0. https:\/\/gcc.gnu.org\/onlinedocs\/gcc\/. Accessed 08 Aug 2023"},{"key":"10430_CR66","doi-asserted-by":"crossref","unstructured":"Stephens N, Grosen J, Salls C, Dutcher A, Wang R, Corbetta J, Shoshitaishvili Y, Kruegel C, Vigna G (2016) Driller: augmenting fuzzing through selective symbolic execution. In: 23rd annual network and distributed system security symposium, NDSS 2016. The Internet Society, San Diego. http:\/\/wp.internetsociety.org\/ndss\/wp-content\/uploads\/sites\/25\/2017\/09\/driller-augmenting-fuzzing-through-selective-symbolic-execution.pdf","DOI":"10.14722\/ndss.2016.23368"},{"key":"10430_CR67","unstructured":"SWE-220 - cyclomatic complexity for safety-critical software. NASA software engineering and assurance handbook. NASA-HDBK-2203. https:\/\/swehb.nasa.gov\/display\/SWEHBVD (2023). Accessed 10 July 2023"},{"key":"10430_CR68","unstructured":"Tanenbaum AS, Appuswamy R, Bos H, Cavallaro L, Giuffrida C, Hrub\u00fd T, Herder JN, van der Kouwe E, van Moolenbroek DC (2010) MINIX 3: status report and current research. Login Usenix Mag 35(3). https:\/\/www.usenix.org\/publications\/login\/june-2010-volume-35-number-3\/minix-3-status-report-and-current-research"},{"key":"10430_CR69","unstructured":"Undefined behavior sanitizer (ubsan) (2023). https:\/\/clang.llvm.org\/docs\/UndefinedBehaviorSanitizer.html. Accessed 08 Aug 2023"},{"key":"10430_CR70","doi-asserted-by":"publisher","unstructured":"Yang X, Chen Y, Eide E, Regehr J (2011) Finding and understanding bugs in C compilers. In: Hall MW, Padua DA (eds) Proceedings of the 32nd ACM SIGPLAN conference on programming language design and implementation, PLDI 2011. ACM, San Jose, pp 283\u2013294. https:\/\/doi.org\/10.1145\/1993498.1993532","DOI":"10.1145\/1993498.1993532"},{"key":"10430_CR71","doi-asserted-by":"publisher","unstructured":"Yao X, Harman M, Jia Y (2014) A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: 36th international conference on software engineering, ICSE \u201914. ACM, Hyderabad, pp 919\u2013930. https:\/\/doi.org\/10.1145\/2568225.2568265","DOI":"10.1145\/2568225.2568265"},{"key":"10430_CR72","doi-asserted-by":"publisher","unstructured":"Younan Y, Joosen W, Piessens F (2012) Runtime countermeasures for code injection attacks against C and C++ programs. ACM Comput Surv 44(3):17:1\u201317:28. https:\/\/doi.org\/10.1145\/2187671.2187679","DOI":"10.1145\/2187671.2187679"},{"key":"10430_CR73","unstructured":"Zalewski M (2016) American fuzzing lop (AFL). http:\/\/lcamtuf.coredump.cx\/afl\/. Accessed 08 Aug 2023"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10430-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-023-10430-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-023-10430-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T02:18:10Z","timestamp":1711160290000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-023-10430-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,15]]},"references-count":73,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10430"],"URL":"https:\/\/doi.org\/10.1007\/s10664-023-10430-8","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,15]]},"assertion":[{"value":"22 November 2023","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing interests that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interest"}}],"article-number":"38"}}