{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T15:18:33Z","timestamp":1783610313671,"version":"3.55.0"},"reference-count":243,"publisher":"Association for Computing Machinery (ACM)","issue":"10","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>\n                    Software defect datasets, which are collections of software bugs, are essential resources to facilitate empirical research and enable standardized benchmarking for a wide range of software engineering techniques, including emerging areas like agentic AI-based software development. Over the years, numerous software defect datasets have been developed, providing rich resources for the community, yet making it increasingly difficult to navigate the landscape. This article provides a comprehensive survey of 151 software defect datasets, covering their scope, construction, availability, usability, and practical uses. We also suggest potential opportunities for future research based on our findings, such as addressing underrepresented kinds of defects. A complete catalog of all surveyed software defect datasets is available at\n                    <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/defect-datasets.github.io\/\">https:\/\/defect-datasets.github.io\/<\/jats:ext-link>\n                    .\n                  <\/jats:p>","DOI":"10.1145\/3797033","type":"journal-article","created":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T21:23:34Z","timestamp":1770845014000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-1492-2446","authenticated-orcid":false,"given":"Hao-Nan","family":"Zhu","sequence":"first","affiliation":[{"name":"University of California, Davis","place":["Davis, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-5373-5901","authenticated-orcid":false,"given":"Robert","family":"Furth","sequence":"additional","affiliation":[{"name":"University of California, Davis","place":["Davis, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1623-498X","authenticated-orcid":false,"given":"Michael","family":"Pradel","sequence":"additional","affiliation":[{"name":"University of Stuttgart","place":["Stuttgart, Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0861-3763","authenticated-orcid":false,"given":"Cindy","family":"Rubio-Gonz\u00e1lez","sequence":"additional","affiliation":[{"name":"University of California, Davis","place":["Davis, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,3,9]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3154717"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","unstructured":"Toufique Ahmed Premkumar T. Devanbu Christoph Treude and Michael Pradel. 2024. Can LLMs replace manual annotation of software engineering artifacts? CoRR abs\/2408.05534 (2024). arXiv:2408.05534. DOI:10.48550\/ARXIV.2408.05534","DOI":"10.48550\/ARXIV.2408.05534"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC53868.2021.00022"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","unstructured":"Reem Aleithan Haoran Xue Mohammad Mahdi Mohajer Elijah Nnorom Gias Uddin and Song Wang. 2024. SWE-Bench+: Enhanced coding benchmark for LLMs. CoRR abs\/2410.06992 (2024). arXiv:2410.06992. DOI:10.48550\/ARXIV.2410.06992","DOI":"10.48550\/ARXIV.2410.06992"},{"issue":"2","key":"e_1_3_2_6_2","first-page":"38","article-title":"Learning how to search: Generating effective test cases through adaptive fitness function selection","volume":"27","author":"Almulla Hussein K.","year":"2022","unstructured":"Hussein K. Almulla and Gregory Gay. 2022. Learning how to search: Generating effective test cases through adaptive fitness function selection. ESE 27, 2 (2022), 38.","journal-title":"ESE"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2903506"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00208"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534370"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/EDCC61798.2024.00016"},{"issue":"5","key":"e_1_3_2_11_2","first-page":"66:1\u201366:36","article-title":"A survey of dynamic analysis and test generation for javascript","volume":"50","author":"Andreasen Esben","year":"2017","unstructured":"Esben Andreasen, Liang Gong, Anders M\u00f8ller, Michael Pradel, Marija Selakovic, Koushik Sen, and Cristian-Alexandru Staicu. 2017. A survey of dynamic analysis and test generation for javascript. ACM CSUR 50, 5 (2017), 66:1\u201366:36.","journal-title":"ACM CSUR"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","unstructured":"G\u00e1bor Antal Norbert V\u00e1ndor Istv\u00e1n Koll\u00e1th Bal\u00e1zs Mosolyg\u00f3 P\u00e9ter Heged\u00fcs and Rudolf Ferenc. 2024. PyBugHive: A comprehensive database of manually validated reproducible Python bugs. IEEE Access 12 (2024) 123739\u2013123756. DOI:10.1109\/ACCESS.2024.3449106","DOI":"10.1109\/ACCESS.2024.3449106"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00040"},{"key":"e_1_3_2_14_2","unstructured":"Association for Computing Machinery (ACM). 2026. Artifact review and badging. Retrieved from https:\/\/www.acm.org\/publications\/policies\/artifactreview-and-badging-current"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00116"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2008.130"},{"key":"e_1_3_2_17_2","first-page":"194","volume-title":"MSR","author":"Azad Md. Abul Kalam","year":"2023","unstructured":"Md. Abul Kalam Azad, Nafees Iqbal, Foyzul Hassan, and Probir Roy. 2023. An empirical study of high performance computing (HPC) performance bugs. In MSR. IEEE, 194\u2013206."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3428282"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/318774.318944"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion.2019.00035"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453146"},{"key":"e_1_3_2_22_2","unstructured":"Fabian Beuke. 2025. GitHut 2.0: GitHub Language Statistics. Retrieved from https:\/\/madnight.github.io\/githut"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644889"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00096"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13198-021-01385-4"},{"key":"e_1_3_2_26_2","unstructured":"David M. Blei Andrew Y. Ng and Michael I. Jordan. 2003. Latent dirichlet allocation. J. Mach. Learn. Res. 3 (2003) 993\u20131022. https:\/\/jmlr.org\/papers\/v3\/blei03a.html"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","unstructured":"Clemens-Alexander Brust Tim Sonnekalb and Bernd Gruner. 2023. ROMEO: A binary vulnerability detection dataset for exploring juliet through the lens of assembly language. Comput. Secur. 128 (2023) 103165. DOI:10.1016\/J.COSE.2023.103165","DOI":"10.1016\/J.COSE.2023.103165"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528482"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST46399.2020.00059"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","unstructured":"Jie Cai Bin Li Tao Zhang Jiale Zhang and Xiaobing Sun. 2024. Fine-grained smart contract vulnerability detection by heterogeneous code feature learning and automated dataset construction. J. Syst. Softw. 209 (2024) 111919. DOI:10.1016\/J.JSS.2023.111919","DOI":"10.1016\/J.JSS.2023.111919"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","unstructured":"Marcos Lordello Chaim Kesina Baral Jeff Offutt Mario Concilio Neto and Roberto Paulo Andrioli de Araujo. 2023. On subsumption relationships in data flow testing. Softw. Test. Verification Reliab. 33 6 (2023). DOI:10.1002\/STVR.1843","DOI":"10.1002\/STVR.1843"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.3020502"},{"issue":"1","key":"e_1_3_2_33_2","first-page":"4:1\u20134:36","article-title":"A survey of compiler testing","volume":"53","author":"Chen Junjie","year":"2021","unstructured":"Junjie Chen, Jibesh Patra, Michael Pradel, Yingfei Xiong, Hongyu Zhang, Dan Hao, and Lu Zhang. 2021. A survey of compiler testing. ACM CSUR 53, 1 (2021), 4:1\u20134:36.","journal-title":"ACM CSUR"},{"issue":"2","key":"e_1_3_2_34_2","first-page":"327","article-title":"Defining smart contract defects on ethereum","volume":"48","author":"Chen Jiachi","year":"2022","unstructured":"Jiachi Chen, Xin Xia, David Lo, John C. Grundy, Xiapu Luo, and Ting Chen. 2022. Defining smart contract defects on ethereum. IEEE TSE 48, 2 (2022), 327\u2013345.","journal-title":"IEEE TSE"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3607199.3607242"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3569932"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00068"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/502034.502042"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/1555860.1555866"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528480"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639207"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-29056-5_34"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","unstructured":"L\u00e9uson M. P. da Silva Jordan Samhi and Foutse Khomh. 2025. LLMs and stack overflow discussions: Reliability impact and challenges. J. Syst. Softw. 230 (2025) 112541. DOI:10.1016\/J.JSS.2025.112541","DOI":"10.1016\/J.JSS.2025.112541"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9173-9"},{"key":"e_1_3_2_46_2","first-page":"15","volume-title":"ASE Workshops","author":"Das Teerath","year":"2024","unstructured":"Teerath Das, Adam Ali, and Tommi Mikkonen. 2024. Taxonomy of security-related issues in android apps: An empirical study. In ASE Workshops. ACM, 15\u201321."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641539"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00038"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.15"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3689799"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE52982.2021.00030"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.51"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338911"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416576"},{"issue":"2","key":"e_1_3_2_55_2","first-page":"159","article-title":"Test case prioritization: A family of empirical studies","volume":"28","author":"Elbaum Sebastian G.","year":"2002","unstructured":"Sebastian G. Elbaum, Alexey G. Malishevsky, and Gregg Rothermel. 2002. Test case prioritization: A family of empirical studies. IEEE TSE 28, 2 (2002), 159\u2013182.","journal-title":"IEEE TSE"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387501"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00128"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.879815"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","unstructured":"Rudolf Ferenc P\u00e9ter Gyimesi G\u00e1bor Gyimesi Zolt\u00e1n T\u00f3th and Tibor Gyim\u00f3thy. 2020. An automatically created novel bug dataset and its validation in bug prediction. J. Syst. Softw. 169 (2020) 110691. DOI:10.1016\/J.JSS.2020.110691","DOI":"10.1016\/J.JSS.2020.110691"},{"key":"e_1_3_2_60_2","first-page":"12","volume-title":"PROMISE","author":"Ferenc Rudolf","year":"2018","unstructured":"Rudolf Ferenc, Zolt\u00e1n T\u00f3th, Gergely Lad\u00e1nyi, Istv\u00e1n Siket, and Tibor Gyim\u00f3thy. 2018. A public unified bug dataset for Java. In PROMISE. ACM, 12\u201321."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115662"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/AsianHOST59942.2023.10409307"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196451"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3236030"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380397"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST53961.2022.00017"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196454"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2019.00012"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00024"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2670979.2670986"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","unstructured":"P\u00e9ter Gyimesi B\u00e9la Vancsics Andrea Stocco Davood Mazinanian \u00c1rp\u00e1d Besz\u00e9des Rudolf Ferenc and Ali Mesbah. 2021. BUGSJS: A benchmark and taxonomy of JavaScript bugs. Softw. Test. Verification Reliab. 31 4 (2021). DOI:10.1002\/STVR.1751","DOI":"10.1002\/STVR.1751"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238213"},{"key":"e_1_3_2_73_2","first-page":"392","volume-title":"ICSE","author":"Herzig Kim","year":"2013","unstructured":"Kim Herzig, Sascha Just, and Andreas Zeller. 2013. It\u2019s not a bug, it\u2019s a feature: How misclassification impacts bug prediction. In ICSE. IEEE Computer Society, 392\u2013401."},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","unstructured":"Thomas Hirsch and Birgit Hofer. 2022. A systematic literature review on benchmarks for evaluating debugging approaches. J. Syst. Softw. 192 (2022) 111423. DOI:10.1016\/J.JSS.2022.111423","DOI":"10.1016\/J.JSS.2022.111423"},{"key":"e_1_3_2_75_2","first-page":"220","volume-title":"ICSME","author":"Ho Sharon Chee Yin","year":"2023","unstructured":"Sharon Chee Yin Ho, Vahid Majdinasab, Mohayeminul Islam, Diego Elias Costa, Emad Shihab, Foutse Khomh, Sarah Nadi, and Muhammad Raza. 2023. An empirical study on bugs inside pytorch: A replication study. In ICSME. IEEE, 220\u2013231."},{"key":"e_1_3_2_76_2","first-page":"3","volume-title":"WorldCIST","author":"Holek Tomas","year":"2023","unstructured":"Tomas Holek, Miroslav Bures, and Tom\u00e1s Cern\u00fd. 2023. Review of open software bug datasets. In WorldCIST, Vol. 801. Springer, 3\u201312."},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3695988"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639230"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","unstructured":"Kai Huang Zhengzi Xu Su Yang Hongyu Sun Xuejun Li Zheng Yan and Yuqing Zhang. 2023. A survey on automated program repair techniques. CoRR abs\/2303.18184 (2023). arXiv:2303.18184. DOI:10.48550\/ARXIV.2303.18184","DOI":"10.48550\/ARXIV.2303.18184"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_81_2","first-page":"279","volume-title":"QRS","author":"Jabbar Emad","year":"2023","unstructured":"Emad Jabbar, Hadi Hemmati, and Robert Feldt. 2023. Investigating execution trace embedding for test case prioritization. In QRS. IEEE, 279\u2013290."},{"key":"e_1_3_2_82_2","volume-title":"HotPar","author":"Jalbert Nicholas","year":"2011","unstructured":"Nicholas Jalbert, Cristiano Pereira, Gilles Pokam, and Koushik Sen. 2011. RADBench: A concurrency bug benchmark suite. In HotPar. USENIX Association."},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3672452"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3177713"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00069"},{"key":"e_1_3_2_86_2","volume-title":"ICLR","author":"Jimenez Carlos E.","year":"2024","unstructured":"Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R. Narasimhan. 2024. SWE-bench: Can language models resolve real-world github issues?. In ICLR. OpenReview.net."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/2254064.2254075"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/2610384.2628055"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00066"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387491"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3527996"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE52982.2021.00036"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST57152.2023.00053"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528028"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00070"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2006.23"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/1321631.1321691"},{"key":"e_1_3_2_98_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2013.6624028"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/185403.185412"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2015.2454513"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1145\/3318162"},{"key":"e_1_3_2_102_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2021.3072577"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST60714.2024.00049"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/1181309.1181314"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579856.3595787"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510454.3516826"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1145\/3135932.3135941"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2993293"},{"key":"e_1_3_2_109_2","first-page":"219","volume-title":"ICICS","author":"Lin Guanjun","year":"2019","unstructured":"Guanjun Lin, Wei Xiao, Jun Zhang, and Yang Xiang. 2019. Deep learning-based vulnerable function detection: A benchmark. In ICICS, Vol. 11999. Springer, 219\u2013232."},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.87"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644870"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579637"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1145\/3672449"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-019-09715-8"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE55969.2022.00048"},{"key":"e_1_3_2_116_2","volume-title":"Bugs","author":"Lu Shan","year":"2005","unstructured":"Shan Lu, Zhenmin Li, Feng Qin, Lin Tan, Pin Zhou, and Yuanyuan Zhou. 2005. Bugbench: Benchmarks for evaluating bug detection tools. In Bugs."},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8667991"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.46"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR59073.2023.00085"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00051"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9470-4"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00169"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1145\/3127005.3127010"},{"issue":"1","key":"e_1_3_2_124_2","first-page":"17:1\u201317:24","article-title":"Automatic software Repair: A bibliography","volume":"51","author":"Monperrus Martin","year":"2018","unstructured":"Martin Monperrus. 2018. Automatic software Repair: A bibliography. ACM CSUR 51, 1 (2018), 17:1\u201317:24.","journal-title":"ACM CSUR"},{"issue":"3","key":"e_1_3_2_125_2","first-page":"62","article-title":"Bugs in machine learning-based systems: A faultload benchmark","volume":"28","author":"Morovati Mohammad Mehdi","year":"2023","unstructured":"Mohammad Mehdi Morovati, Amin Nikanjam, Foutse Khomh, and Zhen Ming (Jack) Jiang. 2023. Bugs in machine learning-based systems: A faultload benchmark. ESE 28, 3 (2023), 62.","journal-title":"ESE"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","unstructured":"Niels M\u00fcndler Mark Niklas M\u00fcller Jingxuan He and Martin T. Vechev. 2024. Code agents are state of the art software testers. CoRR abs\/2406.12952 (2024). arXiv:2406.12952. DOI:10.48550\/ARXIV.2406.12952","DOI":"10.48550\/ARXIV.2406.12952"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3384217.3384218"},{"issue":"1","key":"e_1_3_2_128_2","first-page":"111","article-title":"A comparative study of bug classification algorithms","volume":"24","author":"Nagwani Naresh Kumar","year":"2014","unstructured":"Naresh Kumar Nagwani and Shrish Verma. 2014. A comparative study of bug classification algorithms. IJSEKE 24, 1 (2014), 111\u2013138.","journal-title":"IJSEKE"},{"issue":"3","key":"e_1_3_2_129_2","first-page":"140","article-title":"Automatic test generation: A use case driven approach","volume":"32","author":"Nebut Cl\u00e9mentine","year":"2006","unstructured":"Cl\u00e9mentine Nebut, Franck Fleurey, Yves Le Traon, and Jean-Marc J\u00e9z\u00e9quel. 2006. Automatic test generation: A use case driven approach. IEEE TSE 32, 3 (2006), 140\u2013155.","journal-title":"IEEE TSE"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330225"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1145\/3461001.3473058"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644886"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3473122"},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1145\/3607538"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.78"},{"key":"e_1_3_2_136_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644912"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3652140"},{"key":"e_1_3_2_138_2","doi-asserted-by":"publisher","unstructured":"Jalaj Pachouly Swati Ahirrao and Ketan Kotecha. 2022. SDPTool: A tool for creating datasets and software defect predictions. SoftwareX 18 (2022) 101036. DOI:10.1016\/J.SOFTX.2022.101036","DOI":"10.1016\/J.SOFTX.2022.101036"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527330"},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","unstructured":"Matteo Paltenghi and Michael Pradel. 2024. A survey on testing and analysis of quantum software. CoRR abs\/2410.00650 (2024). arXiv:2410.00650. DOI:10.48550\/ARXIV.2410.00650","DOI":"10.48550\/ARXIV.2410.00650"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion52605.2021.00113"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1109\/PRDC55274.2022.00029"},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2895640"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00064"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2014.27"},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644878"},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00066"},{"key":"e_1_3_2_148_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389267"},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER60148.2024.00024"},{"key":"e_1_3_2_150_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00087"},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678776"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASEW60602.2023.00015"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691620.3695345"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.1145\/3639478.3640023"},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1145\/3196398.3196473"},{"key":"e_1_3_2_156_2","doi-asserted-by":"publisher","unstructured":"Ana Bel\u00e9n S\u00e1nchez Pedro Delgado-P\u00e9rez Inmaculada Medina-Bulo and Sergio Segura. 2020. TANDEM: A taxonomy and a dataset of real-world performance bugs. IEEE Access 8 (2020) 107214\u2013107228. DOI:10.1109\/ACCESS.2020.3000928","DOI":"10.1109\/ACCESS.2020.3000928"},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1109\/FIE61694.2024.10893015"},{"key":"e_1_3_2_158_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSTW55395.2022.00059"},{"key":"e_1_3_2_159_2","first-page":"103","volume-title":"SCAM","author":"Sarhan Qusay Idrees","year":"2021","unstructured":"Qusay Idrees Sarhan, B\u00e9la Vancsics, and \u00c1rp\u00e1d Besz\u00e9des. 2021. Method calls frequency-based tie-breaking strategy for software fault localization. In SCAM. IEEE, 103\u2013113."},{"key":"e_1_3_2_160_2","unstructured":"Michael Schlichtig Steffen Sassalla Krishna Narasimhan and Eric Bodden. 2023. Introducing FUM: A framework for API usage constraint and misuse classification. In Software Engineering 2023 Fachtagung des GI-Fachbereichs Softwaretechnik 20.-24. FebruarY 2023 Paderborn Gregor Engels Regina Hebig and Matthias Tichy (Eds.). Gesellschaft f\u00fcr Informatik e.V. P\u2013332 (2023) 105\u2013106. https:\/\/dl.gi.de\/handle\/20.500.12116\/40106"},{"key":"e_1_3_2_161_2","unstructured":"Marija Selakovic and Michael Pradel. 2017. Performance issues and optimizations in javaScript: An empirical study. In Software Engineering 2017 Fachtagung des GI-Fachbereichs Softwaretechnik 21.-24. February 2017 (LNI Vol. P-267) Jan J\u00fcrjens and Kurt Schneider (Eds.). Hannover Deutschland. GI 63. https:\/\/dl.gi.de\/handle\/20.500.12116\/1272"},{"key":"e_1_3_2_162_2","doi-asserted-by":"publisher","DOI":"10.5220\/0012060400003555"},{"key":"e_1_3_2_163_2","unstructured":"Kostya Serebryany. 2017. OSS-Fuzz - Google\u2019s continuous fuzzing service for open source software. USENIX Association Vancouver BC."},{"key":"e_1_3_2_164_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468591"},{"key":"e_1_3_2_165_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644884"},{"key":"e_1_3_2_166_2","doi-asserted-by":"publisher","DOI":"10.1145\/1082983.1083147"},{"key":"e_1_3_2_167_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330203"},{"key":"e_1_3_2_168_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533767.3534224"},{"key":"e_1_3_2_169_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00201"},{"key":"e_1_3_2_170_2","doi-asserted-by":"publisher","DOI":"10.1109\/FTCS.1991.146625"},{"issue":"3","key":"e_1_3_2_171_2","first-page":"394","article-title":"Identifying metamorphic relations: A data mutation directed approach","volume":"54","author":"Sun Chang-Ai","year":"2024","unstructured":"Chang-Ai Sun, Hui Jin, Siyi Wu, An Fu, Zuoyi Wang, and Wing Kwong Chan. 2024. Identifying metamorphic relations: A data mutation directed approach. SPE 54, 3 (2024), 394\u2013418.","journal-title":"SPE"},{"issue":"1","key":"e_1_3_2_172_2","first-page":"10","article-title":"Silent bugs in deep learning frameworks: An empirical study of Keras and TensorFlow","volume":"29","author":"Tambon Florian","year":"2024","unstructured":"Florian Tambon, Amin Nikanjam, Le An, Foutse Khomh, and Giuliano Antoniol. 2024. Silent bugs in deep learning frameworks: An empirical study of Keras and TensorFlow. ESE 29, 1 (2024), 10.","journal-title":"ESE"},{"key":"e_1_3_2_173_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180243"},{"key":"e_1_3_2_174_2","first-page":"180","volume-title":"ICSE Companion","author":"Tan Shin Hwei","year":"2017","unstructured":"Shin Hwei Tan, Jooyong Yi, Yulis, Sergey Mechtaev, and Abhik Roychoudhury. 2017. Codeflaws: A programming competition benchmark for evaluating automated program repair tools. In ICSE Companion. IEEE Computer Society, 180\u2013182."},{"key":"e_1_3_2_175_2","doi-asserted-by":"publisher","DOI":"10.1109\/PST62714.2024.10788068"},{"key":"e_1_3_2_176_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576039"},{"key":"e_1_3_2_177_2","doi-asserted-by":"publisher","unstructured":"Haoye Tian Weiqi Lu Tsz On Li Xunzhu Tang Shing-Chi Cheung Jacques Klein and Tegawend\u00e9 F. Bissyand\u00e9. 2023. Is ChatGPT the ultimate programming assistant - How far is it? CoRR abs\/2304.11938 (2023). arXiv:2304.11938. DOI:10.48550\/ARXIV.2304.11938","DOI":"10.48550\/ARXIV.2304.11938"},{"key":"e_1_3_2_178_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.247"},{"key":"e_1_3_2_179_2","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337269"},{"issue":"3","key":"e_1_3_2_180_2","first-page":"57","article-title":"ROBUST: 221 bugs in the robot operating system","volume":"29","author":"Timperley Christopher Steven","year":"2024","unstructured":"Christopher Steven Timperley, Gijs van der Hoorn, Andr\u00e9 Santos, Harshavardhan Deshpande, and Andrzej Wasowski. 2024. ROBUST: 221 bugs in the robot operating system. ESE 29, 3 (2024), 57.","journal-title":"ESE"},{"key":"e_1_3_2_181_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00048"},{"key":"e_1_3_2_182_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340544"},{"key":"e_1_3_2_183_2","first-page":"80","volume-title":"PROMISE","author":"Vieira Renan Gomes","year":"2019","unstructured":"Renan Gomes Vieira, Ant\u00f4nio da Silva, Lincoln S. Rocha, and Jo\u00e3o Paulo Pordeus Gomes. 2019. From reports to bug-fix commits: A 10 years dataset of bug-fixing activity from 55 Apache\u2019s open source projects. In PROMISE. ACM, 80\u201389."},{"key":"e_1_3_2_184_2","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468559"},{"key":"e_1_3_2_185_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3559561"},{"key":"e_1_3_2_186_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2949568"},{"key":"e_1_3_2_187_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2017.8115663"},{"key":"e_1_3_2_188_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387464"},{"issue":"1","key":"e_1_3_2_189_2","first-page":"21","article-title":"Demystifying regular expression bugs","volume":"27","author":"Wang Peipei","year":"2022","unstructured":"Peipei Wang, Chris Brown, Jamie A. Jennings, and Kathryn T. Stolee. 2022. Demystifying regular expression bugs. ESE 27, 1 (2022), 21.","journal-title":"ESE"},{"key":"e_1_3_2_190_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2874470"},{"key":"e_1_3_2_191_2","doi-asserted-by":"publisher","DOI":"10.1145\/3639478.3647634"},{"key":"e_1_3_2_192_2","doi-asserted-by":"publisher","unstructured":"Yilin Wang Xiangping Chen Yuan Huang Hao-Nan Zhu Jing Bian and Zibin Zheng. 2023. An empirical study on real bug fixes from solidity smart contract projects. J. Syst. Softw. 204 (2023) 111787. DOI:10.1016\/J.JSS.2023.111787","DOI":"10.1016\/J.JSS.2023.111787"},{"key":"e_1_3_2_193_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00041"},{"key":"e_1_3_2_194_2","doi-asserted-by":"publisher","unstructured":"Ziyuan Wang Dexin Bu Nannan Wang Sijie Yu Shanyi Gou and Aiyue Sun. 2023. An empirical study on bugs in JavaScript engines. Inf. Softw. Technol. 155 (2023) 107105. DOI:10.1016\/J.INFSOF.2022.107105","DOI":"10.1016\/J.INFSOF.2022.107105"},{"key":"e_1_3_2_195_2","first-page":"170","volume-title":"EASE","author":"Waseem Muhammad","year":"2024","unstructured":"Muhammad Waseem, Teerath Das, Aakash Ahmad, Peng Liang, and Tommi Mikkonen. 2024. Issues and their causes in webassembly applications: An empirical study. In EASE. ACM, 170\u2013180."},{"key":"e_1_3_2_196_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2948158"},{"key":"e_1_3_2_197_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00082"},{"key":"e_1_3_2_198_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417943"},{"key":"e_1_3_2_199_2","doi-asserted-by":"publisher","DOI":"10.1145\/1390817.1390826"},{"key":"e_1_3_2_200_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2521368"},{"key":"e_1_3_2_201_2","doi-asserted-by":"publisher","unstructured":"XiaoxueWu Wei Zheng Xiang Chen FangWang and Dejun Mu. 2020. CVE-assisted large-scale security bug report dataset construction method. J. Syst. Softw. 160 (2020). DOI:10.1016\/J.JSS.2019.110456","DOI":"10.1016\/J.JSS.2019.110456"},{"key":"e_1_3_2_202_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598135"},{"key":"e_1_3_2_203_2","doi-asserted-by":"publisher","DOI":"10.1145\/3663529.3663815"},{"key":"e_1_3_2_204_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680323"},{"key":"e_1_3_2_205_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597926.3598138"},{"key":"e_1_3_2_206_2","first-page":"3655","volume-title":"USENIX Security","author":"Xu Jianhao","year":"2023","unstructured":"Jianhao Xu, Kangjie Lu, Zhengjie Du, Zhu Ding, Linke Li, Qiushi Wu, Mathias Payer, and Bing Mao. 2023. Silent bugs matter: A study of compiler-introduced security bugs. In USENIX Security. USENIX Association, 3655\u20133672."},{"key":"e_1_3_2_207_2","first-page":"100","volume-title":"DSA","author":"Xu Jiaxi","year":"2019","unstructured":"Jiaxi Xu, Liang Yan, Fei Wang, and Jun Ai. 2019. A GitHub-based data collection method for software defect prediction. In DSA. IEEE, 100\u2013108."},{"key":"e_1_3_2_208_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680396"},{"key":"e_1_3_2_209_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00127"},{"key":"e_1_3_2_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643788.3648013"},{"key":"e_1_3_2_211_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC54236.2022.00142"},{"key":"e_1_3_2_212_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-91265-9_12"},{"key":"e_1_3_2_213_2","volume-title":"NeurIPS","author":"Yang John","year":"2024","unstructured":"John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: Agent-computer interfaces enable automated software engineering. In NeurIPS."},{"issue":"10","key":"e_1_3_2_214_2","first-page":"206:1\u2013206:73","article-title":"A survey on deep learning for software engineering","volume":"54","author":"Yang Yanming","year":"2022","unstructured":"Yanming Yang, Xin Xia, David Lo, and John C. Grundy. 2022. A survey on deep learning for software engineering. ACM CSUR 54, 10s (2022), 206:1\u2013206:73.","journal-title":"ACM CSUR"},{"key":"e_1_3_2_215_2","first-page":"1899","volume-title":"ASE","author":"Ye He","year":"2023","unstructured":"He Ye, Zimin Chen, and Claire Le Goues. 2023. PreciseBugCollector: Extensible, executable and precise bug-fix collection: Solution for challenge 8: Automating precise data collection for code snippets with bugs, fixes, locations, and types. In ASE. IEEE, 1899\u20131910."},{"key":"e_1_3_2_216_2","first-page":"499","volume-title":"DSC","author":"Ye Wenhao","year":"2023","unstructured":"Wenhao Ye, Jun Xia, Shuo Feng, Xiangyu Zhong, Shuai Yuan, and Zhitao Guan. 2023. FixGPT: A novel three-tier deep learning model for automated program repair. In DSC. IEEE, 499\u2013505."},{"key":"e_1_3_2_217_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680366"},{"key":"e_1_3_2_218_2","first-page":"296","volume-title":"QRS","author":"Yu Sijie","year":"2024","unstructured":"Sijie Yu and Ziyuan Wang. 2024. An empirical study on bugs in rust programming language. In QRS. IEEE, 296\u2013305."},{"key":"e_1_3_2_219_2","doi-asserted-by":"publisher","DOI":"10.1109\/CGO51591.2021.9370317"},{"key":"e_1_3_2_220_2","doi-asserted-by":"publisher","DOI":"10.5555\/2664446.2664477"},{"key":"e_1_3_2_221_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510073"},{"key":"e_1_3_2_222_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338917"},{"key":"e_1_3_2_223_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-014-5241-2"},{"key":"e_1_3_2_224_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME46990.2020.00023"},{"key":"e_1_3_2_225_2","doi-asserted-by":"publisher","DOI":"10.1145\/3631974"},{"key":"e_1_3_2_226_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3354969"},{"key":"e_1_3_2_227_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00104"},{"key":"e_1_3_2_228_2","doi-asserted-by":"publisher","DOI":"10.1145\/3624743"},{"key":"e_1_3_2_229_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"},{"key":"e_1_3_2_230_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680384"},{"key":"e_1_3_2_231_2","first-page":"3699","volume-title":"USENIX Security","author":"Zhang Zenong","year":"2022","unstructured":"Zenong Zhang, Zach Patterson, Michael Hicks, and Shiyi Wei. 2022. FIXREVERTER: A realistic bug injection methodology for benchmarking fuzz testing. In USENIX Security. USENIX Association, 3699\u20133715."},{"issue":"2","key":"e_1_3_2_232_2","first-page":"182202","article-title":"ContextAug: Model-domain failing test augmentation with contextual information","volume":"18","author":"Zhang Zhuo","year":"2024","unstructured":"Zhuo Zhang, Jianxin Xue, Deheng Yang, and Xiaoguang Mao. 2024. ContextAug: Model-domain failing test augmentation with contextual information. FCS 18, 2 (2024), 182202.","journal-title":"FCS"},{"key":"e_1_3_2_233_2","doi-asserted-by":"publisher","unstructured":"Pengzhan Zhao Zhongtao Miao Shuhan Lan and Jianjun Zhao. 2023. Bugs4Q: A benchmark of existing bugs to enable controlled testing and debugging studies for quantum programs. J. Syst. Softw. 205 (2023) 111805. DOI:10.1016\/J.JSS.2023.111805","DOI":"10.1016\/J.JSS.2023.111805"},{"key":"e_1_3_2_234_2","doi-asserted-by":"publisher","DOI":"10.1109\/QSW59989.2023.00018"},{"key":"e_1_3_2_235_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3167628"},{"key":"e_1_3_2_236_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00020"},{"key":"e_1_3_2_237_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigDataCongress.2017.85"},{"key":"e_1_3_2_238_2","doi-asserted-by":"publisher","DOI":"10.1145\/3688834"},{"key":"e_1_3_2_239_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00176"},{"key":"e_1_3_2_240_2","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337226"},{"key":"e_1_3_2_241_2","doi-asserted-by":"publisher","unstructured":"Hao-Nan Zhu Robert M. Furth Michael Pradel and Cindy Rubio-Gonz\u00e1lez. 2025. From Bugs to Benchmarks: A Comprehensive Survey of Software Defect Datasets. DOI:10.5281\/zenodo.17402612","DOI":"10.5281\/zenodo.17402612"},{"key":"e_1_3_2_242_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00015"},{"key":"e_1_3_2_243_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00195"},{"key":"e_1_3_2_244_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.63"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3797033","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,10]],"date-time":"2026-04-10T20:44:14Z","timestamp":1775853854000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3797033"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,9]]},"references-count":243,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3797033"],"URL":"https:\/\/doi.org\/10.1145\/3797033","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,9]]},"assertion":[{"value":"2025-04-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}