{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:24:07Z","timestamp":1778811847451,"version":"3.51.4"},"reference-count":124,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSERC Alliance","award":["ALLRP\/556396-2020"],"award-info":[{"award-number":["ALLRP\/556396-2020"]}]},{"name":"Alberta Innovates CASBE Program","award":["#202102242"],"award-info":[{"award-number":["#202102242"]}]},{"name":"Eyes High Postdoctoral Match-Funding Program"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Fixing software bugs can be colossally expensive, especially if they are discovered in the later phases of the software development life cycle. As such, bug prediction has been a classic problem for the research community. As of now, the Google Scholar site generates \u223c113,000 hits if searched with the \u201cbug prediction\u201d phrase. Despite this staggering effort by the research community, bug prediction research is criticized for not being decisively adopted in practice. A significant problem of the existing research is the granularity level (i.e., class\/file level) at which bug prediction is historically studied. Practitioners find it difficult and time-consuming to locate bugs at the class\/file level granularity. Consequently, method-level bug prediction has become popular in the past decade. We ask,<jats:italic>are these method-level bug prediction models ready for industry use?<\/jats:italic>Unfortunately, the answer is<jats:italic>no<\/jats:italic>. The reported high accuracies of these models dwindle significantly if we evaluate them in different realistic time-sensitive contexts. It may seem hopeless at first, but, encouragingly, we show that future method-level bug prediction can be improved significantly. In general, we show how to reliably evaluate future method-level bug prediction models and how to improve them by focusing on four different improvement avenues: building noise-free bug data, addressing concept drift, selecting similar training projects, and developing a mixture of models. Our findings are based on three publicly available method-level bug datasets and a newly built bug dataset of 774,051 Java methods originating from 49 open-source software projects.<\/jats:p>","DOI":"10.1145\/3640331","type":"journal-article","created":{"date-parts":[[2024,1,13]],"date-time":"2024-01-13T13:38:31Z","timestamp":1705153111000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Method-level Bug Prediction: Problems and Promises"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2301-6104","authenticated-orcid":false,"given":"Shaiful","family":"Chowdhury","sequence":"first","affiliation":[{"name":"University of Manitoba, Winnipeg, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1376-095X","authenticated-orcid":false,"given":"Gias","family":"Uddin","sequence":"additional","affiliation":[{"name":"York University, Toronto, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0204-9812","authenticated-orcid":false,"given":"Hadi","family":"Hemmati","sequence":"additional","affiliation":[{"name":"York University, Toronto, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4213-494X","authenticated-orcid":false,"given":"Reid","family":"Holmes","sequence":"additional","affiliation":[{"name":"University of British Columbia, Vancouver, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,4,18]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Investigating the Impact of Methodological Choices on Source Code Maintenance Analyses","author":"Ahmad Syed Ishtiaque","year":"2021","unstructured":"Syed Ishtiaque Ahmad. 2021. Investigating the Impact of Methodological Choices on Source Code Maintenance Analyses. Master\u2019s Thesis. University of British Columbia."},{"key":"e_1_3_2_3_2","first-page":"1","volume-title":"Proceedings of the 9th IEEE-GCC Conference and Exhibition","author":"Alfadel M.","year":"2017","unstructured":"M. Alfadel, A. Kobilica, and J. Hassine. 2017. Evaluation of Halstead and cyclomatic complexity metrics in measuring defect density. In Proceedings of the 9th IEEE-GCC Conference and Exhibition. 1\u20139."},{"key":"e_1_3_2_4_2","first-page":"716","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution","author":"Alsolai H.","year":"2018","unstructured":"H. Alsolai, M. Roper, and D. Nassar. 2018. Predicting software maintainability in object-oriented systems using ensemble techniques. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution. 716\u2013721."},{"key":"e_1_3_2_5_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance","author":"Alves T. L.","year":"2010","unstructured":"T. L. Alves, C. Ypma, and J. Visser. 2010. Deriving metric thresholds from benchmark data. In Proceedings of the IEEE International Conference on Software Maintenance. 1\u201310."},{"key":"e_1_3_2_6_2","first-page":"44","volume-title":"Proceedings of the IEEE\/ACM 6th International Workshop on Emerging Trends in Software Metrics","author":"Fontana Francesca Arcelli","year":"2015","unstructured":"Francesca Arcelli Fontana, Vincenzo Ferme, Marco Zanoni, and Aiko Yamashita. 2015. Automatic metric thresholds derivation for code smell detection. In Proceedings of the IEEE\/ACM 6th International Workshop on Emerging Trends in Software Metrics. 44\u201353."},{"key":"e_1_3_2_7_2","first-page":"649","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME\u201921)","author":"Asano Takuya","year":"2021","unstructured":"Takuya Asano, Masateru Tsunoda, Koji Toda, Amjed Tahir, Kwabena Ebo Bennin, Keitaro Nakasai, Akito Monden, and Kenichi Matsumoto. 2021. Using bandit algorithms for project selection in cross-project defect prediction. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME\u201921). 649\u2013653."},{"key":"e_1_3_2_8_2","first-page":"363","volume-title":"Proceedings of the International Conference on Computational Science and Its Applications","author":"Balogun Abdullateef O.","year":"2022","unstructured":"Abdullateef O. Balogun, Babajide J. Odejide, Amos O. Bajeh, Zubair O. Alanamu, Fatima E. Usman-Hamza, Hammid O. Adeleke, Modinat A. Mabayoje, and Shakirat R. Yusuff. 2022. Empirical analysis of data sampling-based ensemble methods in software defect prediction. In Proceedings of the International Conference on Computational Science and Its Applications. 363\u2013379."},{"issue":"6","key":"e_1_3_2_9_2","article-title":"On the time-based conclusion stability of cross-project defect prediction models","volume":"25","author":"Bangash Abdul Ali","year":"2020","unstructured":"Abdul Ali Bangash, Hareem Sahar, Abram Hindle, and Karim Ali. 2020. On the time-based conclusion stability of cross-project defect prediction models. Empir. Softw. Eng. 25, 6 (2020).","journal-title":"Empir. Softw. Eng."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.544352"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2367027"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/2020390.2020392"},{"key":"e_1_3_2_13_2","first-page":"53","volume-title":"Proceedings of the IEEE 20th International Conference on Software Quality, Reliability and Security (QRS\u201920)","author":"Bennin Kwabena E.","year":"2020","unstructured":"Kwabena E. Bennin, Nauman bin Ali, J\u00fcrgen B\u00f6rstler, and Xiao Yu. 2020. Revisiting the impact of concept drift on just-in-time quality assurance. In Proceedings of the IEEE 20th International Conference on Software Quality, Reliability and Security (QRS\u201920). 53\u201359."},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Christian Bird Adrian Bachmann Eirik Aune John Duffy Abraham Bernstein Vladimir Filkov and Premkumar Devanbu. 2009. Fair and balanced? Bias in bug-fix datasets. 7th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering.121\u2013130.","DOI":"10.1145\/1595696.1595716"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.70"},{"key":"e_1_3_2_16_2","unstructured":"Celerity. The True Cost of a Software Bug: Part One. Retrieved from https:\/\/www.celerity.com\/insights\/the-true-cost-of-a-software-bug. (n.d.)."},{"key":"e_1_3_2_17_2","first-page":"15","volume-title":"Proceedings of the International Conference on Software Maintenance","author":"Chapin Ned","year":"2000","unstructured":"Ned Chapin. 2000. Do we know what preventive maintenance is? In Proceedings of the International Conference on Software Maintenance. 15\u201317."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-021-10018-0"},{"key":"e_1_3_2_19_2","first-page":"1580","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy (SP\u201920)","author":"Chen Yaohui","year":"2020","unstructured":"Yaohui Chen, Peng Li, Jun Xu, Shengjian Guo, Rundong Zhou, Yulong Zhang, Tao Wei, and Long Lu. 2020. Savior: Towards bug-driven hybrid testing. In Proceedings of the IEEE Symposium on Security and Privacy (SP\u201920). 1580\u20131596."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.295895"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9640-7"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9545-x"},{"issue":"6","key":"e_1_3_2_23_2","article-title":"Revisiting the debate: Are code metrics useful for measuring maintenance effort?","volume":"27","author":"Chowdhury Shaiful","year":"2022","unstructured":"Shaiful Chowdhury, Reid Holmes, Andy Zaidman, and Rick Kazman. 2022. Revisiting the debate: Are code metrics useful for measuring maintenance effort? Empir. Softw. Eng. 27, 6 (2022), 31.","journal-title":"Empir. Softw. Eng."},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1145\/3524842.3527975","volume-title":"Proceedings of the International Conference on Mining Software Repositories (MSR\u201922)","author":"Chowdhury Shaiful","year":"2022","unstructured":"Shaiful Chowdhury, Gias Uddin, and Reid Holmes. 2022. An empirical study on maintainable method size in Java. In Proceedings of the International Conference on Mining Software Repositories (MSR\u201922). 252\u2013264."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1979.234165"},{"key":"e_1_3_2_26_2","first-page":"31","volume-title":"Proceedings of the 7th IEEE Working Conference on Mining Software Repositories (MSR\u201910)","author":"D\u2019Ambros Marco","year":"2010","unstructured":"Marco D\u2019Ambros, Michele Lanza, and Romain Robbes. 2010. An extensive comparison of bug prediction approaches. In Proceedings of the 7th IEEE Working Conference on Mining Software Repositories (MSR\u201910). IEEE, 31\u201341."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.1619"},{"issue":"1","key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/TSE.2017.2659747","article-title":"A developer centered bug prediction model","volume":"44","author":"Nucci Dario Di","year":"2018","unstructured":"Dario Di Nucci, Fabio Palomba, Giuseppe De Rosa, Gabriele Bavota, Rocco Oliveto, and Andrea De Lucia. 2018. A developer centered bug prediction model. IEEE Trans. Softw. Eng. 44, 1 (2018), 5\u201324.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_29_2","first-page":"341","volume-title":"Proceedings of the IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201915)","author":"Dias Mart\u00edn","year":"2015","unstructured":"Mart\u00edn Dias, Alberto Bacchelli, Georgios Gousios, Damien Cassou, and St\u00e9phane Ducasse. 2015. Untangling fine-grained code changes. In Proceedings of the IEEE 22nd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201915). 341\u2013350."},{"key":"e_1_3_2_30_2","first-page":"51","volume-title":"Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories","author":"Ekanayake Jayalath","year":"2009","unstructured":"Jayalath Ekanayake, Jonas Tappolet, Harald C. Gall, and Abraham Bernstein. 2009. Tracking concept drift of software projects using defect prediction quality. In Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories. 51\u201360."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.935855"},{"key":"e_1_3_2_32_2","first-page":"2652","volume-title":"Proceedings of the International Joint Conference on Neural Networks (IJCNN\u201917)","author":"Fard Farzaneh S.","year":"2017","unstructured":"Farzaneh S. Fard, Paul Hollensen, Stuart Mcilory, and Thomas Trappenberg. 2017. Impact of biased mislabeling on learning with deep networks. In Proceedings of the International Joint Conference on Neural Networks (IJCNN\u201917). 2652\u20132657."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.815326"},{"key":"e_1_3_2_34_2","article-title":"An automatically created novel bug dataset and its validation in bug prediction","volume":"169","author":"Ferenc Rudolf","year":"2020","unstructured":"Rudolf Ferenc, P\u00e9ter Gyimesi, G\u00e1bor Gyimesi, Zolt\u00e1n T\u00f3th, and Tibor Gyim\u00f3thy. 2020. An automatically created novel bug dataset and its validation in bug prediction. J. Syst. Softw. 169 (2020).","journal-title":"J. Syst. Softw."},{"key":"e_1_3_2_35_2","unstructured":"Christine Fisher. 2020. Boeing Found Another Software Bug on the 737 Max. Retrieved from https:\/\/www.engadget.com\/2020-02-06-boeing-737-max-software-bug.html"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2372251.2372285"},{"issue":"1","key":"e_1_3_2_37_2","first-page":"2:1\u201325","article-title":"When do software complexity metrics mean nothing? \u2013 When examined out of context","volume":"15","author":"Gil Yossi","year":"2016","unstructured":"Yossi Gil and Gal Lalouche. 2016. When do software complexity metrics mean nothing? \u2013 When examined out of context. J. Object Technol. 15, 1 (Feb.2016), 2:1\u201325.","journal-title":"J. Object Technol."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9513-5"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.859533"},{"key":"e_1_3_2_40_2","first-page":"221","volume-title":"Proceedings of the International Conference on Software Engineering (ICSE\u201921)","author":"Grund Felix","year":"2021","unstructured":"Felix Grund, Shaiful Chowdhury, Nick C. Bradley, Braxton Hall, and Reid Holmes. 2021. CodeShovel: A reusable and available tool for extracting source code histories. In Proceedings of the International Conference on Software Engineering (ICSE\u201921). 221\u2013222."},{"key":"e_1_3_2_41_2","first-page":"1510","volume-title":"Proceedings of the International Conference on Software Engineering (ICSE\u201921)","author":"Grund Felix","year":"2021","unstructured":"Felix Grund, Shaiful Chowdhury, Nick C. Bradley, Braxton Hall, and Reid Holmes. 2021. CodeShovel: Constructing method-level source code histories. In Proceedings of the International Conference on Software Engineering (ICSE\u201921). 1510\u20131522."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2019.11.146"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.112"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1145\/2024445.2024463","volume-title":"Proceedings of the International Workshop on Principles of Software Evolution and ERCIM Workshop on Software Evolution","author":"Hata Hideaki","year":"2011","unstructured":"Hideaki Hata, Osamu Mizuno, and Tohru Kikuno. 2011. Historage: Fine-grained version control system for Java. In Proceedings of the International Workshop on Principles of Software Evolution and ERCIM Workshop on Software Evolution. 96\u2013100."},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"Hideaki Hata Osamu Mizuno and Tohru Kikuno. 2012. Bug prediction based on fine-grained module histories. 200\u2013210.","DOI":"10.1109\/ICSE.2012.6227193"},{"key":"e_1_3_2_46_2","first-page":"153","volume-title":"Proceedings of the IEEE 23rd International Symposium on Software Reliability Engineering Workshops","author":"Hays Mark","year":"2012","unstructured":"Mark Hays and Jane Hayes. 2012. The effect of testability on fault proneness: A case study of the Apache HTTP server. In Proceedings of the IEEE 23rd International Symposium on Software Reliability Engineering Workshops. 153\u2013158."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.11.006"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/QUATIC.2007.7"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9468-y"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9376-6"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.5555\/2487085.2487113"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110571"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2770124"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"e_1_3_2_55_2","first-page":"432","volume-title":"Proceedings of the IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR\u201921)","author":"Jiarpakdee Jirayus","year":"2021","unstructured":"Jirayus Jiarpakdee, Chakkrit Kla Tantithamthavorn, and John Grundy. 2021. Practitioners\u2019 perceptions of the goals and visual explanations of defect prediction models. In Proceedings of the IEEE\/ACM 18th International Conference on Mining Software Repositories (MSR\u201921). 432\u2013443."},{"key":"e_1_3_2_56_2","first-page":"53","volume-title":"Proceedings of the IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC\u201919)","volume":"1","author":"Kabir Md Alamgir","year":"2019","unstructured":"Md Alamgir Kabir, Jacky W. Keung, Kwabena E. Bennin, and Miao Zhang. 2019. Assessing the significant impact of concept drift in software defect prediction. In Proceedings of the IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC\u201919), Vol. 1. 53\u201358."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1987.233164"},{"key":"e_1_3_2_58_2","first-page":"33","volume-title":"Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201916)","author":"Kamei Yasutaka","year":"2016","unstructured":"Yasutaka Kamei and Emad Shihab. 2016. Defect prediction: Accomplishments and future challenges. In Proceedings of the IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER\u201916). 33\u201345."},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2006.23"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2597008.2597798"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-016-9437-5"},{"key":"e_1_3_2_62_2","first-page":"221","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution","author":"Landman D.","year":"2014","unstructured":"D. Landman, A. Serebrenik, and J. Vinju. 2014. Empirical analysis of the relationship between CC and SLOC in a large corpus of Java methods. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution. 221\u2013230."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2016.156"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2014.07.005"},{"key":"e_1_3_2_65_2","first-page":"707","volume-title":"Soviet Physics Doklady","author":"Levenshtein Vladimir I.","year":"1966","unstructured":"Vladimir I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady, Vol. 10. 707\u2013710."},{"key":"e_1_3_2_66_2","unstructured":"C. Lewis and R. Ou. 2011. Bug Prediction at Google. Retrieved from http:\/\/google-engtools.blogspot.com\/2011\/12\/bug-prediction-at-google.html"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-sen.2017.0148"},{"key":"e_1_3_2_68_2","first-page":"393","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME\u201918)","author":"Liu Xiaoyu","year":"2018","unstructured":"Xiaoyu Liu, LiGuo Huang, Chuanyi Li, and Vincent Ng. 2018. Linking source code to untangled change intents. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME\u201918). 393\u2013403."},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2011.09.007"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.11144\/Javeriana.upsy10-2.cdcp"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1976.233837"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","DOI":"10.1002\/9780470479216.corpsy0524","article-title":"Mann-Whitney U test","author":"McKnight Patrick E.","year":"2010","unstructured":"Patrick E. McKnight and Julius Najab. 2010. Mann-Whitney U test. Corsini Encyc. Psychol. 1, 1 (2010).","journal-title":"Corsini Encyc. Psychol."},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.256941"},{"key":"e_1_3_2_74_2","unstructured":"Microsoft. 2022. Code Metrics Maintainability Index. Retrieved from https:\/\/docs.microsoft.com\/en-us\/visualstudio\/code-quality\/code-metrics-maintainability-index-range-and-meaning?view=vs-2022"},{"key":"e_1_3_2_75_2","article-title":"An exploratory study of bug prediction at the method level","volume":"144","author":"Mo Ran","year":"2022","unstructured":"Ran Mo, Shaozhi Wei, Qiong Feng, and Zengyang Li. 2022. An exploratory study of bug prediction at the method level. Inf. Softw. Technol. 144, C (Apr.2022).","journal-title":"Inf. Softw. Technol."},{"key":"e_1_3_2_76_2","unstructured":"Manishankar Mondal Banani Roy Chanchal K. Roy and Kevin A. Schneider. 2019. Investigating the relationship between evolutionary coupling and software bug-proneness. In Proceedings of the 29th Annual International Conference on Computer Science and Software Engineering. 173\u2013182."},{"key":"e_1_3_2_77_2","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1145\/1414004.1414063","volume-title":"Proceedings of the 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM\u201908)","author":"Moser Raimund","year":"2008","unstructured":"Raimund Moser, Witold Pedrycz, and Giancarlo Succi. 2008. Analysis of the reliability of a subset of change metrics for defect prediction. In Proceedings of the 2nd ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM\u201908). 309\u2013311."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062514"},{"key":"e_1_3_2_79_2","doi-asserted-by":"crossref","unstructured":"Nachiappan Nagappan Thomas Ball and Andreas Zeller. 2006. Mining metrics to predict component failures. In Proceedings of the 28th International Conference on Software Engineering. 452\u2013461.","DOI":"10.1145\/1134285.1134349"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1080\/00107510500052444"},{"key":"e_1_3_2_81_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance","author":"Olbrich Steffen M.","year":"2010","unstructured":"Steffen M. Olbrich, Daniela S. Cruzes, and Dag I. K. Sj\u00f8berg. 2010. Are all code smells harmful? A study of God Classes and Brain Classes in the evolution of three open source systems. In Proceedings of the IEEE International Conference on Software Maintenance. 1\u201310."},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.1992.242525"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2017.38"},{"key":"e_1_3_2_84_2","article-title":"On the performance of method-level bug prediction: A negative result","volume":"161","author":"Pascarella Luca","year":"2020","unstructured":"Luca Pascarella, Fabio Palomba, and Alberto Bacchelli. 2020. On the performance of method-level bug prediction: A negative result. J. Syst. Softw. 161 (2020).","journal-title":"J. Syst. Softw."},{"key":"e_1_3_2_85_2","first-page":"296","volume-title":"Proceedings of the 28th International Conference on Program Comprehension","author":"Pecorelli Fabiano","unstructured":"Fabiano Pecorelli, Gemma Catolino, Filomena Ferrucci, Andrea De Lucia, and Fabio Palomba. Testing of mobile applications in the wild: A large-scale empirical study on Android apps. In Proceedings of the 28th International Conference on Program Comprehension. 296\u2013307."},{"issue":"8","key":"e_1_3_2_86_2","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1016\/j.infsof.2013.02.009","article-title":"Software fault prediction metrics: A systematic literature review","volume":"55","author":"Radjenovi\u00e9 Danijel","year":"2013","unstructured":"Danijel Radjenovi\u00e9, Marjan Heri\u010dko, Richard Torkar, and Ale\u0161 \u017divkovi\u010d. 2013. Software fault prediction metrics: A systematic literature review. Inf. Softw. Technol. 55, 8 (2013), 1397\u20131418.","journal-title":"Inf. Softw. Technol."},{"key":"e_1_3_2_87_2","doi-asserted-by":"crossref","unstructured":"Foyzur Rahman Daryl Posnett Abram Hindle Earl Barr and Premkumar Devanbu. 2011. BugCache for inspections: Hit or miss?19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering. 322\u2013331.","DOI":"10.1145\/2025113.2025157"},{"key":"e_1_3_2_88_2","first-page":"131","volume-title":"Proceedings of the IEEE 17th International Working Conference on Source Code Analysis and Manipulation (SCAM\u201917)","author":"Rahman Md Saidur","year":"2017","unstructured":"Md Saidur Rahman and Chanchal K. Roy. 2017. On the relationships between stability and bug-proneness of code clones: An empirical study. In Proceedings of the IEEE 17th International Working Conference on Source Code Analysis and Manipulation (SCAM\u201917). 131\u2013140."},{"key":"e_1_3_2_89_2","first-page":"13","volume-title":"Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering","author":"Ralph Paul","year":"2018","unstructured":"Paul Ralph and Ewan Tempero. 2018. Construct validity in software engineering research and software metrics. In Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering. 13\u201323."},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884848"},{"issue":"1","key":"e_1_3_2_91_2","first-page":"21","article-title":"Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests","volume":"2","author":"Razal Nornadiah Mohd","year":"2011","unstructured":"Nornadiah Mohd Razal and Bee Wah Yap. 2011. Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. J. Stat. Model. Analyt. 2, 1 (2011), 21\u201333.","journal-title":"J. Stat. Model. Analyt."},{"key":"e_1_3_2_92_2","first-page":"708","volume-title":"Proceedings of the European Conference on Machine Learning","author":"Rebbapragada Umaa","year":"2007","unstructured":"Umaa Rebbapragada and Carla E. Brodley. 2007. Class noise mitigation through instance weighting. In Proceedings of the European Conference on Machine Learning. 708\u2013715."},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-39940-9_565"},{"key":"e_1_3_2_94_2","first-page":"303","volume-title":"Proceedings of the 27th IEEE International Conference on Software Maintenance","author":"Romano D.","year":"2011","unstructured":"D. Romano and M. Pinzger. 2011. Using source code metrics to predict change-prone Java interfaces. In Proceedings of the 27th IEEE International Conference on Software Maintenance. 303\u2013312."},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00049"},{"key":"e_1_3_2_96_2","first-page":"1","volume-title":"Proceedings of the IEEE 24th International Conference on Program Comprehension","author":"Scalabrino S.","year":"2016","unstructured":"S. Scalabrino, M. Linares-V\u00e1squez, D. Poshyvanyk, and R. Oliveto. 2016. Improving code readability models with textual features. In Proceedings of the IEEE 24th International Conference on Program Comprehension. 1\u201310."},{"key":"e_1_3_2_97_2","unstructured":"Matteson Scott. 2018. Report: Software Failure Caused $1.7 Trillion in Financial Losses in 2017. Retrieved from https:\/\/www.techrepublic.com\/article\/report-software-failure-caused-1-7-trillion-in-financial-losses-in-2017\/"},{"key":"e_1_3_2_98_2","first-page":"746","volume-title":"Proceedings of the International Conference on Software Engineering (ICSE\u201917)","author":"Servant Francisco","year":"2017","unstructured":"Francisco Servant and James A. Jones. 2017. Fuzzy fine-grained code-history analysis. In Proceedings of the International Conference on Software Engineering (ICSE\u201917). 746\u2013757."},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026582314146"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/2393596.2393670"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.81"},{"key":"e_1_3_2_102_2","doi-asserted-by":"crossref","unstructured":"Thomas Shippey Tracy Hall Steve Counsell and David Bowes. 2016. So you need more method level datasets for your software defect prediction? Voil\u00e0! InProceedings of the ACM-IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM\u201916).","DOI":"10.1145\/2961111.2962620"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2012.43"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/1082983.1083147"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2010.90"},{"key":"e_1_3_2_106_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance and Evolution","author":"Spadini D.","year":"2018","unstructured":"D. Spadini, F. Palomba, A. Zaidman, M. Bruntink, and A. Bacchelli. 2018. On the relation of test smells to software code quality. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution. 1\u201312."},{"key":"e_1_3_2_107_2","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1145\/2597073.2597111","volume-title":"Proceedings of the Working Conference on Mining Software Repositories (MSR\u201914)","author":"Steidl Daniela","year":"2014","unstructured":"Daniela Steidl, Benjamin Hummel, and Elmar Juergens. 2014. Incremental origin analysis of source code files. In Proceedings of the Working Conference on Mining Software Repositories (MSR\u201914). 42\u201351."},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2020.106940"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2012.2226152"},{"key":"e_1_3_2_110_2","doi-asserted-by":"publisher","DOI":"10.1145\/3183519.3183547"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/2557833.2557853"},{"key":"e_1_3_2_112_2","unstructured":"VerifySoft. 2022. VerifySoft Maintainability Index. Retrieved from https:\/\/verifysoft.com\/en_maintainability.html"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2877678"},{"key":"e_1_3_2_114_2","doi-asserted-by":"crossref","unstructured":"Song Wang Junjie Wang Jaechang Nam and Nachiappan Nagappan. 2021. Continuous software bug prediction. 15th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement. Article 14 12 pages.","DOI":"10.1145\/3475716.3475790"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-015-0179-1"},{"key":"e_1_3_2_116_2","first-page":"1","article-title":"BugPre: An intelligent software version-to-version bug prediction system using graph convolutional neural networks","author":"Wang Zixu","year":"2022","unstructured":"Zixu Wang, Weiyuan Tong, Peng Li, Guixin Ye, Hao Chen, Xiaoqing Gong, and Zhanyong Tang. 2022. BugPre: An intelligent software version-to-version bug prediction system using graph convolutional neural networks. Complex Intell. Syst. 9, 4 (2022), 1\u201321.","journal-title":"Complex Intell. Syst."},{"issue":"05","key":"e_1_3_2_117_2","doi-asserted-by":"crossref","first-page":"1480","DOI":"10.1109\/TSE.2020.3023177","article-title":"Predicting defective lines using a model-agnostic technique","volume":"48","author":"Wattanakriengkrai S.","year":"2022","unstructured":"S. Wattanakriengkrai, P. Thongtanunam, C. Tantithamthavorn, H. Hata, and K. Matsumoto. 2022. Predicting defective lines using a model-agnostic technique. IEEE Trans. Softw. Eng. 48, 05 (May2022), 1480\u20131496.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_118_2","volume-title":"Data Mining","author":"Witten Ian H.","year":"2005","unstructured":"Ian H. Witten, Eibe Frank, Mark A. Hall, and Christopher J. Pal. 2005. Practical machine learning tools and techniques. In Data Mining, Vol. 2. Elsevier."},{"key":"e_1_3_2_119_2","first-page":"1","volume-title":"Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA\u201916)","author":"Wong Sebastien C.","year":"2016","unstructured":"Sebastien C. Wong, Adam Gatt, Victor Stamatescu, and Mark D. McDonnell. 2016. Understanding data augmentation for classification: When to warp? In Proceedings of the International Conference on Digital Image Computing: Techniques and Applications (DICTA\u201916). 1\u20136."},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1080\/17457300.2020.1746814"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2017.03.007"},{"key":"e_1_3_2_122_2","first-page":"350","volume-title":"Proceedings of the IEEE International Conference on Software Maintenance","author":"Zhang F.","year":"2013","unstructured":"F. Zhang, A. Mockus, Y. Zou, F. Khomh, and A. E. Hassan. 2013. How does context affect the distribution of software maintainability metrics? In Proceedings of the IEEE International Conference on Software Maintenance. 350\u2013359."},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2009.11.704"},{"key":"e_1_3_2_124_2","first-page":"920","volume-title":"Proceedings of the 20th International Conference on Machine Learning (ICML\u201903)","author":"Zhu Xingquan","year":"2003","unstructured":"Xingquan Zhu, Xindong Wu, and Qijun Chen. 2003. Eliminating class noise in large datasets. In Proceedings of the 20th International Conference on Machine Learning (ICML\u201903). 920\u2013927."},{"key":"e_1_3_2_125_2","volume-title":"Proceedings of the 3rd International Workshop on Predictor Models in Software Engineering","author":"Zimmermann Thomas","year":"2007","unstructured":"Thomas Zimmermann, Rahul Premraj, and Andreas Zeller. 2007. Predicting defects for eclipse. In Proceedings of the 3rd International Workshop on Predictor Models in Software Engineering."}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3640331","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3640331","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:26Z","timestamp":1750287026000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3640331"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,18]]},"references-count":124,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3640331"],"URL":"https:\/\/doi.org\/10.1145\/3640331","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,18]]},"assertion":[{"value":"2022-11-18","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-19","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}