{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T22:56:37Z","timestamp":1773442597526,"version":"3.50.1"},"reference-count":98,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T00:00:00Z","timestamp":1649894400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T00:00:00Z","timestamp":1649894400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Code reviewing is a widespread practice used by software engineers to maintain high code quality. To date, the knowledge on the effect of code review on source code is still limited. Some studies have addressed this problem by classifying the types of changes that take place during the review process (<jats:italic>a.k.a. review changes<\/jats:italic>), as this strategy can, for example, pinpoint the immediate effect of reviews on code. Nevertheless, this classification (1) is not scalable, as it was conducted manually, and (2) was not assessed in terms of how meaningful the provided information is for practitioners. This paper aims at addressing these limitations: First, we investigate to what extent a machine learning-based technique can automatically classify review changes. Then, we evaluate the relevance of information on review change types and its potential usefulness, by conducting (1) semi-structured interviews with 12 developers and (2) a qualitative study with 17 developers, who are asked to assess reports on the review changes of their project. Key results of the study show that not only it is possible to automatically classify code review changes, but this information is also perceived by practitioners as valuable to improve the code review process. Data and materials: <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"doi\" xlink:href=\"https:\/\/doi.org\/10.5281\/zenodo.5592254\">10.5281\/zenodo.5592254<\/jats:ext-link><\/jats:p>","DOI":"10.1007\/s10664-021-10075-5","type":"journal-article","created":{"date-parts":[[2022,4,14]],"date-time":"2022-04-14T11:04:20Z","timestamp":1649934260000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["What happens in my code reviews? An investigation on automatically classifying review changes"],"prefix":"10.1007","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6897-7665","authenticated-orcid":false,"given":"Enrico","family":"Fregnan","sequence":"first","affiliation":[]},{"given":"Fernando","family":"Petrulio","sequence":"additional","affiliation":[]},{"given":"Linda","family":"Di Geronimo","sequence":"additional","affiliation":[]},{"given":"Alberto","family":"Bacchelli","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,4,14]]},"reference":[{"key":"10075_CR1","unstructured":"Crucible official website (2019) https:\/\/www.atlassian.com\/software\/crucible"},{"key":"10075_CR2","unstructured":"Gerrit Code Review (2019) https:\/\/www.gerritcodereview.com"},{"key":"10075_CR3","unstructured":"GitHub official website (2019) https:\/\/github.com"},{"key":"10075_CR4","unstructured":"About qt (2019) https:\/\/wiki.qt.io\/About_Qt"},{"issue":"1","key":"10075_CR5","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1007\/s10664-013-9278-4","volume":"20","author":"U Abelein","year":"2015","unstructured":"Abelein U, Paech B (2015) Understanding the influence of user participation and involvement on system success\u2013a systematic mapping study. Empir Softw Eng 20(1):28\u201381","journal-title":"Empir Softw Eng"},{"key":"10075_CR6","unstructured":"Android (2020) Android gerrit online repository https:\/\/git.eclipse.org\/r\/q\/status:open+-is:wip"},{"key":"10075_CR7","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1214\/09-SS054","volume":"4","author":"S Arlot","year":"2010","unstructured":"Arlot S, Celisse A, et al. (2010) A survey of cross-validation procedures for model selection. Statistics Surveys 4:40\u201379","journal-title":"Statistics Surveys"},{"key":"10075_CR8","doi-asserted-by":"publisher","unstructured":"Bacchelli A, Bird C (2013) Expectations, outcomes, and challenges of modern code review. In: Proceedings of the 2013 international conference on software engineering, ICSE \u201913. ISBN 978-1-4673-3076-3, https:\/\/doi.org\/10.1109\/ICSE.2013.6606617. IEEE Press, Piscataway, pp 712\u2013721","DOI":"10.1109\/ICSE.2013.6606617"},{"key":"10075_CR9","volume-title":"Modern Information Retrieval, vol 463","author":"R Baeza-Yates","year":"1999","unstructured":"Baeza-Yates R, Ribeiro-Neto B, et al. (1999) Modern Information Retrieval, vol 463. ACM Press, New York"},{"key":"10075_CR10","unstructured":"Ball T, Kim J-M, Porter AA, Siy HP (1997) If your version control system could talk. In: ICSE Workshop on process modelling and empirical studies of software engineering, vol 11"},{"key":"10075_CR11","doi-asserted-by":"crossref","unstructured":"Baum T, Liskin O, Niklas K, Schneider K (2016) A faceted classification scheme for change-based industrial code review processes. In: 2016 IEEE International conference on software quality, reliability and security (QRS), pp 74\u201385","DOI":"10.1109\/QRS.2016.19"},{"key":"10075_CR12","doi-asserted-by":"crossref","unstructured":"Baum T, Le\u00dfmann H, Schneider K (2017) The choice of code review process: A survey on the state of the practice. In: Product-focused software process improvement. ISBN 978-3-319-69926-4. Springer International Publishing, Cham, pp 111\u2013127","DOI":"10.1007\/978-3-319-69926-4_9"},{"key":"10075_CR13","doi-asserted-by":"crossref","unstructured":"Baum T, Schneider K, Bacchelli A (2019) Associating working memory capacity and code change ordering with code review performance. Empirical Software Engineering, pp 1\u201337","DOI":"10.1007\/s10664-018-9676-8"},{"key":"10075_CR14","doi-asserted-by":"crossref","unstructured":"Bavota G, Russo B (2015) Four eyes are better than two: on the impact of code reviews on software quality. In: 2015 IEEE International conference on software maintenance and evolution (ICSME), IEEE, pp 81\u201390","DOI":"10.1109\/ICSM.2015.7332454"},{"issue":"3","key":"10075_CR15","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1007\/s10664-015-9366-8","volume":"21","author":"O Baysal","year":"2016","unstructured":"Baysal O, Kononenko O, Holmes R, Godfrey MW (2016) Investigating technical and non-technical factors influencing modern code review. Empir Softw Eng 21(3):932\u2013959","journal-title":"Empir Softw Eng"},{"key":"10075_CR16","doi-asserted-by":"crossref","unstructured":"Beckwith L, Kissinger C, Burnett M, Wiedenbeck S, Lawrance J, Blackwell A, Cook C (2006) Tinkering and gender in end-user programmers\u2019 debugging. In: Proceedings of the SIGCHI conference on Human Factors in computing systems, pp 231\u2013240","DOI":"10.1145\/1124772.1124808"},{"key":"10075_CR17","doi-asserted-by":"publisher","unstructured":"Beller M, Bacchelli A, Zaidman A, Juergens E (2014) Modern code reviews in open-source projects: Which problems do they fix?. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014. ISBN 978-1-4503-2863-0. https:\/\/doi.org\/10.1145\/2597073.2597082. ACM, New York, pp 202\u2013211","DOI":"10.1145\/2597073.2597082"},{"key":"10075_CR18","doi-asserted-by":"crossref","unstructured":"Bettenburg N, Just S, Schr\u00f6ter A., Weiss C, Premraj R, Zimmermann T (2008) What makes a good bug report?. In: Proceedings of the 16th ACM SIGSOFT international symposium on foundations of software engineering, pp 308\u2013318","DOI":"10.1145\/1453101.1453146"},{"key":"10075_CR19","doi-asserted-by":"crossref","unstructured":"Bird C, Carnahan T, Greiler M (2015) Lessons learned from building and deploying a code review analytics platform. In: 2015 IEEE\/ACM 12Th working conference on mining software repositories, IEEE, pp 191\u2013201","DOI":"10.1109\/MSR.2015.25"},{"key":"10075_CR20","unstructured":"Bishop CM (2006) Pattern recognition and machine learning. Springer"},{"key":"10075_CR21","doi-asserted-by":"crossref","unstructured":"Bosu A, Carver JC (2014) Impact of developer reputation on code review outcomes in oss projects: an empirical investigation. In: Proceedings of the 8th ACM\/IEEE international symposium on empirical software engineering and measurement, ACM, pp 33","DOI":"10.1145\/2652524.2652544"},{"issue":"5","key":"10075_CR22","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1016\/j.intcom.2011.06.004","volume":"23","author":"MM Burnett","year":"2011","unstructured":"Burnett MM, Beckwith L, Wiedenbeck S, Fleming SD, Cao J, Park TH, Grigoreanu V, Rector K (2011) Gender pluralism in problem-solving software. Interacting with Computers 23(5):450\u2013460","journal-title":"Interacting with Computers"},{"issue":"4","key":"10075_CR23","doi-asserted-by":"publisher","first-page":"546","DOI":"10.1109\/TSE.2009.70","volume":"36","author":"RPL Buse","year":"2010","unstructured":"Buse RPL, Weimer WR (2010) Learning a metric for code readability. IEEE Trans Softw Eng 36(4):546\u2013558. ISSN 0098-5589. https:\/\/doi.org\/10.1109\/TSE.2009.70","journal-title":"IEEE Trans Softw Eng"},{"issue":"1","key":"10075_CR24","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.compeleceng.2013.11.024","volume":"40","author":"G Chandrashekar","year":"2014","unstructured":"Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Computers & Electrical Engineering 40(1):16\u201328","journal-title":"Computers & Electrical Engineering"},{"key":"10075_CR25","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1613\/jair.953","volume":"16","author":"NV Chawla","year":"2002","unstructured":"Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research 16:321\u2013357","journal-title":"Journal of Artificial Intelligence Research"},{"key":"10075_CR26","unstructured":"Couchbase (2020) Couchbase gerrit online repository http:\/\/review.couchbase.org\/q\/status:open"},{"key":"10075_CR27","doi-asserted-by":"crossref","unstructured":"Czerwonka J, Greiler M, Tilford J (2015) Code reviews do not find bugs: how the current code review best practice slows us down. In: Proceedings of the 37th international conference on software engineering, Vol 2, IEEE Press, pp 27\u201328","DOI":"10.1109\/ICSE.2015.131"},{"issue":"10","key":"10075_CR28","doi-asserted-by":"publisher","first-page":"1469","DOI":"10.1016\/j.infsof.2009.04.013","volume":"51","author":"M Di Penta","year":"2009","unstructured":"Di Penta M, Cerulo L, Aversano L (2009) The life and death of statically detected vulnerabilities: an empirical study. Inf Softw Technol 51(10):1469\u20131484","journal-title":"Inf Softw Technol"},{"issue":"10","key":"10075_CR29","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2347736.2347755","volume":"55","author":"PM Domingos","year":"2012","unstructured":"Domingos PM (2012) A few useful things to know about machine learning. Commun Acm 55(10):78\u201387","journal-title":"Commun Acm"},{"key":"10075_CR30","unstructured":"Eclipse (2020) Eclipse gerrit online repository https:\/\/git.eclipse.org\/r\/q\/status:open+-is:wip"},{"key":"10075_CR31","unstructured":"Elkan C (2001) The foundations of cost-sensitive learning. In: International joint conference on artificial intelligence, vol 17, Lawrence Erlbaum Associates ltd, pp 973\u2013978"},{"key":"10075_CR32","doi-asserted-by":"crossref","unstructured":"Fagan M (2002) Design and code inspections to reduce errors in program development. In: Software pioneers, Springer, pp 575\u2013607","DOI":"10.1007\/978-3-642-59412-0_35"},{"key":"10075_CR33","doi-asserted-by":"crossref","unstructured":"Fink A (2003) How to design survey studies. Sage","DOI":"10.4135\/9781412984447"},{"issue":"11","key":"10075_CR34","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1109\/TSE.2007.70731","volume":"33","author":"B Fluri","year":"2007","unstructured":"Fluri B, Wuersch M, Pinzger M, Gall H (2007) Change distilling: Tree differencing for fine-grained source code change extraction. IEEE Trans Softw Eng 33(11):725\u2013743","journal-title":"IEEE Trans Softw Eng"},{"key":"10075_CR35","doi-asserted-by":"publisher","unstructured":"Fregnan E, Petrulio F, Di Geronimo L, Bacchelli A (2020) What happens in my code reviews? replication package. https:\/\/doi.org\/10.5281\/zenodo.5592254","DOI":"10.5281\/zenodo.5592254"},{"key":"10075_CR36","doi-asserted-by":"crossref","unstructured":"Gata W, Grand G, Fatmasari R, Baharuddin B, Patras YE, Hidayat R, Tohari S, Wardhani NK (2019) Prediction of teachers\u2019 lateness factors coming to school using c4. 5, random tree, random forest algorithm. In: 2Nd international conference on research of educational administration and management (ICREAM 2018), Atlantis Press, pp 161\u2013166","DOI":"10.2991\/icream-18.2019.34"},{"key":"10075_CR37","doi-asserted-by":"crossref","unstructured":"Giger E, D\u2019Ambros M, Pinzger M, Gall HC (2012) Method-level bug prediction. In: Proceedings of the 2012 ACM-IEEE international symposium on empirical software engineering and measurement, IEEE, pp 171\u2013180","DOI":"10.1145\/2372251.2372285"},{"issue":"1","key":"10075_CR38","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1145\/1656274.1656278","volume":"11","author":"M Hall","year":"2009","unstructured":"Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. ACM SIGKDD Explorations Newsletter 11(1):10\u201318","journal-title":"ACM SIGKDD Explorations Newsletter"},{"key":"10075_CR39","unstructured":"Hall MA (1999) Correlation-based feature selection for machine learning"},{"key":"10075_CR40","unstructured":"Jiarpakdee J, Tantithamthavorn C, Hassan AE (2019) The impact of correlated metrics on the interpretation of defect models. IEEE Transactions on Software Engineering"},{"key":"10075_CR41","doi-asserted-by":"crossref","unstructured":"Johnson B, Song Y, Murphy-Hill E, Bowdidge R (2013) Why don\u2019t software developers use static analysis tools to find bugs? In: Proceedings of the 2013 international conference on software engineering, IEEE Press, pp 672\u2013681","DOI":"10.1109\/ICSE.2013.6606613"},{"issue":"7","key":"10075_CR42","doi-asserted-by":"publisher","first-page":"14","DOI":"10.3102\/0013189X033007014","volume":"33","author":"RB Johnson","year":"2004","unstructured":"Johnson RB, Onwuegbuzie AJ (2004) Mixed methods research: a research paradigm whose time has come. Educational Researcher 33(7):14\u201326","journal-title":"Educational Researcher"},{"key":"10075_CR43","doi-asserted-by":"crossref","unstructured":"Kamei Y, Matsumoto S, Monden A, Matsumoto K-i, Adams B, Hassan AE (2010) Revisiting common bug prediction findings using effort-aware models. In: 2010 IEEE International conference on software maintenance, IEEE, pp 1\u201310","DOI":"10.1109\/ICSM.2010.5609530"},{"key":"10075_CR44","unstructured":"Kaner C, Falk J, Nguyen HQ (2000) Testing computer software. 2nd edn, Dreamtech press"},{"issue":"2","key":"10075_CR45","first-page":"271","volume":"2","author":"AG Karegowda","year":"2010","unstructured":"Karegowda AG, Manjunath A, Jayaram M (2010) Comparative study of attribute selection using gain ratio and correlation based feature selection. International Journal of Information Technology and Knowledge Management 2(2):271\u2013277","journal-title":"International Journal of Information Technology and Knowledge Management"},{"issue":"4","key":"10075_CR46","doi-asserted-by":"publisher","first-page":"534","DOI":"10.1109\/TSE.2009.27","volume":"35","author":"CF Kemerer","year":"2009","unstructured":"Kemerer CF, Paulk MC (2009) The impact of design and code reviews on software quality: an empirical study based on psp data. IEEE Trans Softw Eng 35 (4):534\u2013550","journal-title":"IEEE Trans Softw Eng"},{"key":"10075_CR47","doi-asserted-by":"crossref","unstructured":"Kononenko O, Baysal O, Guerrouj L, Cao Y, Godfrey MW (2015) Investigating code review quality: Do people and participation matter? In: 2015 IEEE international conference on software maintenance and evolution (ICSME)","DOI":"10.1109\/ICSM.2015.7332457"},{"issue":"2","key":"10075_CR48","first-page":"111","volume":"1","author":"S Kotsiantis","year":"2006","unstructured":"Kotsiantis S, Kanellopoulos D, Pintelas P (2006) Data preprocessing for supervised leaning. Int J Comput Sci 1(2):111\u2013117","journal-title":"Int J Comput Sci"},{"issue":"7","key":"10075_CR49","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1109\/TSE.2018.2868367","volume":"46","author":"V Kovalenko","year":"2020","unstructured":"Kovalenko V, Tintarev N, Pasynkov E, Bird C, Bacchelli A (2020) Does reviewer recommendation help developers?. IEEE Trans Softw Eng 46(7):710\u2013731. https:\/\/doi.org\/10.1109\/TSE.2018.2868367","journal-title":"IEEE Trans Softw Eng"},{"issue":"4","key":"10075_CR50","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","volume":"5","author":"B Krawczyk","year":"2016","unstructured":"Krawczyk B (2016) Learning from imbalanced data: open challenges and future directions. Progress in Artif Intell 5(4):221\u2013232","journal-title":"Progress in Artif Intell"},{"key":"10075_CR51","unstructured":"Krippendorff K (2011) Computing krippendorff\u2019s alpha-reliability"},{"key":"10075_CR52","doi-asserted-by":"crossref","unstructured":"Kumar L, Satapathy SM, Murthy LB (2019) Method level refactoring prediction on five open source java projects using machine learning techniques. In: Proceedings of the 12th innovations on software engineering conference (formerly known as India Software Engineering Conference), pp 1\u201310","DOI":"10.1145\/3299771.3299777"},{"issue":"4","key":"10075_CR53","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1109\/TSE.2008.35","volume":"34","author":"S Lessmann","year":"2008","unstructured":"Lessmann S, Baesens B, Mues C, Pietsch S (2008) Benchmarking classification models for software defect prediction: a proposed framework and novel findings. IEEE Trans Softw Eng 34(4):485\u2013496","journal-title":"IEEE Trans Softw Eng"},{"key":"10075_CR54","unstructured":"Likert R (1932) A technique for the measurement of attitudes. Archives of psychology"},{"key":"10075_CR55","first-page":"143","volume":"3","author":"R Longhurst","year":"2003","unstructured":"Longhurst R (2003) Semi-structured interviews and focus groups. Key Methods in Geography 3:143\u2013156","journal-title":"Key Methods in Geography"},{"key":"10075_CR56","doi-asserted-by":"crossref","unstructured":"Lyons EE, Coyle AE (2007) Analysing qualitative data in psychology. Sage Publications Ltd","DOI":"10.4135\/9781446207536"},{"key":"10075_CR57","doi-asserted-by":"crossref","unstructured":"Mahboob T, Irfan S, Karamat A (2016) A machine learning approach for student assessment in e-learning using quinlan\u2019s c4. 5, naive bayes and random forest algorithms. In: 2016 19Th international multi-topic conference (INMIC), IEEE, pp 1\u20138","DOI":"10.1109\/INMIC.2016.7840094"},{"issue":"3","key":"10075_CR58","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1109\/TSE.2008.71","volume":"35","author":"MV M\u00e4ntyl\u00e4","year":"2009","unstructured":"M\u00e4ntyl\u00e4 MV, Lassenius C (2009) What types of defects are really discovered in code reviews?. IEEE Transactions on Software Engineering 35(3):430\u2013448. ISSN 0098-5589. https:\/\/doi.org\/10.1109\/TSE.2008.71","journal-title":"IEEE Transactions on Software Engineering"},{"key":"10075_CR59","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/TSE.1976.233837","volume":"4","author":"TJ McCabe","year":"1976","unstructured":"McCabe TJ (1976) A complexity measure. IEEE Transactions on software Engineering 4:308\u2013320","journal-title":"IEEE Transactions on software Engineering"},{"key":"10075_CR60","doi-asserted-by":"crossref","unstructured":"McIntosh S, Kamei Y, Adams B, Hassan AE (2014) The impact of code review coverage and code review participation on software quality: a case study of the qt, vtk, and itk projects. In: Proceedings of the 11th working conference on mining software repositories, ACM, pp 192\u2013201","DOI":"10.1145\/2597073.2597076"},{"issue":"5","key":"10075_CR61","doi-asserted-by":"publisher","first-page":"2146","DOI":"10.1007\/s10664-015-9381-9","volume":"21","author":"S McIntosh","year":"2016","unstructured":"McIntosh S, Kamei Y, Adams B, Hassan AE (2016) An empirical study of the impact of modern code review practices on software quality. Empir Softw Eng 21(5):2146\u20132189","journal-title":"Empir Softw Eng"},{"key":"10075_CR62","doi-asserted-by":"crossref","unstructured":"Morales R, McIntosh S, Khomh F (2015) Do code review practices impact design quality? a case study of the qt, vtk, and itk projects. In: 2015 IEEE 22Nd international conference on software analysis, evolution, and reengineering (SANER), IEEE, pp 171\u2013180","DOI":"10.1109\/SANER.2015.7081827"},{"key":"10075_CR63","doi-asserted-by":"crossref","unstructured":"Newcomer KE, Hatry HP, Wholey JS (2015) Conducting semi-structured interviews. Handbook of practical program evaluation, 492","DOI":"10.1002\/9781119171386"},{"key":"10075_CR64","doi-asserted-by":"crossref","unstructured":"Ouni A, Kula RG, Inoue K (2016) Search-based peer reviewers recommendation in modern code review. In: 2016 IEEE International conference on software maintenance and evolution (ICSME), IEEE, pp 367\u2013377","DOI":"10.1109\/ICSME.2016.65"},{"key":"10075_CR65","doi-asserted-by":"crossref","unstructured":"Paixao M, Krinke J, Han D, Harman M (2018) Crop: Linking code reviews to source code changes. In: 2018 IEEE\/ACM 15Th international conference on mining software repositories (MSR), pp 46\u201349","DOI":"10.1145\/3196398.3196466"},{"key":"10075_CR66","doi-asserted-by":"crossref","unstructured":"Palomba F, Panichella A, De Lucia A, Oliveto R, Zaidman A (May 2016) A textual-based technique for smell detection. In: 2016 IEEE 24Th international conference on program comprehension (ICPC), pp 1\u201310","DOI":"10.1109\/ICPC.2016.7503704"},{"key":"10075_CR67","doi-asserted-by":"crossref","unstructured":"Pantiuchina J, Bavota G, Tufano M, Poshyvanyk D (2018) Towards just-in-time refactoring recommenders. In: 2018 IEEE\/ACM 26Th international conference on program comprehension (ICPC), IEEE, pp 312\u20133123","DOI":"10.1145\/3196321.3196365"},{"issue":"CSCW","key":"10075_CR68","doi-asserted-by":"publisher","first-page":"135:1","DOI":"10.1145\/3274404","volume":"2","author":"L Pascarella","year":"2018","unstructured":"Pascarella L, Spadini D, Palomba F, Bruntink M, Bacchelli A (2018) Information needs in contemporary code review. Proc ACM Hum-Comput Interact 2(CSCW):135:1\u2013135:27. ISSN 2573-0142. https:\/\/doi.org\/10.1145\/3274404","journal-title":"Proc ACM Hum-Comput Interact"},{"key":"10075_CR69","doi-asserted-by":"crossref","unstructured":"Pecorelli F, Palomba F, Di Nucci D, De Lucia A (2019) Comparing heuristic and machine learning approaches for metric-based code smell detection. In: Proceedings of the 27th international conference on program comprehension, IEEE Press, pp 93\u2013104","DOI":"10.1109\/ICPC.2019.00023"},{"key":"10075_CR70","doi-asserted-by":"crossref","unstructured":"Porter A, Siy H, Votta L (1996) A review of software inspections. volume 42 of Advances in Computers, pp 39\u201376. Elsevier","DOI":"10.1016\/S0065-2458(08)60484-2"},{"issue":"1","key":"10075_CR71","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1145\/268411.268421","volume":"7","author":"A Porter","year":"1998","unstructured":"Porter A, Siy H, Mockus A, Votta L (1998) Understanding the sources of variation in software inspections. ACM Trans Softw Eng Methodol (TOSEM) 7(1):41\u201379","journal-title":"ACM Trans Softw Eng Methodol (TOSEM)"},{"key":"10075_CR72","first-page":"313","volume-title":"Readings in information retrieval. chapter An Algorithm for Suffix Stripping","author":"MF Porter","year":"1997","unstructured":"Porter MF (1997) Readings in information retrieval. chapter An Algorithm for Suffix Stripping. Morgan Kaufmann Publishers Inc., San Francisco, pp 313\u2013316. ISBN 1-55860-454-5. http:\/\/dl.acm.org\/citation.cfm?id=275537.275705"},{"key":"10075_CR73","unstructured":"Portigal S (2013) Interviewing users: how to uncover compelling insights. Rosenfeld Media"},{"key":"10075_CR74","doi-asserted-by":"publisher","unstructured":"Ram A, Sawant AA, Castelluccio M, Bacchelli A (2018) What makes a code change easier to review: An empirical investigation on code change reviewability. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering, ESEC\/FSE 2018, New York, ACM, pp 201\u2013212, ISBN 978-1-4503-5573-5. https:\/\/doi.org\/10.1145\/3236024.3236080","DOI":"10.1145\/3236024.3236080"},{"issue":"3","key":"10075_CR75","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/S0954-1810(98)00021-1","volume":"13","author":"Y Reich","year":"1999","unstructured":"Reich Y, Barai S (1999) Evaluating machine learning models for engineering problems. Artif Intell Eng 13(3):257\u2013272","journal-title":"Artif Intell Eng"},{"key":"10075_CR76","doi-asserted-by":"crossref","unstructured":"Rigby PC, Bird C (2013) Convergent contemporary software peer review practices. In: Proceedings of the 2013 9th joint meeting on foundations of software engineering, ACM, pp 202\u2013212","DOI":"10.1145\/2491411.2491444"},{"issue":"4","key":"10075_CR77","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1145\/2594458","volume":"23","author":"PC Rigby","year":"2014","unstructured":"Rigby PC, German DM, Cowen L, Storey M-A (2014) Peer review on open-source software projects: Parameters, statistical models, and theory. ACM Trans Softw Eng Methodol (TOSEM) 23(4):35","journal-title":"ACM Trans Softw Eng Methodol (TOSEM)"},{"key":"10075_CR78","doi-asserted-by":"crossref","unstructured":"Sadowski C, Van Gogh J, Jaspan C, Soderberg E, Winter C (2015) Tricorder: Building a program analysis ecosystem. In: 2015 IEEE\/ACM 37Th IEEE international conference on software engineering, vol 1, IEEE, pp 598\u2013608","DOI":"10.1109\/ICSE.2015.76"},{"issue":"4","key":"10075_CR79","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1145\/3188720","volume":"61","author":"C Sadowski","year":"2018","unstructured":"Sadowski C, Aftandilian E, Eagle A, Miller-Cushon L, Jaspan C (2018) Lessons from building static analysis tools at google. Commun ACM 61 (4):58\u201366","journal-title":"Commun ACM"},{"issue":"4","key":"10075_CR80","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1109\/MCI.2018.2866730","volume":"13","author":"MS Santos","year":"2018","unstructured":"Santos MS, Soares JP, Abreu PH, Araujo H, Santos J (2018) Cross-validation for imbalanced datasets: Avoiding overoptimistic and overfitting approaches [research frontier]. IEEE Comput Intell Mag 13(4):59\u201376","journal-title":"IEEE Comput Intell Mag"},{"issue":"1","key":"10075_CR81","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/32.825763","volume":"26","author":"C Sauer","year":"2000","unstructured":"Sauer C, Jeffery DR, Land L, Yetton P (2000) The effectiveness of software development technical reviews: a behaviorally motivated program of research. IEEE Trans Softw Eng 26(1):1\u201314","journal-title":"IEEE Trans Softw Eng"},{"key":"10075_CR82","doi-asserted-by":"crossref","unstructured":"Shibuya B, Tamai T (2009) Understanding the process of participating in open source communities. In: Proceedings of the 2009 ICSE workshop on emerging trends in Free\/Libre\/Open source software research and development, IEEE Computer Society, pp 1\u20136","DOI":"10.1109\/FLOSS.2009.5071352"},{"key":"10075_CR83","doi-asserted-by":"crossref","unstructured":"Spadini D, Aniche M, Storey M-A, Bruntink M, Bacchelli A (2018) When testing meets code review: Why and how developers review tests. In: 2018 IEEE\/ACM 40Th international conference on software engineering (ICSE), IEEE, pp 677\u2013687","DOI":"10.1145\/3180155.3180192"},{"key":"10075_CR84","doi-asserted-by":"crossref","unstructured":"Spadini D, Palomba F, Baum T, Hanenberg S, Bruntink M, Bacchelli A (2019) Test-driven code review: an empirical study. In: Proceedings of the 41st international conference on software engineering, IEEE Press, pp 1061\u20131072","DOI":"10.1109\/ICSE.2019.00110"},{"key":"10075_CR85","doi-asserted-by":"crossref","unstructured":"Str\u00fcder S, Mukelabai M, Str\u00fcber D, Berger T (2020) Feature-oriented defect prediction. In: Proceedings of the 24th ACM conference on systems and software product line: Volume A-Volume A, pp 1\u201312","DOI":"10.1145\/3382025.3414960"},{"key":"10075_CR86","doi-asserted-by":"crossref","unstructured":"Tantithamthavorn C, Hassan AE (2018) An experience report on defect modelling in practice: Pitfalls and challenges. In: Proceedings of the 40th international conference on software engineering: software engineering in practice, ACM, pp 286\u2013295","DOI":"10.1145\/3183519.3183547"},{"issue":"1","key":"10075_CR87","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TSE.2016.2584050","volume":"43","author":"C Tantithamthavorn","year":"2016","unstructured":"Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2016) An empirical comparison of model validation techniques for defect prediction models. IEEE Trans Softw Eng 43(1):1\u201318","journal-title":"IEEE Trans Softw Eng"},{"key":"10075_CR88","doi-asserted-by":"crossref","unstructured":"Tantithamthavorn C, McIntosh S, Hassan AE, Matsumoto K (2018) The impact of automated parameter optimization on defect prediction models. IEEE Transactions on Software Engineering","DOI":"10.1109\/TSE.2018.2794977"},{"key":"10075_CR89","doi-asserted-by":"crossref","unstructured":"Thongtanunam P, McIntosh S, Hassan AE, Iida H (2015a) Investigating code review practices in defective files: an empirical study of the qt system. In: Proceedings of the 12th working conference on mining software repositories, IEEE Press, pp 168\u2013179","DOI":"10.1109\/MSR.2015.23"},{"key":"10075_CR90","doi-asserted-by":"crossref","unstructured":"Thongtanunam P, Tantithamthavorn C, Kula RG, Yoshida N, Iida H, Matsumoto K. -i. (2015b) Who should review my code? a file location-based code-reviewer recommendation approach for modern code review. In: 2015 IEEE 22Nd international conference on software analysis, evolution, and reengineering (SANER), IEEE, pp 141\u2013150","DOI":"10.1109\/SANER.2015.7081824"},{"key":"10075_CR91","doi-asserted-by":"crossref","unstructured":"Thongtanunam P, McIntosh S, Hassan AE, Iida H (2016) Revisiting code ownership and its relationship with software quality in the scope of modern code review. In: Proceedings of the 38th international conference on software engineering, ACM, pp 1039\u20131050","DOI":"10.1145\/2884781.2884852"},{"key":"10075_CR92","doi-asserted-by":"crossref","unstructured":"Vassallo C, Panichella S, Palomba F, Proksch S, Zaidman A, Gall HC (2018) Context is king: the developer perspective on the usage of static analysis tools. In: 2018 IEEE 25Th international conference on software analysis, evolution and reengineering (SANER), IEEE, pp 38\u201349","DOI":"10.1109\/SANER.2018.8330195"},{"key":"10075_CR93","doi-asserted-by":"crossref","unstructured":"Vassallo C, Panichella S, Palomba F, Proksch S, Gall HC, Zaidman A (2019a) How developers engage with static analysis tools in different contexts. Empirical Software Engineering","DOI":"10.1007\/s10664-019-09750-5"},{"key":"10075_CR94","doi-asserted-by":"crossref","unstructured":"Vassallo C, Proksch S, Gall HC, Di Penta M (2019a) Automated reporting of anti-patterns and decay in continuous integration. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering (ICSE), IEEE, pp 105\u2013115","DOI":"10.1109\/ICSE.2019.00028"},{"key":"10075_CR95","volume-title":"Peer Reviews in Software: A Practical Guide","author":"K Wiegers","year":"2002","unstructured":"Wiegers K (2002) Peer Reviews in Software: A Practical Guide. Addison-Wesley Longman Publishing Co., Inc., Boston. ISBN 0-201-73485-0"},{"key":"10075_CR96","doi-asserted-by":"crossref","unstructured":"Wolf T, Schroter A, Damian D, Nguyen T (2009) Predicting build failures using social network analysis on developer communication. In: 2009 IEEE 31St international conference on software engineering, IEEE, pp 1\u201311","DOI":"10.1109\/ICSE.2009.5070503"},{"issue":"6","key":"10075_CR97","doi-asserted-by":"publisher","first-page":"1091","DOI":"10.1109\/TPAMI.2007.1078","volume":"29","author":"L Yujian","year":"2007","unstructured":"Yujian L, Bo L (2007) A normalized levenshtein distance metric. IEEE Trans Pattern Anal Mach Intell 29(6):1091\u20131095. ISSN 0162-8828. https:\/\/doi.org\/10.1109\/TPAMI.2007.1078","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"6","key":"10075_CR98","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1109\/TSE.2015.2500238","volume":"42","author":"MB Zanjani","year":"2015","unstructured":"Zanjani MB, Kagdi H, Bird C (2015) Automatically recommending peer reviewers in modern code review. IEEE Trans Softw Eng 42(6):530\u2013543","journal-title":"IEEE Trans Softw Eng"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10075-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-10075-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-10075-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,9]],"date-time":"2022-06-09T21:01:57Z","timestamp":1654808517000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-10075-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,14]]},"references-count":98,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10075"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-10075-5","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,14]]},"assertion":[{"value":"28 October 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 April 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"89"}}