{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T14:48:44Z","timestamp":1778770124363,"version":"3.51.4"},"reference-count":53,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,9,30]],"date-time":"2021-09-30T00:00:00Z","timestamp":1632960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Understanding how developers refactor their code is critical to support the design improvement process of software. This paper investigates to what extent code metrics are good indicators for predicting refactoring activity in the source code. In order to perform this, we formulated the prediction of refactoring operation types as a multi-class classification problem. Our solution relies on measuring metrics extracted from committed code changes in order to extract the corresponding features (i.e., metric variations) that better represent each class (i.e., refactoring type) in order to automatically predict, for a given commit, the method-level type of refactoring being applied, namely Move Method, Rename Method, Extract Method, Inline Method, Pull-up Method, and Push-down Method. We compared various classifiers, in terms of their prediction performance, using a dataset of 5004 commits and extracted 800 Java projects. Our main findings show that the random forest model trained with code metrics resulted in the best average accuracy of 75%. However, we detected a variation in the results per class, which means that some refactoring types are harder to detect than others.<\/jats:p>","DOI":"10.3390\/a14100289","type":"journal-article","created":{"date-parts":[[2021,10,1]],"date-time":"2021-10-01T10:55:40Z","timestamp":1633085740000},"page":"289","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Comparing Commit Messages and Source Code Metrics for the Prediction Refactoring Activities"],"prefix":"10.3390","volume":"14","author":[{"given":"Priyadarshni Suresh","family":"Sagar","sequence":"first","affiliation":[{"name":"Rochester Institute of Technology, Rochester, New York, NY 14623, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1800-9268","authenticated-orcid":false,"given":"Eman Abdulah","family":"AlOmar","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, Rochester, New York, NY 14623, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohamed Wiem","family":"Mkaouer","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, Rochester, New York, NY 14623, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali","family":"Ouni","sequence":"additional","affiliation":[{"name":"Ecole de Technologie Superieure, University of Quebec, Quebec City, QC H3C 1K3, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8838-4074","authenticated-orcid":false,"given":"Christian D.","family":"Newman","sequence":"additional","affiliation":[{"name":"Rochester Institute of Technology, Rochester, New York, NY 14623, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tsantalis, N., Chaikalis, T., and Chatzigeorgiou, A. (2008, January 1\u20134). JDeodorant: Identification and removal of type-checking bad smells. Proceedings of the 2008 12th European Conference on Software Maintenance and Reengineering, Athens, Greece.","DOI":"10.1109\/CSMR.2008.4493342"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, M., Baddoo, N., Wernick, P., and Hall, T. (2011, January 21\u201325). Prioritising refactoring using code bad smells. Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, Berlin, Germany.","DOI":"10.1109\/ICSTW.2011.69"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2729974","article-title":"Many-objective software remodularization using NSGA-III","volume":"24","author":"Mkaouer","year":"2015","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1145\/2932631","article-title":"Multi-criteria code refactoring using search-based software engineering: An industrial case study","volume":"25","author":"Ouni","year":"2016","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Veerappa, V., and Harrison, R. (2013, January 10\u201311). An empirical validation of coupling metrics using automated refactoring. Proceedings of the 2013 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Baltimore, MD, USA.","DOI":"10.1109\/ESEM.2013.37"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Naiya, N., Counsell, S., and Hall, T. (2015, January 26\u201328). The Relationship between Depth of Inheritance and Refactoring: An Empirical Study of Eclipse Releases. Proceedings of the 2015 41st Euromicro Conference on Software Engineering and Advanced Applications, Madeira, Portugal.","DOI":"10.1109\/SEAA.2015.42"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ubayashi, N., Kamei, Y., and Sato, R. (2018, January 22\u201324). Can Abstraction Be Taught? Refactoring-based Abstraction Learning. Proceedings of the 6th International Conference on Model-Driven Engineering and Software Development, Madeira, Portugal.","DOI":"10.5220\/0006604804290437"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Counsell, S., Swift, S., Arzoky, M., and Destefanis, G. (2018, January 28\u201330). Do developers really worry about refactoring re-test? An empirical study of open-source systems. Proceedings of the International Conference on Product-Focused Software Process Improvement, Wolfsburg, Germany.","DOI":"10.1007\/978-3-030-03673-7_12"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Counsell, S., Arzoky, M., Destefanis, G., and Taibi, D. (2019, January 19\u201320). On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint. Proceedings of the 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Recife, Brazil.","DOI":"10.1109\/ESEM.2019.8870191"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pantiuchina, J., Lanza, M., and Bavota, G. (2018, January 23\u201329). Improving Code: The (Mis) perception of Quality Metrics. Proceedings of the 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME), Madrid, Spain.","DOI":"10.1109\/ICSME.2018.00017"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"AlOmar, E.A., Mkaouer, M.W., Ouni, A., and Kessentini, M. (2019, January 19\u201320). On the impact of refactoring on the relationship between quality attributes and design metrics. Proceedings of the 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Recife, Brazil.","DOI":"10.1109\/ESEM.2019.8870177"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"AlOmar, E.A., Rodriguez, P.T., Bowman, J., Wang, T., Adepoju, B., Lopez, K., Newman, C., Ouni, A., and Mkaouer, M.W. (2020, January 9\u201311). How do developers refactor code to improve code reusability?. Proceedings of the International Conference on Software and Software Reuse, Hammamet, Tunisia.","DOI":"10.1007\/978-3-030-64694-3_16"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Alrubaye, H., Alshoaibi, D., Alomar, E., Mkaouer, M.W., and Ouni, A. (2020, January 9\u201311). How does library migration impact software quality and comprehension? An empirical study. Proceedings of the International Conference on Software and Software Reuse, Hammamet, Tunisia.","DOI":"10.1007\/978-3-030-64694-3_15"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"106332","DOI":"10.1016\/j.infsof.2020.106332","article-title":"Recommending Refactorings via Commit Message Analysis","volume":"126","author":"Rebai","year":"2020","journal-title":"Inf. Softw. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Stroggylos, K., and Spinellis, D. (2007, January 20\u201326). Refactoring\u2013Does It Improve Software Quality?. Proceedings of the Fifth International Workshop on Software Quality (WoSQ\u201907: ICSE Workshops 2007), Minneapolis, MN, USA.","DOI":"10.1109\/WOSQ.2007.11"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ratzinger, J., Sigmund, T., and Gall, H.C. (2008, January 10\u201311). On the Relation of Refactorings and Software Defect Prediction. Proceedings of the 2008 International Working Conference on Mining Software Repositories, Leipzig, Germany.","DOI":"10.1145\/1370750.1370759"},{"key":"ref_17","unstructured":"Ratzinger, J. (2007). sPACE: Software Project Assessment in the Course of Evolution. [Ph.D. Thesis, Vienna University of Technology]."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1109\/TSE.2011.41","article-title":"How we refactor, and how we know it","volume":"38","author":"Parnin","year":"2012","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1006","DOI":"10.1016\/j.jss.2012.10.040","article-title":"Comparing approaches to analyze refactoring activity on software repositories","volume":"86","author":"Soares","year":"2013","journal-title":"J. Syst. Softw."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1109\/TSE.2014.2318734","article-title":"An empirical study of refactoring challenges and benefits at microsoft","volume":"40","author":"Kim","year":"2014","journal-title":"IEEE Trans. Softw. Eng."},{"key":"ref_21","unstructured":"Zhang, D., Bing, L., Zengyang, L., and Liang, P. (2018, January 1\u20133). A Preliminary Investigation of Self-Admitted Refactorings in Open Source Software (S). Proceedings of the 30th International Conference on Software Engineering and Knowledge Engineering, Hotel Pullman, Redwood City, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"AlOmar, E.A., Mkaouer, M.W., and Ouni, A. (2019, January 28). Can refactoring be self-affirmed? an exploratory study on how developers document their refactoring activities in commit messages. Proceedings of the 3nd International Workshop on Refactoring-Accepted, Montreal, QC, USA.","DOI":"10.1109\/IWoR.2019.00017"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"114176","DOI":"10.1016\/j.eswa.2020.114176","article-title":"How we refactor and how we document it? On the use of supervised machine learning algorithms to classify refactoring documentation","volume":"167","author":"AlOmar","year":"2020","journal-title":"Expert Syst. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"110821","DOI":"10.1016\/j.jss.2020.110821","article-title":"Toward the automatic classification of self-affirmed refactoring","volume":"171","author":"AlOmar","year":"2020","journal-title":"J. Syst. Softw."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"AlOmar, E.A., Mkaouer, M.W., and Ouni, A. (2020). Mining and Managing Big Data Refactoring for Design Improvement: Are We There Yet?. Knowledge Management in the Development of Data-Intensive Systems, Taylor & Francis.","DOI":"10.1201\/9781003001188-9"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Casalnuovo, C., Suchak, Y., Ray, B., and Rubio-Gonz\u00e1lez, C. (2017, January 10\u201314). Gitcproc: A tool for processing and classifying github commits. Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA, USA.","DOI":"10.1145\/3092703.3098230"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gharbi, S., Mkaouer, M.W., Jenhani, I., and Messaoud, M.B. (2019, January 8\u201312). On the classification of software change messages using multi-label active learning. Proceedings of the 34th ACM\/SIGAPP Symposium on Applied Computing, Limassol, Cyprus.","DOI":"10.1145\/3297280.3297452"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zafar, S., Malik, M.Z., and Walia, G.S. (2019, January 19\u201320). Towards standardizing and improving classification of bug-fix commits. Proceedings of the 2019 ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), Recife, Brazil.","DOI":"10.1109\/ESEM.2019.8870174"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shekarforoush, S., Green, R., and Dyer, R. (2017, January 14\u201319). Classifying commit messages: A case study in resampling techniques. Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN), Anchorage, AK, USA.","DOI":"10.1109\/IJCNN.2017.7965999"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Xie, R., Chen, L., Ye, W., Li, Z., Hu, T., Du, D., and Zhang, S. (2019, January 24\u201327). DeepLink: A code knowledge graph based deep learning approach for issue-commit link recovery. Proceedings of the 2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering (SANER), Hangzhou, China.","DOI":"10.1109\/SANER.2019.8667969"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"H\u00f6nel, S., Ericsson, M., L\u00f6we, W., and Wingkvist, A. (2019, January 22\u201326). Importance and aptitude of source code density for commit classification into maintenance activities. Proceedings of the 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS), Sofia, Bulgaria.","DOI":"10.1109\/QRS.2019.00027"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Levin, S., and Yehudai, A. (2017, January 8). Boosting automatic commit classification into maintenance activities by utilizing source code changes. Proceedings of the 13th International Conference on Predictive Models and Data Analytics in Software Engineering, Toronto, ON, Canada.","DOI":"10.1145\/3127005.3127016"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Mauczka, A., Brosch, F., Schanes, C., and Grechenig, T. (2015, January 16\u201317). Dataset of developer-labeled commit messages. Proceedings of the 2015 IEEE\/ACM 12th Working Conference on Mining Software Repositories, Florence, Italy.","DOI":"10.1109\/MSR.2015.71"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mockus, A., and Votta, L.G. (2000, January 11\u201314). Identifying Reasons for Software Changes using Historic Databases. Proceedings of the 2000 International Conference on Software Maintenance, ICSM, San Jose, CA, USA.","DOI":"10.1109\/ICSM.2000.883028"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Hassan, A.E. (2008, January 16\u201320). Automated Classification of Change Messages in Open Source Projects. Proceedings of the 2008 ACM Symposium on Applied Computing, Ceara, Brazil.","DOI":"10.1145\/1363686.1363876"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"de Lara, J., and Zisman, A. (2012). Tracing Your Maintenance Work\u2013A Cross-Project Validation of an Automated Classification Dictionary for Commit Messages. Fundamental Approaches to Software Engineering: 15th International Conference, FASE 2012, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2012, Tallinn, Estonia, 24 March \u20131 April 2012. Proceedings, Springer.","DOI":"10.1007\/978-3-642-28872-2"},{"key":"ref_37","unstructured":"Swanson, E.B. (1976, January 13\u201315). The Dimensions of Maintenance. Proceedings of the 2nd International Conference on Software Engineering, San Francisco, CA, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Hindle, A., German, D.M., and Holt, R. (2008, January 10\u201311). What Do Large Commits Tell Us?: A Taxonomical Study of Large Commits. Proceedings of the 2008 International Working Conference on Mining Software Repositories, Leipzig, Germany.","DOI":"10.1145\/1370750.1370773"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hindle, A., German, D.M., Godfrey, M.W., and Holt, R.C. (2009, January 17\u201319). Automatic classication of large changes into maintenance categories. Proceedings of the 2009 IEEE 17th International Conference on Program Comprehension, Vancouver, BC, Canada.","DOI":"10.1109\/ICPC.2009.5090025"},{"key":"ref_40","unstructured":"Hindle, A., Ernst, N.A., Godfrey, M.W., and Mylopoulos, J. (2008, January 10\u201311). Automated Topic Naming to Support Cross-project Analysis of Software Maintenance Activities. Proceedings of the 8th Working Conference on Mining Software Repositories, Leipzig, Germany."},{"key":"ref_41","unstructured":"Amor, J., Robles, G., Gonzalez-Barahona, J., Navarro Gsyc, A., Carlos, J., and Madrid, S. (2021, September 30). Discriminating Development Activities in Versioning Systems: A Case Study. Available online: https:\/\/www.researchgate.net\/profile\/Jesus-Gonzalez-Barahona\/publication\/228968358_Discriminating_development_activities_in_versioning_systems_A_case_study\/links\/0c9605200b2fd8eed9000000\/Discriminating-development-activities-in-versioning-systems-A-case-study.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Mahmoodian, N., Abdullah, R., and Murad, M.A.A. (2010, January 15\u201317). Text-based classification incoming maintenance requests to maintenance type. Proceedings of the 2010 International Symposium on Information Technology, Kuala Lumpur, Malaysia.","DOI":"10.1109\/ITSIM.2010.5561540"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"McMillan, C., Linares-Vasquez, M., Poshyvanyk, D., and Grechanik, M. (2011, January 25\u201330). Categorizing Software Applications for Maintenance. Proceedings of the 2011 27th IEEE International Conference on Software Maintenance, Williamsburg, VA, USA.","DOI":"10.1109\/ICSM.2011.6080801"},{"key":"ref_44","unstructured":"Aniche, M., Maziero, E., Durelli, R., and Durelli, V. (2020). The Effectiveness of Supervised Machine Learning Algorithms in Predicting Software Refactoring. IEEE Trans. Softw. Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"4315","DOI":"10.1007\/s10664-020-09871-2","article-title":"Feature requests-based recommendation of software refactorings","volume":"25","author":"Nyamawe","year":"2020","journal-title":"Empir. Softw. Eng."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"e1893","DOI":"10.1002\/smr.1893","article-title":"Identifying unusual commits on GitHub","volume":"30","author":"Goyal","year":"2018","journal-title":"J. Softw. Evol. Process."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"3219","DOI":"10.1007\/s10664-017-9512-6","article-title":"Curating GitHub for engineered software projects","volume":"22","author":"Munaiah","year":"2017","journal-title":"Empir. Softw. Eng."},{"key":"ref_48","unstructured":"Tsantalis, N., Mansouri, M., Eshkevari, L.M., Mazinanian, D., and Dig, D. (June, January 27). Accurate and efficient refactoring detection in commit history. Proceedings of the 40th International Conference on Software Engineering, Gothenburg, Sweden."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Mkaouer, M.W., Kessentini, M., Bechikh, S., Deb, K., and \u00d3 Cinn\u00e9ide, M. (2014, January 12\u201316). High dimensional search-based software engineering: Finding tradeoffs among 15 objectives for automating software refactoring using NSGA-III. Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, Vancouver, BC, Canada.","DOI":"10.1145\/2576768.2598366"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2503","DOI":"10.1007\/s10664-015-9414-4","article-title":"On the use of many quality attributes for software refactoring: A many-objective search-based software engineering approach","volume":"21","author":"Mkaouer","year":"2016","journal-title":"Empir. Softw. Eng."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"894","DOI":"10.1007\/s10664-016-9426-8","article-title":"A robust multi-objective approach to balance severity and importance of refactoring opportunities","volume":"22","author":"Mkaouer","year":"2017","journal-title":"Empir. Softw. Eng."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Hamdi, O., Ouni, A., AlOmar, E.A., Cinn\u00e9ide, M.\u00d3., and Mkaouer, M.W. (2021, January 17\u201319). An Empirical Study on the Impact of Refactoring on Quality Metrics in Android Applications. Proceedings of the 2021 IEEE\/ACM 8th International Conference on Mobile Software Engineering and Systems (MobileSoft), Madrid, Spain.","DOI":"10.1109\/MobileSoft52590.2021.00010"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Silva, D., Tsantalis, N., and Valente, M.T. (2016, January 13\u201318). Why We Refactor? Confessions of GitHub Contributors. Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Seattle, WA, USA.","DOI":"10.1145\/2950290.2950305"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/10\/289\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:08:13Z","timestamp":1760166493000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/10\/289"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,30]]},"references-count":53,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["a14100289"],"URL":"https:\/\/doi.org\/10.3390\/a14100289","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,30]]}}}