{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T13:35:59Z","timestamp":1775741759771,"version":"3.50.1"},"reference-count":131,"publisher":"Association for Computing Machinery (ACM)","issue":"13s","license":[{"start":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T00:00:00Z","timestamp":1689206400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>The accuracy reported for code smell-detecting tools varies depending on the dataset used to evaluate the tools. Our survey of 45 existing datasets reveals that the adequacy of a dataset for detecting smells highly depends on relevant properties such as the size, severity level, project types, number of each type of smell, number of smells, and the ratio of smelly to non-smelly samples in the dataset. Most existing datasets support God Class, Long Method, and Feature Envy, while six smells in Fowler and Beck's catalog are not supported by any datasets. We conclude that existing datasets suffer from imbalanced samples, lack of supporting severity level, and restriction to Java language.<\/jats:p>","DOI":"10.1145\/3596908","type":"journal-article","created":{"date-parts":[[2023,5,13]],"date-time":"2023-05-13T11:14:29Z","timestamp":1683976469000},"page":"1-48","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["A Systematic Literature Review on the Code Smells Datasets and Validation Mechanisms"],"prefix":"10.1145","volume":"55","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4289-0606","authenticated-orcid":false,"given":"Morteza","family":"Zakeri-Nasrabadi","sequence":"first","affiliation":[{"name":"School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4381-2773","authenticated-orcid":false,"given":"Saeed","family":"Parsa","sequence":"additional","affiliation":[{"name":"School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3522-5002","authenticated-orcid":false,"given":"Ehsan","family":"Esmaili","sequence":"additional","affiliation":[{"name":"School of Computer Engineering, Iran University of Science and Technology, Tehran, Iran"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9337-5116","authenticated-orcid":false,"given":"Fabio","family":"Palomba","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Salerno, Salerno, Italy"}]}],"member":"320","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Refactoring: Improving the Design of Existing Code","author":"Fowler M.","year":"2018","unstructured":"M. Fowler and K. Beck. 2018. Refactoring: Improving the Design of Existing Code, 2nd ed. Addison-Wesley."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.12.009"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330266"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110693"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34706-2_8"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.69"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-020-09498-y"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.13"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2017.12.034"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.3745\/JIPS.04.0184"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1142\/S021819401950013X"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.2174\/2352096513999200922125839"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.00015"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2880977"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2003.1235447"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3422392.3422415"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13369-019-04311-w"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3021460.3021489"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134500"},{"key":"e_1_3_2_21_2","unstructured":"B. Kitchenham and S. Charters. 2007. Guidelines for performing systematic literature reviews in software engineering Technical Report EBSE 2007-001 Keele University and Durham University Joint Report UK."},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.7533548"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2601248.2601268"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351723"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2013.56"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.7533578"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-017-9535-z"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2018.02.004"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2015.69"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9378-4"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2017.04.014"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34706-2_8"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3345629.3345630"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-020-09498-y"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383219.3383264"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.00015"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2019.00023"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/MLDS.2017.8"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2015.7381819"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2016.7503705"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/JCSSE.2011.5930143"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.5220\/0006338804740482"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-39742-4_6"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2019.110486"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.11591\/ijece.v7i6.pp3613-3621"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-10-3153-3_56"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2012.56"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2010.11.921"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/QUATIC.2010.60"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcs.2005.02.059"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-014-9316-x"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/2351676.2351701"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/QUATIC.2010.61"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2009.23"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/QSIC.2009.47"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRITO.2016.7784998"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-011-9171-y"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2009.50"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379597.3387457"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.3390\/e20050372"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1080\/09720510.2020.1799576"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2770122"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387906.3388618"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2018.2883603"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/2897073.2897100"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2013.6693086"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.4018\/IJRSDA.2019040103"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2936376"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/TENCON.2019.8929628"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2016.84"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2017.13"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3264598"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1170 2020.3007722"},{"key":"e_1_3_2_74_2","unstructured":"SonarSource SA. 2021. SonarQube. Retrieved September 22 2021 from https:\/\/www.sonarsource.com."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2007.4362684"},{"key":"e_1_3_2_76_2","volume-title":"Models and Methodology of System Dependability. Proceedings of RELCOMEX 2010: 5th International Conference on Dependability of Computer Systems DepCoS, Monographs of System Dependability","author":"Jureczko M.","year":"2010","unstructured":"M. Jureczko and D. Spinellis. 2010. Using object-oriented design metrics to predict software defects. In Models and Methodology of System Dependability. Proceedings of RELCOMEX 2010: 5th International Conference on Dependability of Computer Systems DepCoS, Monographs of System Dependability. 69\u201381."},{"key":"e_1_3_2_77_2","unstructured":"P. Team. 2021. POM. Retrieved September 21 2021 from https:\/\/wiki.ptidej.net\/doku.php?id=pom."},{"key":"e_1_3_2_78_2","unstructured":"Intooitus Srl. 2021. inFusion Hydrogen. Retrieved September 22 2021 from https:\/\/marketplace.eclipse.org\/content\/infusion-hydrogen."},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.5555\/1622467.1622471"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372760"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSMR.2010.44"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2018.8330192"},{"key":"e_1_3_2_83_2","unstructured":"Robusta. 2017. Retrieved March 23 2022 from https:\/\/github.com\/NTUTLAB1321Robusta\/Robusta."},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCCC.2015.7416572"},{"key":"e_1_3_2_85_2","unstructured":"F. A. Fontana. 2013. JCodeOdor: A software quality Advisor through design flaws detection. Retrieved from https:\/\/www.researchgate.net\/publication\/279190536_JCodeOdor_A_Software_Quality_Advisor_Through_Design_Flaws_Detection."},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.46"},{"key":"e_1_3_2_87_2","volume-title":"Object-oriented Metrics in Practice: Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems","author":"Lanza M.","year":"2006","unstructured":"M. Lanza and R. Marinescu. 2006. Object-oriented Metrics in Practice: Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems, 1st ed. Springer, Berlin.","edition":"1"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSM.2010.5609564"},{"key":"e_1_3_2_89_2","volume-title":"Proceedings of the 21st IEEE International Conference on Software Maintenance - Industrial and Tool volume (ICSM)","author":"Marinescu C.","year":"2005","unstructured":"C. Marinescu, R. Marinescu, P. F. Mihancea, D. Rat, and R. Wettel. 2005. iPlasma: An integrated platform for quality assessment of object-oriented design. In Proceedings of the 21st IEEE International Conference on Software Maintenance - Industrial and Tool volume (ICSM). 25\u201330."},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2004.21"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.295895"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/76380.76382"},{"key":"e_1_3_2_93_2","unstructured":"Apache. 2021. Apache Xerces. Retrieved September 21 2021 from https:\/\/xerces.apache.org\/."},{"key":"e_1_3_2_94_2","unstructured":"Eclipse Git repositories. 2022. Retrieved March 24 2022 from https:\/\/git.eclipse.org\/c\/."},{"key":"e_1_3_2_95_2","unstructured":"A. Thomas. 2021. GanttProject. Retrieved September 21 2021 from https:\/\/www.ganttproject.biz\/."},{"key":"e_1_3_2_96_2","unstructured":"J. E. Robbins. 2021. ArgoUML. Retrieved April 28 2021 from https:\/\/argouml.en.softonic.com\/."},{"key":"e_1_3_2_97_2","unstructured":"Apache. 2021. Apache Ant. Retrieved September 21 2021 from https:\/\/ant.apache.org\/."},{"key":"e_1_3_2_98_2","unstructured":"S. Pestov. 2021. jEdit\u2014programmer's text editor. Retrieved April 28 2021 from http:\/\/www.jedit.org\/."},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1109\/APSEC.2010.46"},{"key":"e_1_3_2_100_2","unstructured":"CERN. 2021. Zenodo. Retrieved September 21 2021 from https:\/\/zenodo.org\/."},{"key":"e_1_3_2_101_2","unstructured":"A. Goldbloom. 2021. Kaggle. Retrieved September 21 2021 from https:\/\/www.kaggle.com\/."},{"key":"e_1_3_2_102_2","unstructured":"Figshare LLC. 2023. Figshare. Retrieved January 7 2023 from https:\/\/figshare.com\/."},{"key":"e_1_3_2_103_2","unstructured":"Microsoft Corporation. 2023. GitHub. Retrieved January 7 2023 from https:\/\/github.com\/."},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10009-007-0059-5"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aci.2018.08.003"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.14"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.5555\/1622407.1622416"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.17"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335388"},{"key":"e_1_3_2_110_2","doi-asserted-by":"crossref","unstructured":"Y. Mirsky T. Doitshman Y. Elovici and A. Shabtai. 2018. Kitsune: An ensemble of autoencoders for online network intrusion detection. arXiv:1802.09089. Retrieved from http:\/\/arxiv.org\/abs\/1802.09089.","DOI":"10.14722\/ndss.2018.23204"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2018.03367731"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629648"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-015-9414-4"},{"key":"e_1_3_2_114_2","unstructured":"Microsoft Corporation. 2021. GitHub. Retrieved September 21 2021 from https:\/\/github.com\/."},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/SAM.2015.8"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1109\/SCET.2012.6342082"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110610"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/CSICC52343.2021.9420548"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2022.109562"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1002\/int.22722"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1109\/32.979986"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11219-020-09515-0"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1109\/MTD.2015.7332620"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.JSS.2022.111221"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34706-2_8"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/2915970.2915984"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4614-2113-9"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.53"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453478"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2022.111304"},{"key":"e_1_3_2_132_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.109737"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596908","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3596908","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:47Z","timestamp":1750182527000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3596908"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,13]]},"references-count":131,"journal-issue":{"issue":"13s","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3596908"],"URL":"https:\/\/doi.org\/10.1145\/3596908","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,13]]},"assertion":[{"value":"2022-04-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-05-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}