{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T04:50:17Z","timestamp":1777697417030,"version":"3.51.4"},"reference-count":43,"publisher":"SAGE Publications","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDT"],"published-print":{"date-parts":[[2022,1,10]]},"abstract":"<jats:p>In short, clustering is the process of partitioning a given set of objects into groups containing highly related instances. This relation is determined by a specific distance metric with which the intra-cluster similarity is estimated. Finding an optimal number of such partitions is usually the key step in the entire process, yet a rather difficult one. Selecting an unsuitable number of clusters might lead to incorrect conclusions and, consequently, to wrong decisions: the term \u201coptimal\u201d is quite ambiguous. Furthermore, various inherent characteristics of the datasets, such as clusters that overlap or clusters containing subclusters, will most often increase the level of difficulty of the task. Thus, the methods used to detect similarities and the parameter selection of the partition algorithm have a major impact on the quality of the groups and the identification of their optimal number. Given that each dataset constitutes a rather distinct case, validity indices are indicators introduced to address the problem of selecting such an optimal number of clusters. In this work, an extensive set of well-known validity indices, based on the approach of the so-called relative criteria, are examined comparatively. A total of 26 cluster validation measures were investigated in two distinct case studies: one in real-world and one in artificially generated data. To ensure a certain degree of difficulty, both real-world and generated data were selected to exhibit variations and inhomogeneity. Each of the indices is being deployed under the schemes of 9 different clustering methods, which incorporate 5 different distance metrics. All results are presented in various explanatory forms.<\/jats:p>","DOI":"10.3233\/idt-210187","type":"journal-article","created":{"date-parts":[[2021,12,7]],"date-time":"2021-12-07T14:07:58Z","timestamp":1638886078000},"page":"809-824","source":"Crossref","is-referenced-by-count":2,"title":["Investigating cluster validation metrics for optimal number of clusters determination"],"prefix":"10.1177","volume":"15","author":[{"given":"Aikaterini","family":"Karanikola","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Charalampos M.","family":"Liapis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sotiris","family":"Kotsiantis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"issue":"1","key":"10.3233\/IDT-210187_ref2","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1145\/568574.568575","article-title":"Why so many clustering algorithms","volume":"4","author":"Estivill-Castro","year":"2002","journal-title":"ACM SIGKDD Explor Newsl"},{"issue":"3","key":"10.3233\/IDT-210187_ref4","doi-asserted-by":"crossref","first-page":"442","DOI":"10.2307\/2344977","article-title":"Pattern classification and scene analysis","volume":"137","author":"Clarke","year":"1974","journal-title":"J R Stat Soc Ser A"},{"issue":"6","key":"10.3233\/IDT-210187_ref7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v061.i06","article-title":"Nbclust: An R package for determining the relevant number of clusters in a data set","volume":"61","author":"Charrad","year":"2014","journal-title":"J Stat Softw"},{"issue":"2","key":"10.3233\/IDT-210187_ref8","first-page":"226","article-title":"Research on k-value selection method of k-means clustering algorithm","volume":"2","author":"Yuan","year":"2019","journal-title":"J"},{"issue":"2-3","key":"10.3233\/IDT-210187_ref9","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1023\/A:1012801612483","article-title":"On clustering validation techniques","volume":"17","author":"Halkidi","year":"2001","journal-title":"J Intell Inf Syst"},{"issue":"3","key":"10.3233\/IDT-210187_ref10","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1016\/j.patcog.2006.06.026","article-title":"Model-based evaluation of clustering validation measures","volume":"40","author":"Brun","year":"2007","journal-title":"Pattern Recognit"},{"issue":"4","key":"10.3233\/IDT-210187_ref13","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1207\/s15327906mbr2104_5","article-title":"A study of the comparability of external criteria for hierarchical cluster analysis","volume":"21","author":"Milligan","year":"1986","journal-title":"Multivariate Behav Res"},{"key":"10.3233\/IDT-210187_ref15","doi-asserted-by":"crossref","unstructured":"H\u00e4m\u00e4l\u00e4inen J, Jauhiainen S, K\u00e4rkk\u00e4inen T. Comparison of internal clustering validation indices for prototype-based clustering. Algorithms. 2017; 10(3).","DOI":"10.3390\/a10030105"},{"issue":"1","key":"10.3233\/IDT-210187_ref16","first-page":"27","article-title":"Internal versus external cluster validation indexes","volume":"5","author":"Rend\u00f3n","year":"2011","journal-title":"Int J Comput Commun [Internet]"},{"key":"10.3233\/IDT-210187_ref17","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.eswa.2019.01.074","article-title":"Estimating the number of clusters in a dataset via consensus clustering","volume":"125","author":"\u00dcnl\u00fc","year":"2019","journal-title":"Expert Syst Appl [Internet]"},{"issue":"9","key":"10.3233\/IDT-210187_ref18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/e19090452","article-title":"An approach for determining the number of clusters in a modelbased cluster analysis","volume":"19","author":"Akogul","year":"2017","journal-title":"Entropy"},{"issue":"6","key":"10.3233\/IDT-210187_ref19","doi-asserted-by":"publisher","first-page":"2251","DOI":"10.1016\/j.patcog.2011.12.017","article-title":"Determining the number of clusters using information entropy for mixed data","volume":"45","author":"Liang","year":"2012","journal-title":"Pattern Recognit [Internet]"},{"key":"10.3233\/IDT-210187_ref20","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.ins.2017.02.010","article-title":"Using the stability of objects to determine the number of clusters in datasets","volume":"393","author":"Lord","year":"2017","journal-title":"Inf Sci (Ny) [Internet]"},{"issue":"2","key":"10.3233\/IDT-210187_ref21","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1007\/s41019-019-0091-y","article-title":"Estimating the optimal number of clusters k in a dataset using data depth","volume":"4","author":"Patil","year":"2019","journal-title":"Data Sci Eng [Internet]"},{"key":"10.3233\/IDT-210187_ref22","doi-asserted-by":"publisher","first-page":"105682","DOI":"10.1016\/j.knosys.2020.105682","article-title":"A meta-learning approach for recommending the number of clusters for clustering algorithms","volume":"195","author":"Pimentel","year":"2020","journal-title":"Knowledge-Based Syst [Internet]"},{"key":"10.3233\/IDT-210187_ref23","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1016\/j.eswa.2016.08.059","article-title":"Determination of the optimal number of clusters using a spectral clustering optimization","volume":"65","author":"Mur","year":"2016","journal-title":"Expert Syst Appl [Internet]"},{"key":"10.3233\/IDT-210187_ref25","first-page":"2825","article-title":"Scikitlearn: Machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"10.3233\/IDT-210187_ref26","first-page":"281","article-title":"Some methods for classification and analysis of MultiVariate observations","volume":"1","author":"MacQueen","year":"1967","journal-title":"Proc fifth Berkeley Symp Math Stat Probab"},{"issue":"1","key":"10.3233\/IDT-210187_ref27","first-page":"100","article-title":"Algorithm AS 136: A k-means clustering algorithm","volume":"28","author":"Hartigan","year":"1979","journal-title":"J R Stat Soc Ser C (Applied Stat [Internet]"},{"key":"10.3233\/IDT-210187_ref29","doi-asserted-by":"crossref","first-page":"282","DOI":"10.4064\/cm-2-3-4-282-285","article-title":"Sur la liaison et la division des points d\u2019un ensemble fini","volume":"2","author":"Florek","year":"1951","journal-title":"Colloq Math"},{"issue":"1","key":"10.3233\/IDT-210187_ref30","first-page":"54","article-title":"Minimum spanning trees and single linkage cluster analysis","volume":"18","author":"Gower","year":"1969","journal-title":"J R Stat Soc Ser C (Applied Stat [Internet]"},{"issue":"3","key":"10.3233\/IDT-210187_ref32","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1007\/BF02852438","article-title":"Estimation of the mean floristic similarity between and within sets of vegetational relev\u00e9s","volume":"1","author":"\u010ce\u0161ka","year":"1966","journal-title":"Folia Geobot Phytotaxon"},{"key":"10.3233\/IDT-210187_ref33","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1093\/comjnl\/20.4.364","article-title":"An efficient algorithm for a complete link method","volume":"20","author":"Defays","year":"1977","journal-title":"Comput J"},{"issue":"4","key":"10.3233\/IDT-210187_ref34","doi-asserted-by":"publisher","first-page":"825","DOI":"10.1177\/001316446602600402","article-title":"Similarity analysis by reciprocal pairs for discrete and continuous data","volume":"26","author":"McQuitty","year":"1966","journal-title":"Educ Psychol Meas [Internet]"},{"issue":"4","key":"10.3233\/IDT-210187_ref35","doi-asserted-by":"crossref","first-page":"623","DOI":"10.2307\/2528417","article-title":"A comparison of some methods of cluster analysis","volume":"23","author":"Gower","year":"1967","journal-title":"Biometrics [Internet]"},{"issue":"1","key":"10.3233\/IDT-210187_ref36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/03610927408827101","article-title":"A dendrite method for cluster analysis","volume":"3","author":"Cali\u0144ski","year":"1974","journal-title":"Commun Stat [Internet]"},{"issue":"3","key":"10.3233\/IDT-210187_ref40","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1007\/BF02293907","article-title":"An examination of the effect of six types of error perturbation on fifteen clustering algorithms","volume":"45","author":"Milligan","year":"1980","journal-title":"Psychometrika"},{"key":"10.3233\/IDT-210187_ref41","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1109\/TPAMI.1979.4766909","article-title":"A cluster separation measure","author":"Davies","year":"1979","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"3","key":"10.3233\/IDT-210187_ref44","first-page":"115","article-title":"Criterion for determining the number of groups in a classification","volume":"10","author":"Ratkowsky","year":"1978","journal-title":"Aust Comput J"},{"issue":"2","key":"10.3233\/IDT-210187_ref45","doi-asserted-by":"crossref","first-page":"387","DOI":"10.2307\/2529003","article-title":"Clustering methods based on likelihood ratio criteria","volume":"27","author":"Scott","year":"2012","journal-title":"Biometrics"},{"issue":"3","key":"10.3233\/IDT-210187_ref46","doi-asserted-by":"crossref","first-page":"501","DOI":"10.2307\/2528592","article-title":"Practical problems in a method of cluster analysis","volume":"27","author":"Marriott","year":"1971","journal-title":"Biometrics"},{"issue":"2","key":"10.3233\/IDT-210187_ref48","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/BF02294245","article-title":"An examination of procedures for determining the number of clusters in a data set","volume":"50","author":"Milligan","year":"1985","journal-title":"Psychometrika"},{"key":"10.3233\/IDT-210187_ref49","doi-asserted-by":"crossref","first-page":"917","DOI":"10.1109\/T-C.1970.222799","article-title":"A criterion and an algorithm for grouping data","author":"Fukunaga","year":"1970","journal-title":"IEEE Trans Comput"},{"issue":"4","key":"10.3233\/IDT-210187_ref51","first-page":"456","article-title":"CLUSTISZ: A program to test for the quality of clustering of a set of objects","volume":"12","author":"McClain","year":"1975","journal-title":"J Mark Res [Internet]"},{"issue":"1","key":"10.3233\/IDT-210187_ref52","doi-asserted-by":"crossref","first-page":"23","DOI":"10.2307\/2531893","article-title":"A criterion for determining the number of groups in a data set using sum-of-squares clustering","volume":"44","author":"Krzanowski","year":"1988","journal-title":"Biometrics"},{"key":"10.3233\/IDT-210187_ref53","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/0377-0427(87)90125-7","article-title":"Silhouettes: A graphical aid to the interpretation and validation of cluster analysis","volume":"20","author":"Rousseeuw","year":"1987","journal-title":"J Comput Appl Math [Internet]"},{"issue":"1","key":"10.3233\/IDT-210187_ref55","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1080\/01969727408546059","article-title":"Well-separated clusters and optimal fuzzy partitions","volume":"4","author":"Dunn","year":"1974","journal-title":"J Cybern [Internet]"},{"issue":"1","key":"10.3233\/IDT-210187_ref56","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/BF01908075","article-title":"Comparing partitions","volume":"2","author":"Hubert","year":"1985","journal-title":"J Classif [Internet]"},{"issue":"3","key":"10.3233\/IDT-210187_ref57","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1145\/601858.601862","article-title":"Clustering validity checking methods: Part II","volume":"31","author":"Halkidi","year":"2002","journal-title":"SIGMOD Rec"},{"issue":"200","key":"10.3233\/IDT-210187_ref59","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1080\/01621459.1937.10503522","article-title":"The use of ranks to avoid the assumption of normality implicit in the analysis of variance","volume":"32","author":"Friedman","year":"1937","journal-title":"J Am Stat Assoc"},{"issue":"293","key":"10.3233\/IDT-210187_ref60","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1080\/01621459.1961.10482090","article-title":"Multiple comparisons among means","volume":"56","author":"Dunn","year":"1961","journal-title":"J Am Stat Assoc"},{"issue":"3","key":"10.3233\/IDT-210187_ref62","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1142\/S0218001411008683","article-title":"A survey of clustering ensemble algorithms","volume":"25","author":"Vega-Pons","year":"2011","journal-title":"Int J Pattern Recognit Artif Intell"},{"issue":"2","key":"10.3233\/IDT-210187_ref64","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1007\/s11704-019-8208-z","article-title":"A survey on ensemble learning","volume":"14","author":"Dong","year":"2020","journal-title":"Front Comput Sci [Internet]"}],"container-title":["Intelligent Decision Technologies"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDT-210187","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:23:00Z","timestamp":1777454580000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDT-210187"}},"subtitle":[],"editor":[{"given":"George A.","family":"Tsihrintzis","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Maria","family":"Virvou","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]},{"given":"Ioannis","family":"Hatzilygeroudis","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2022,1,10]]},"references-count":43,"journal-issue":{"issue":"4"},"URL":"https:\/\/doi.org\/10.3233\/idt-210187","relation":{},"ISSN":["1872-4981","1875-8843"],"issn-type":[{"value":"1872-4981","type":"print"},{"value":"1875-8843","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,10]]}}}