{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T07:28:20Z","timestamp":1769239700219,"version":"3.49.0"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2009,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Uncovering subtypes of disease from microarray samples has important clinical implications such as survival time and sensitivity of individual patients to specific therapies. Unsupervised clustering methods have been used to classify this type of data. However, most existing methods focus on clusters with compact shapes and do not reflect the geometric complexity of the high dimensional microarray clusters, which limits their performance.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>We present a cluster-number-based ensemble clustering algorithm, called <jats:italic>MULTI-K<\/jats:italic>, for microarray sample classification, which demonstrates remarkable accuracy. The method amalgamates multiple <jats:italic>k<\/jats:italic>-means runs by varying the number of clusters and identifies clusters that manifest the most robust co-memberships of elements. In addition to the original algorithm, we newly devised the <jats:italic>entropy-plot<\/jats:italic> to control the separation of singletons or small clusters. MULTI-K, unlike the simple <jats:italic>k<\/jats:italic>-means or other widely used methods, was able to capture clusters with complex and high-dimensional structures accurately. MULTI-K outperformed other methods including a recently developed ensemble clustering algorithm in tests with five simulated and eight real gene-expression data sets.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>The geometric complexity of clusters should be taken into account for accurate classification of microarray data, and ensemble clustering applied to the number of clusters tackles the problem very well. The C++ code and the data sets tested are available from the authors.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/1471-2105-10-260","type":"journal-article","created":{"date-parts":[[2009,8,22]],"date-time":"2009-08-22T18:13:54Z","timestamp":1250964834000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":57,"title":["MULTI-K: accurate classification of microarray subtypes using ensemble k-means clustering"],"prefix":"10.1186","volume":"10","author":[{"given":"Eun-Youn","family":"Kim","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seon-Young","family":"Kim","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Ashlock","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dougu","family":"Nam","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2009,8,22]]},"reference":[{"issue":"12","key":"2990_CR1","doi-asserted-by":"publisher","first-page":"3273","DOI":"10.1091\/mbc.9.12.3273","volume":"9","author":"PT Spellman","year":"1998","unstructured":"Spellman PT, Sherlock G, Zhang MQ, Iyer VR, Anders K, Eisen MB, Brown PO, Botstein D, Futcher B: Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. Mol Biol Cell 1998, 9(12):3273\u20133297.","journal-title":"Mol Biol Cell"},{"issue":"25","key":"2990_CR2","doi-asserted-by":"publisher","first-page":"14863","DOI":"10.1073\/pnas.95.25.14863","volume":"95","author":"MB Eisen","year":"1998","unstructured":"Eisen MB, Spellman PT, Brown PO, Botstein D: Cluster analysis and display of genome-wide expression patterns. Proc Natl Acad Sci USA 1998, 95(25):14863\u201314868. 10.1073\/pnas.95.25.14863","journal-title":"Proc Natl Acad Sci USA"},{"issue":"11","key":"2990_CR3","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1038\/ncponc0636","volume":"3","author":"F Andre","year":"2006","unstructured":"Andre F, Pusztai L: Molecular classification of breast cancer: implications for selection of adjuvant chemotherapy. Nat Clin Pract Oncol 2006, 3(11):621\u2013632. 10.1038\/ncponc0636","journal-title":"Nat Clin Pract Oncol"},{"issue":"6797","key":"2990_CR4","doi-asserted-by":"publisher","first-page":"747","DOI":"10.1038\/35021093","volume":"406","author":"CM Perou","year":"2000","unstructured":"Perou CM, Sorlie T, Eisen MB, Rijn M, Jeffrey SS, Rees CA, Pollack JR, Ross DT, Johnsen H, Akslen LA, et al.: Molecular portraits of human breast tumours. Nature 2000, 406(6797):747\u2013752. 10.1038\/35021093","journal-title":"Nature"},{"issue":"14","key":"2990_CR5","doi-asserted-by":"publisher","first-page":"8418","DOI":"10.1073\/pnas.0932692100","volume":"100","author":"T Sorlie","year":"2003","unstructured":"Sorlie T, Tibshirani R, Parker J, Hastie T, Marron JS, Nobel A, Deng S, Johnsen H, Pesich R, Geisler S, et al.: Repeated observation of breast tumor subtypes in independent gene expression data sets. Proc Natl Acad Sci USA 2003, 100(14):8418\u20138423. 10.1073\/pnas.0932692100","journal-title":"Proc Natl Acad Sci USA"},{"issue":"5439","key":"2990_CR6","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1126\/science.286.5439.531","volume":"286","author":"TR Golub","year":"1999","unstructured":"Golub TR, Slonim DK, Tamayo P, Huard C, Gaasenbeek M, Mesirov JP, Coller H, Loh ML, Downing JR, Caligiuri MA, et al.: Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 1999, 286(5439):531\u2013537. 10.1126\/science.286.5439.531","journal-title":"Science"},{"issue":"15","key":"2990_CR7","doi-asserted-by":"publisher","first-page":"3201","DOI":"10.1093\/bioinformatics\/bti517","volume":"21","author":"J Handl","year":"2005","unstructured":"Handl J, Knowles J, Kell DB: Computational cluster validation in post-genomic data analysis. Bioinformatics 2005, 21(15):3201\u20133212. 10.1093\/bioinformatics\/bti517","journal-title":"Bioinformatics"},{"issue":"25","key":"2990_CR8","doi-asserted-by":"publisher","first-page":"1937","DOI":"10.1056\/NEJMoa012914","volume":"346","author":"A Rosenwald","year":"2002","unstructured":"Rosenwald A, Wright G, Chan WC, Connors JM, Campo E, Fisher RI, Gascoyne RD, Muller-Hermelink HK, Smeland EB, Giltnane JM, et al.: The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell lymphoma. N Engl J Med 2002, 346(25):1937\u20131947. 10.1056\/NEJMoa012914","journal-title":"N Engl J Med"},{"issue":"7","key":"2990_CR9","doi-asserted-by":"publisher","first-page":"RESEARCH0036","DOI":"10.1186\/gb-2002-3-7-research0036","volume":"3","author":"S Dudoit","year":"2002","unstructured":"Dudoit S, Fridlyand J: A prediction-based resampling method for estimating the number of clusters in a dataset. Genome Biol 2002, 3(7):RESEARCH0036. 10.1186\/gb-2002-3-7-research0036","journal-title":"Genome Biol"},{"key":"2990_CR10","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1007\/BF02294245","volume":"50","author":"G Milligan","year":"1985","unstructured":"Milligan G, Cooper M: An examination of procedures for determining the number of clusters in a data set. Psychometrika 1985, 50: 159\u2013179. 10.1007\/BF02294245","journal-title":"Psychometrika"},{"key":"2990_CR11","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1111\/1467-9868.00293","volume":"63","author":"R Tibshirani","year":"2001","unstructured":"Tibshirani R, Walther G, Hastie T: Estimating the number of clusters in a data set via the gap statistic. J R Stat Soc B 2001, 63: 411\u2013423. 10.1111\/1467-9868.00293","journal-title":"J R Stat Soc B"},{"key":"2990_CR12","first-page":"6","volume":"7","author":"A Ben-Hur","year":"2002","unstructured":"Ben-Hur A, Elisseeff A, Guyon I: A stability based method for discovering structure in clustered data. Pac Symp Biocomput 2002, 7: 6\u201317.","journal-title":"Pac Symp Biocomput"},{"key":"2990_CR13","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1162\/153244303321897735","volume":"3","author":"A Strehl","year":"2002","unstructured":"Strehl A, Ghosh J: Cluster ensembles \u2013 a knowledge reuse framework for combining multiple partitions. J Mach Learn Res 2002, 3: 583\u2013617. 10.1162\/153244303321897735","journal-title":"J Mach Learn Res"},{"key":"2990_CR14","doi-asserted-by":"publisher","first-page":"R94","DOI":"10.1186\/gb-2004-5-11-r94","volume":"5","author":"S Swift","year":"2003","unstructured":"Swift S, Tucker A, Vinciotti V, Orengo C, Liu X, Kellam P: Consensus clustering and functional interpretation of gene-expression data. Genome Biol 2003, 5: R94. 10.1186\/gb-2004-5-11-r94","journal-title":"Genome Biol"},{"key":"2990_CR15","doi-asserted-by":"publisher","first-page":"1462","DOI":"10.1093\/bioinformatics\/18.11.1462","volume":"18","author":"LM Mc Shane","year":"2002","unstructured":"Mc Shane LM: Methods for assessing reproducibility of clustering patterns observed in analyses of microarray data. Bioinformatics 2002, 18: 1462\u20131469. 10.1093\/bioinformatics\/18.11.1462","journal-title":"Bioinformatics"},{"key":"2990_CR16","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1023\/A:1023949509487","volume":"52","author":"S Monti","year":"2003","unstructured":"Monti S, Tamayo P, Mesirov J, Golub T: Consensus clustering: a resampling-based method for class discovery and visualization of gene expression microarray data. Mach Learn 2003, 52: 91\u2013118. 10.1023\/A:1023949509487","journal-title":"Mach Learn"},{"key":"2990_CR17","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1186\/1471-2105-4-36","volume":"4","author":"M Smolkin","year":"2003","unstructured":"Smolkin M, Ghosh D: Cluster stability scores for microarray data in cancer studies. BMC Bioinformatics 2003, 4: 36. 10.1186\/1471-2105-4-36","journal-title":"BMC Bioinformatics"},{"key":"2990_CR18","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1007\/11731177_5","volume":"3931","author":"A Bertoni","year":"2005","unstructured":"Bertoni A, Valentini G: Ensembles based on random projections to improve the accuracy of clustering algorithms. Neural Nets, (WIRN 2005), LNCS 2005, 3931: 31\u201337. full_text","journal-title":"Neural Nets, (WIRN 2005), LNCS"},{"issue":"21","key":"2990_CR19","doi-asserted-by":"publisher","first-page":"2888","DOI":"10.1093\/bioinformatics\/btm463","volume":"23","author":"Z Yu","year":"2007","unstructured":"Yu Z, Wong H-S, Wang H: Graph-based consensus clustering for class discovery from gene expression data. Bioinformatics 2007, 23(21):2888\u20132896. 10.1093\/bioinformatics\/btm463","journal-title":"Bioinformatics"},{"key":"2990_CR20","first-page":"453","volume-title":"ANNIE: 2005","author":"DA Ashlock","year":"2005","unstructured":"Ashlock DA, Kim EY, Guo L: Multi-clustering: avoiding the natural shape of underlying metrics. In ANNIE: 2005. Volume 15. ASME press; 2005:453\u2013461."},{"key":"2990_CR21","first-page":"128","volume":"2","author":"ALN Fred","year":"2003","unstructured":"Fred ALN, Jain AK: Robust data clustering. Proc IEEE CS Conf Computer Vision and Pattern Recognition 2003, 2: 128\u2013133.","journal-title":"Proc IEEE CS Conf Computer Vision and Pattern Recognition"},{"issue":"11","key":"2990_CR22","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2006.226","volume":"28","author":"LI Kuncheva","year":"2006","unstructured":"Kuncheva LI, Vetrov DP: Evaluation of stability of k-means cluster ensembles with respect to random initialization. IEEE Trans Pattern Anal Mach Intell 2006, 28(11):1798\u20131808. 10.1109\/TPAMI.2006.226","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2990_CR23","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","volume":"27","author":"CE Shannon","year":"1948","unstructured":"Shannon CE: A mathematical theory of communication. Bell Syst Tech J 1948, 27: 379\u2013413.","journal-title":"Bell Syst Tech J"},{"issue":"9","key":"2990_CR24","doi-asserted-by":"publisher","first-page":"1194","DOI":"10.1093\/bioinformatics\/18.9.1194","volume":"18","author":"M Medvedovic","year":"2002","unstructured":"Medvedovic M, Sivaganesan S: Bayesian infinite mixture model based clustering of gene expression data. Bioinformatics 2002, 18(9):1194\u20131206. 10.1093\/bioinformatics\/18.9.1194","journal-title":"Bioinformatics"},{"issue":"3","key":"2990_CR25","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1093\/bioinformatics\/18.3.413","volume":"18","author":"GJ McLachlan","year":"2002","unstructured":"McLachlan GJ, Bean RW, Peel D: A mixture model-based approach to the clustering of microarray expression data. Bioinformatics 2002, 18(3):413\u2013422. 10.1093\/bioinformatics\/18.3.413","journal-title":"Bioinformatics"},{"issue":"16","key":"2990_CR26","doi-asserted-by":"publisher","first-page":"1988","DOI":"10.1093\/bioinformatics\/btl284","volume":"22","author":"ZS Qin","year":"2006","unstructured":"Qin ZS: Clustering microarray gene expression data using weighted Chinese restaurant process. Bioinformatics 2006, 22(16):1988\u20131997. 10.1093\/bioinformatics\/btl284","journal-title":"Bioinformatics"},{"key":"2990_CR27","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/BF01908075","volume":"2","author":"L Hubert","year":"1985","unstructured":"Hubert L, Arabie P: Comparing partitions. Journal of classification 1985, 2: 193\u2013218. 10.1007\/BF01908075","journal-title":"Journal of classification"},{"key":"2990_CR28","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1207\/s15327906mbr2104_5","volume":"21","author":"G Milligan","year":"1986","unstructured":"Milligan G, Cooper M: A study of comparability of external criteria for hierarchical cluster analysis. Multivariate Behavioral Research 1986, 21: 441\u2013458. 10.1207\/s15327906mbr2104_5","journal-title":"Multivariate Behavioral Research"},{"key":"2990_CR29","doi-asserted-by":"publisher","first-page":"2319","DOI":"10.1126\/science.290.5500.2319","volume":"290","author":"JB Tenenbaum","year":"2000","unstructured":"Tenenbaum JB, de Silva V, Langford JC: A global geometric framework for nonlinear dimensionality reduction. Science 2000, 290: 2319\u20132322. 10.1126\/science.290.5500.2319","journal-title":"Science"},{"issue":"6769","key":"2990_CR30","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1038\/35000501","volume":"403","author":"AA Alizadeh","year":"2000","unstructured":"Alizadeh AA, Eisen MB, Davis RE, Ma C, Lossos IS, Rosenwald A, Boldrick JC, Sabet H, Tran T, Yu X, et al.: Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature 2000, 403(6769):503\u2013511. 10.1038\/35000501","journal-title":"Nature"},{"issue":"2","key":"2990_CR31","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1016\/S1535-6108(02)00032-6","volume":"1","author":"EJ Yeoh","year":"2002","unstructured":"Yeoh EJ, Ross ME, Shurtleff SA, Williams WK, Patel D, Mahfouz R, Behm FG, Raimondi SC, Relling MV, Patel A, et al.: Classification, subtype discovery, and prediction of outcome in pediatric acute lymphoblastic leukemia by gene expression profiling. Cancer Cell 2002, 1(2):133\u2013143. 10.1016\/S1535-6108(02)00032-6","journal-title":"Cancer Cell"},{"issue":"26","key":"2990_CR32","doi-asserted-by":"publisher","first-page":"15149","DOI":"10.1073\/pnas.211566398","volume":"98","author":"S Ramaswamy","year":"2001","unstructured":"Ramaswamy S, Tamayo P, Rifkin R, Mukherjee S, Yeang CH, Angelo M, Ladd C, Reich M, Latulippe E, Mesirov JP, et al.: Multiclass cancer diagnosis using tumor gene expression signatures. Proc Natl Acad Sci USA 2001, 98(26):15149\u201315154. 10.1073\/pnas.211566398","journal-title":"Proc Natl Acad Sci USA"},{"issue":"52","key":"2990_CR33","doi-asserted-by":"publisher","first-page":"19075","DOI":"10.1073\/pnas.0509603102","volume":"102","author":"H He","year":"2005","unstructured":"He H, Jazdzewski K, Li W, Liyanarachchi S, Nagy R, Volinia S, Calin GA, Liu CG, Franssila K, Suster S, et al.: The role of microRNA genes in papillary thyroid carcinoma. Proc Natl Acad Sci USA 2005, 102(52):19075\u201319080. 10.1073\/pnas.0509603102","journal-title":"Proc Natl Acad Sci USA"},{"issue":"5","key":"2990_CR34","doi-asserted-by":"publisher","first-page":"650","DOI":"10.1093\/bioinformatics\/bti042","volume":"21","author":"I Yanai","year":"2005","unstructured":"Yanai I, Benjamin H, Shmoish M, Chalifa-Caspi V, Shklar M, Ophir R, Bar-Even A, Horn-Saban S, Safran M, Domany E, et al.: Genome-wide midrange transcription profiles reveal expression level relationships in human tissue specification. Bioinformatics 2005, 21(5):650\u2013659. 10.1093\/bioinformatics\/bti042","journal-title":"Bioinformatics"},{"issue":"14","key":"2990_CR35","doi-asserted-by":"publisher","first-page":"8418","DOI":"10.1073\/pnas.0932692100","volume":"100","author":"T Sorlie","year":"2003","unstructured":"Sorlie T, Tibshirani R, Parker J, Hastie T, Marron JS, Nobel A, Deng S, Johnsen H, Pesich R, Geisler S, et al.: Repeated observation of breast tumor subtypes in independent gene expression data sets. Proc Natl Acad Sci USA 2003, 100(14):8418\u20138423. 10.1073\/pnas.0932692100","journal-title":"Proc Natl Acad Sci USA"},{"key":"2990_CR36","doi-asserted-by":"publisher","first-page":"1090","DOI":"10.1093\/bioinformatics\/btg038","volume":"19","author":"S Dudoit","year":"2003","unstructured":"Dudoit S, Fridlyand J: Bagging to improve the accuracy of a clustering procedure. Bioinformatics 2003, 19: 1090\u20131099. 10.1093\/bioinformatics\/btg038","journal-title":"Bioinformatics"},{"key":"2990_CR37","doi-asserted-by":"publisher","first-page":"i205","DOI":"10.1093\/bioinformatics\/btg1027","volume":"19","author":"PL Martelli","year":"2003","unstructured":"Martelli PL, Fariselli P, Casadio R: An ENSEMBLE machine learning approach for the prediction of all-alpha membrane proteins. Bioinformatics 2003, 19: i205-i211. 10.1093\/bioinformatics\/btg1027","journal-title":"Bioinformatics"},{"key":"2990_CR38","doi-asserted-by":"publisher","first-page":"1717","DOI":"10.1093\/bioinformatics\/btl170","volume":"22","author":"H-B Shen","year":"2006","unstructured":"Shen H-B, Chou K-C: Ensemble classifier for protein fold recognition. Bioinformatics 2006, 22: 1717\u20131722. 10.1093\/bioinformatics\/btl170","journal-title":"Bioinformatics"},{"key":"2990_CR39","doi-asserted-by":"publisher","first-page":"3225","DOI":"10.1093\/bioinformatics\/btm514","volume":"23","author":"D Nam","year":"2007","unstructured":"Nam D, Yoon SH, Kim JF: Ensemble learning of genetic networks from time-series expression data. Bioinformatics 2007, 23: 3225\u20133231. 10.1093\/bioinformatics\/btm514","journal-title":"Bioinformatics"},{"key":"2990_CR40","doi-asserted-by":"publisher","first-page":"3114","DOI":"10.1093\/bioinformatics\/bti483","volume":"21","author":"P Qiu","year":"2005","unstructured":"Qiu P, Wang ZJ, Liu KJR: Ensemble dependence model for classification and prediction of cancer and normal gene expression data. Bioinformatics 2005, 21: 3114\u20133121. 10.1093\/bioinformatics\/bti483","journal-title":"Bioinformatics"},{"issue":"9","key":"2990_CR41","doi-asserted-by":"publisher","first-page":"1927","DOI":"10.1093\/bioinformatics\/bti251","volume":"21","author":"DW Kim","year":"2005","unstructured":"Kim DW, Lee KH, Lee D: Detecting clusters of different geometrical shapes in microarray gene expression data. Bioinformatics 2005, 21(9):1927\u20131934. 10.1093\/bioinformatics\/bti251","journal-title":"Bioinformatics"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2105-10-260.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,31]],"date-time":"2021-08-31T21:34:14Z","timestamp":1630445654000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-10-260"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,8,22]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2009,12]]}},"alternative-id":["2990"],"URL":"https:\/\/doi.org\/10.1186\/1471-2105-10-260","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,8,22]]},"assertion":[{"value":"14 February 2009","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 August 2009","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 August 2009","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"260"}}