{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T13:46:51Z","timestamp":1770817611986,"version":"3.50.1"},"reference-count":78,"publisher":"World Scientific Pub Co Pte Ltd","issue":"04","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61772367"],"award-info":[{"award-number":["61772367"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"crossref","award":["2016YFC0901704"],"award-info":[{"award-number":["2016YFC0901704"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Bioinform. Comput. Biol."],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:p> Clustering analysis has been widely applied to single-cell RNA-sequencing (scRNA-seq) data to discover cell types and cell states. Algorithms developed in recent years have greatly helped the understanding of cellular heterogeneity and the underlying mechanisms of biological processes. However, these algorithms often use different techniques, were evaluated on different datasets and compared with some of their counterparts usually using different performance metrics. Consequently, there lacks an accurate and complete picture of their merits and demerits, which makes it difficult for users to select proper algorithms for analyzing their data. To fill this gap, we first do a review on the major existing scRNA-seq data clustering methods, and then conduct a comprehensive performance comparison among them from multiple perspectives. We consider 13 state of the art scRNA-seq data clustering algorithms, and collect 12 publicly available real scRNA-seq datasets from the existing works to evaluate and compare these algorithms. Our comparative study shows that the existing methods are very diverse in performance. Even the top-performance algorithms do not perform well on all datasets, especially those with complex structures. This suggests that further research is required to explore more stable, accurate, and efficient clustering algorithms for scRNA-seq data. <\/jats:p>","DOI":"10.1142\/s0219720020400053","type":"journal-article","created":{"date-parts":[[2020,8,14]],"date-time":"2020-08-14T14:44:52Z","timestamp":1597416292000},"page":"2040005","source":"Crossref","is-referenced-by-count":24,"title":["Single-cell RNA-seq data clustering: A survey with performance comparison study"],"prefix":"10.1142","volume":"18","author":[{"given":"Ruiyi","family":"Li","sequence":"first","affiliation":[{"name":"Department of Computer Science and Technology, Tongji University, 4800 Caoan Road, Shanghai, P.\u00a0R.\u00a0China"}]},{"given":"Jihong","family":"Guan","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tongji University, 4800 Caoan Road, Shanghai, P.\u00a0R.\u00a0China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1949-2768","authenticated-orcid":false,"given":"Shuigeng","family":"Zhou","sequence":"additional","affiliation":[{"name":"Shanghai Key Lab of Intelligent Information Processing and School of Computer Science, Fudan University, 220 Handan Road, Shanghai, P.\u00a0R.\u00a0China"}]}],"member":"219","published-online":{"date-parts":[[2020,8,14]]},"reference":[{"key":"S0219720020400053BIB001","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-016-0927-y"},{"key":"S0219720020400053BIB002","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gku555"},{"key":"S0219720020400053BIB003","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10798-1_3"},{"key":"S0219720020400053BIB004","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa1934"},{"key":"S0219720020400053BIB005","doi-asserted-by":"publisher","DOI":"10.1126\/science.1245316"},{"key":"S0219720020400053BIB006","doi-asserted-by":"publisher","DOI":"10.1126\/science.1254257"},{"key":"S0219720020400053BIB007","doi-asserted-by":"publisher","DOI":"10.1038\/nrn2402"},{"key":"S0219720020400053BIB008","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2008.01.002"},{"key":"S0219720020400053BIB009","doi-asserted-by":"publisher","DOI":"10.2307\/2346830"},{"key":"S0219720020400053BIB010","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289588"},{"key":"S0219720020400053BIB011","first-page":"226","volume-title":"Proceedings of the Second International Conference on Knowledge Discovery and Data Mining","author":"Ester M","year":"1996"},{"key":"S0219720020400053BIB012","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.2967"},{"key":"S0219720020400053BIB013","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.2772"},{"key":"S0219720020400053BIB014","doi-asserted-by":"publisher","DOI":"10.1186\/gb-2010-11-12-220"},{"key":"S0219720020400053BIB015","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004333"},{"key":"S0219720020400053BIB016","doi-asserted-by":"publisher","DOI":"10.1101\/gr.161034.113"},{"key":"S0219720020400053BIB017","first-page":"1070","volume-title":"Int Conf Machine Learning","author":"Prabhakaran S","year":"2016"},{"key":"S0219720020400053BIB018","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btx490"},{"key":"S0219720020400053BIB019","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.4236"},{"key":"S0219720020400053BIB020","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-017-1188-0"},{"key":"S0219720020400053BIB021","doi-asserted-by":"publisher","DOI":"10.1038\/nature14966"},{"key":"S0219720020400053BIB022","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-016-1010-4"},{"key":"S0219720020400053BIB023","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-016-1175-6"},{"key":"S0219720020400053BIB024","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-016-0984-y"},{"key":"S0219720020400053BIB025","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btw607"},{"key":"S0219720020400053BIB026","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btv088"},{"key":"S0219720020400053BIB027","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.3192"},{"key":"S0219720020400053BIB028","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.4096"},{"key":"S0219720020400053BIB029","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004575"},{"key":"S0219720020400053BIB030","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRev.72.26"},{"key":"S0219720020400053BIB031","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.2930"},{"key":"S0219720020400053BIB032","doi-asserted-by":"publisher","DOI":"10.2307\/1937992"},{"key":"S0219720020400053BIB033","doi-asserted-by":"publisher","DOI":"10.1002\/sim.5840"},{"issue":"2","key":"S0219720020400053BIB034","first-page":"105","volume":"26","author":"Feng C","year":"2014","journal-title":"Shanghai Arch Psychiatry"},{"key":"S0219720020400053BIB035","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-015-0805-z"},{"key":"S0219720020400053BIB036","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-017-02554-5"},{"key":"S0219720020400053BIB037","doi-asserted-by":"publisher","DOI":"10.1016\/0169-7439(87)80084-9"},{"key":"S0219720020400053BIB038","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2008.4587670"},{"key":"S0219720020400053BIB039","first-page":"2579","volume":"9","author":"van der Maaten L","year":"2008","journal-title":"J Mach Learn Res"},{"key":"S0219720020400053BIB040","doi-asserted-by":"publisher","DOI":"10.1126\/science.290.5500.2323"},{"key":"S0219720020400053BIB041","doi-asserted-by":"publisher","DOI":"10.1126\/science.290.5500.2319"},{"key":"S0219720020400053BIB042","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkp248"},{"key":"S0219720020400053BIB043","doi-asserted-by":"publisher","DOI":"10.1140\/epjb\/e2013-40829-0"},{"key":"S0219720020400053BIB044","doi-asserted-by":"publisher","DOI":"10.1198\/016214506000000302"},{"key":"S0219720020400053BIB045","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2007.19.10.2756"},{"key":"S0219720020400053BIB046","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2007.09.010"},{"key":"S0219720020400053BIB047","doi-asserted-by":"publisher","DOI":"10.1101\/gr.177725.114"},{"key":"S0219720020400053BIB048","doi-asserted-by":"publisher","DOI":"10.1038\/nsmb.2660"},{"key":"S0219720020400053BIB049","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2016.01.047"},{"key":"S0219720020400053BIB050","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.2967"},{"key":"S0219720020400053BIB051","doi-asserted-by":"publisher","DOI":"10.1016\/j.stem.2015.09.011"},{"key":"S0219720020400053BIB052","doi-asserted-by":"publisher","DOI":"10.1038\/nature13173"},{"key":"S0219720020400053BIB053","doi-asserted-by":"publisher","DOI":"10.1016\/j.celrep.2014.08.029"},{"key":"S0219720020400053BIB054","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3881"},{"key":"S0219720020400053BIB055","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2015.04.044"},{"key":"S0219720020400053BIB056","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.2282"},{"key":"S0219720020400053BIB057","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.1315"},{"key":"S0219720020400053BIB058","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.2639"},{"key":"S0219720020400053BIB059","doi-asserted-by":"publisher","DOI":"10.1101\/gr.110882.110"},{"key":"S0219720020400053BIB060","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.1621"},{"key":"S0219720020400053BIB061","doi-asserted-by":"publisher","DOI":"10.1007\/BF01908075"},{"key":"S0219720020400053BIB062","first-page":"2837","volume":"11","author":"Vinh NX","year":"2010","journal-title":"J Mach Learn Res"},{"key":"S0219720020400053BIB063","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1983.10478008"},{"key":"S0219720020400053BIB064","first-page":"410","volume-title":"Proc 2007 Joint Conf Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL)","author":"Rosenberg A","year":"2007"},{"key":"S0219720020400053BIB065","doi-asserted-by":"publisher","DOI":"10.1016\/0377-0427(87)90125-7"},{"key":"S0219720020400053BIB066","doi-asserted-by":"publisher","DOI":"10.1080\/03610927408827101"},{"key":"S0219720020400053BIB067","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1979.4766909"},{"key":"S0219720020400053BIB068","first-page":"361","volume-title":"Int Conf Web Social Media","volume":"8","author":"Bastian M","year":"2009"},{"key":"S0219720020400053BIB069","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0098679"},{"key":"S0219720020400053BIB070","doi-asserted-by":"publisher","DOI":"10.1111\/1467-9868.00293"},{"key":"S0219720020400053BIB071","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-017-1305-0"},{"key":"S0219720020400053BIB072","doi-asserted-by":"publisher","DOI":"10.1038\/nrg3833"},{"key":"S0219720020400053BIB073","doi-asserted-by":"publisher","DOI":"10.1038\/nmeth.4292"},{"key":"S0219720020400053BIB074","doi-asserted-by":"publisher","DOI":"10.12688\/f1000research.15666.2"},{"key":"S0219720020400053BIB075","doi-asserted-by":"publisher","DOI":"10.1016\/j.mam.2017.07.002"},{"key":"S0219720020400053BIB076","first-page":"e27041","volume":"6","author":"Regev A","year":"2017","journal-title":"Life"},{"key":"S0219720020400053BIB077","doi-asserted-by":"publisher","DOI":"10.1038\/nbt.4091"},{"key":"S0219720020400053BIB078","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2019.05.031"}],"container-title":["Journal of Bioinformatics and Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0219720020400053","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,9,7]],"date-time":"2020-09-07T06:52:19Z","timestamp":1599461539000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0219720020400053"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":78,"journal-issue":{"issue":"04","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.1142\/S0219720020400053"],"URL":"https:\/\/doi.org\/10.1142\/s0219720020400053","relation":{},"ISSN":["0219-7200","1757-6334"],"issn-type":[{"value":"0219-7200","type":"print"},{"value":"1757-6334","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8]]}}}