{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T13:09:38Z","timestamp":1784120978967,"version":"3.55.0"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2018,9,8]],"date-time":"2018-09-08T00:00:00Z","timestamp":1536364800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01HG006292"],"award-info":[{"award-number":["R01HG006292"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01HL129132"],"award-info":[{"award-number":["R01HL129132"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["P01CA142538"],"award-info":[{"award-number":["P01CA142538"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,4,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>Accurately clustering cell types from a mass of heterogeneous cells is a crucial first step for the analysis of single-cell RNA-seq (scRNA-Seq) data. Although several methods have been recently developed, they utilize different characteristics of data and yield varying results in terms of both the number of clusters and actual cluster assignments.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Here, we present SAFE-clustering, single-cell aggregated (From Ensemble) clustering, a flexible, accurate and robust method for clustering scRNA-Seq data. SAFE-clustering takes as input, results from multiple clustering methods, to build one consensus solution. SAFE-clustering currently embeds four state-of-the-art methods, SC3, CIDR, Seurat and t-SNE + k-means; and ensembles solutions from these four methods using three hypergraph-based partitioning algorithms. Extensive assessment across 12 datasets with the number of clusters ranging from 3 to 14, and the number of single cells ranging from 49 to 32, 695 showcases the advantages of SAFE-clustering in terms of both cluster number (18.2\u201358.1% reduction in absolute deviation to the truth) and cluster assignment (on average 36.0% improvement, and up to 18.5% over the best of the four methods, measured by adjusted rand index). Moreover, SAFE-clustering is computationally efficient to accommodate large datasets, taking &amp;lt;10\u2009min to process 28 733 cells.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>SAFEclustering, including source codes and tutorial, is freely available at https:\/\/github.com\/yycunc\/SAFEclustering.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/bty793","type":"journal-article","created":{"date-parts":[[2018,9,6]],"date-time":"2018-09-06T15:28:19Z","timestamp":1536247699000},"page":"1269-1277","source":"Crossref","is-referenced-by-count":111,"title":["SAFE-clustering: Single-cell Aggregated (from Ensemble) clustering for single-cell RNA-seq data"],"prefix":"10.1093","volume":"35","author":[{"given":"Yuchen","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruth","family":"Huh","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Houston W","family":"Culpepper","sequence":"additional","affiliation":[{"name":"Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuan","family":"Lin","sequence":"additional","affiliation":[{"name":"Center for Bioinformatics, School of Life Sciences, Peking University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8401-0545","authenticated-orcid":false,"given":"Michael I","family":"Love","sequence":"additional","affiliation":[{"name":"Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"},{"name":"Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yun","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Genetics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"},{"name":"Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,9,8]]},"reference":[{"key":"2023012808150926500_bty793-B1","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1038\/ni.2842","article-title":"Early specification of CD8+ T lymphocyte fates during adaptive immunity revealed by single-cell gene-expression analyses","volume":"15","author":"Arsenio","year":"2014","journal-title":"Nat. Immunol"},{"key":"2023012808150926500_bty793-B2","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1016\/j.cels.2016.08.011","article-title":"A single-cell transcriptomic map of the human and mouse pancreas reveals inter-and intra-cell population structure","volume":"3","author":"Baron","year":"2016","journal-title":"Cell Syst"},{"key":"2023012808150926500_bty793-B3","doi-asserted-by":"crossref","first-page":"1787","DOI":"10.1101\/gr.177725.114","article-title":"Cell fate inclination within 2-cell and 4-cell mouse embryos revealed by single-cell RNA sequencing","volume":"24","author":"Biase","year":"2014","journal-title":"Genome Res"},{"key":"2023012808150926500_bty793-B4","doi-asserted-by":"crossref","first-page":"1209","DOI":"10.1016\/j.cell.2012.08.023","article-title":"Single-cell expression analyses during cellular reprogramming reveal an early stochastic and a late hierarchic phase","volume":"150","author":"Buganim","year":"2012","journal-title":"Cell"},{"key":"2023012808150926500_bty793-B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/03610927408827101","article-title":"A dendrite method for cluster analysis","volume":"3","author":"Calinski","year":"1974","journal-title":"Commun. Stat. Methods"},{"key":"2023012808150926500_bty793-B6","doi-asserted-by":"crossref","first-page":"7285","DOI":"10.1073\/pnas.1507125112","article-title":"A survey of human brain transcriptome diversity at the single cell level","volume":"112","author":"Darmanis","year":"2015","journal-title":"Proc. Natl. Acad. Sci"},{"key":"2023012808150926500_bty793-B7","article-title":"Cluster headache: comparing clustering tools for 10X single cell sequencing data","author":"Freytag","year":"2017","journal-title":"bioRxiv"},{"key":"2023012808150926500_bty793-B8","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1002\/widm.32","article-title":"Cluster ensembles","volume":"1","author":"Ghosh","year":"2011","journal-title":"WIREs Data Mining Knowl. Discov"},{"key":"2023012808150926500_bty793-B9","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1038\/nature14966","article-title":"Single-cell messenger RNA sequencing reveals rare intestinal cell types","volume":"525","author":"Gr\u00fcn","year":"2015","journal-title":"Nature"},{"key":"2023012808150926500_bty793-B10","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/BF01908075","article-title":"Comparing partitions","volume":"2","author":"Hubert","year":"1985","journal-title":"J. Classif"},{"key":"2023012808150926500_bty793-B11","doi-asserted-by":"crossref","first-page":"18914","DOI":"10.1073\/pnas.0502181102","article-title":"High-resolution protein folding with a transferable potential","volume":"102","author":"Hubner","year":"2005","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023012808150926500_bty793-B12","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1126\/science.1247651","article-title":"Massively parallel single-cell RNA-seq for marker-free decomposition of tissues into cell types","volume":"343","author":"Jaitin","year":"2014","journal-title":"Science"},{"key":"2023012808150926500_bty793-B13","doi-asserted-by":"crossref","first-page":"10978","DOI":"10.1093\/nar\/gkx754","article-title":"Accounting for technical noise in differential expression analysis of single-cell RNA sequencing data","volume":"45","author":"Jia","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023012808150926500_bty793-B14","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1038\/nmeth0411-311","article-title":"Single-cell genomics","volume":"8","author":"Kalisky","year":"2011","journal-title":"Nat. Methods"},{"key":"2023012808150926500_bty793-B15","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1109\/92.748202","article-title":"Multilevel hypergraph partitioning: applications in VLSI domain","volume":"7","author":"Karypis","year":"1999","journal-title":"IEEE Trans. Very Large Scale Integr. Syst"},{"key":"2023012808150926500_bty793-B16","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1137\/S1064827595287997","article-title":"A fast and high quality multilevel scheme for partitioning irregular graphs","volume":"20","author":"Karypis","year":"1998","journal-title":"SIAM J. Sci. Comput"},{"key":"2023012808150926500_bty793-B17","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1038\/nmeth.4236","article-title":"SC3: consensus clustering of single-cell RNA-seq data","volume":"14","author":"Kiselev","year":"2017","journal-title":"Nat. Methods"},{"key":"2023012808150926500_bty793-B18","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1038\/nbt1330","article-title":"Ensemble modeling for analysis of cell signaling dynamics","volume":"25","author":"Kuepfer","year":"2007","journal-title":"Nat. Biotechnol"},{"key":"2023012808150926500_bty793-B19","doi-asserted-by":"crossref","first-page":"59.","DOI":"10.1186\/s13059-017-1188-0","article-title":"CIDR: ultrafast and accurate clustering through imputation for single-cell RNA-seq data","volume":"18","author":"Lin","year":"2017","journal-title":"Genome Biol"},{"key":"2023012808150926500_bty793-B20","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1016\/j.cell.2015.05.002","article-title":"Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets","volume":"161","author":"Macosko","year":"2015","journal-title":"Cell"},{"key":"2023012808150926500_bty793-B21","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1016\/j.celrep.2014.04.011","article-title":"Single-cell RNA sequencing reveals T helper cells synthesizing steroids de novo to contribute to immune homeostasis","volume":"7","author":"Mahata","year":"2014","journal-title":"Cell Rep"},{"key":"2023012808150926500_bty793-B22","doi-asserted-by":"crossref","first-page":"e190.","DOI":"10.1371\/journal.pgen.0020190","article-title":"Population structure and eigenanalysis","volume":"2","author":"Patterson","year":"2006","journal-title":"PLoS Genet"},{"key":"2023012808150926500_bty793-B23","doi-asserted-by":"crossref","first-page":"780","DOI":"10.1080\/08839510802170546","article-title":"Consensus-based ensembles of soft clusterings","volume":"22","author":"Punera","year":"2008","journal-title":"Appl. Artif. Intell"},{"key":"2023012808150926500_bty793-B24","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1038\/550451a","article-title":"The human cell atlas: from vision to reality","volume":"550","author":"Rozenblatt-Rosen","year":"2017","journal-title":"Nature"},{"key":"2023012808150926500_bty793-B25","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1038\/nbt.3192","article-title":"Spatial reconstruction of single-cell gene expression data","volume":"33","author":"Satija","year":"2015","journal-title":"Nat. Biotechnol"},{"key":"2023012808150926500_bty793-B26","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1038\/nature12172","article-title":"Single-cell transcriptomics reveals bimodality in expression and splicing in immune cells","volume":"498","author":"Shalek","year":"2013","journal-title":"Nature"},{"key":"2023012808150926500_bty793-B27","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1016\/j.stem.2015.07.013","article-title":"Single-cell RNA-seq with waterfall reveals molecular cascades underlying adult neurogenesis","volume":"17","author":"Shin","year":"2015","journal-title":"Cell Stem Cell"},{"key":"2023012808150926500_bty793-B28","first-page":"93","volume-title":"Aaai\/Iaai","author":"Strehl","year":"2002"},{"key":"2023012808150926500_bty793-B29","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1093\/bioinformatics\/btx490","article-title":"DIMM-SC: a Dirichlet mixture model for clustering droplet-based single cell transcriptomic data","volume":"34","author":"Sun","year":"2018","journal-title":"Bioinformatics"},{"key":"2023012808150926500_bty793-B30","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1016\/j.stem.2010.03.015","article-title":"Tracing the derivation of embryonic stem cells from the inner cell mass by single-cell RNA-Seq analysis","volume":"6","author":"Tang","year":"2010","journal-title":"Cell Stem Cell"},{"key":"2023012808150926500_bty793-B31","doi-asserted-by":"crossref","first-page":"1905","DOI":"10.1016\/j.celrep.2014.08.029","article-title":"Single-cell RNA sequencing identifies extracellular matrix gene expression by pancreatic circulating tumor cells","volume":"8","author":"Ting","year":"2014","journal-title":"Cell Rep"},{"key":"2023012808150926500_bty793-B32","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/BF02100489","article-title":"Level-spacing distributions and the Airy kernel","volume":"159","author":"Tracy","year":"1994","journal-title":"Commun. Math. Phys"},{"key":"2023012808150926500_bty793-B33","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1038\/nbt.2859","article-title":"The dynamics and regulators of cell fate decisions are revealed by pseudotemporal ordering of single cells","volume":"32","author":"Trapnell","year":"2014","journal-title":"Nat. Biotechnol"},{"key":"2023012808150926500_bty793-B34","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1038\/nature13173","article-title":"Reconstructing lineage hierarchies of the distal lung epithelium using single-cell RNA-seq","volume":"509","author":"Treutlein","year":"2014","journal-title":"Nature"},{"key":"2023012808150926500_bty793-B35","first-page":"2579","article-title":"Visualizing data using t-sne","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J. Mach. Learn. Res"},{"key":"2023012808150926500_bty793-B36","doi-asserted-by":"crossref","first-page":"471.","DOI":"10.1140\/epjb\/e2013-40829-0","article-title":"A smart local moving algorithm for large-scale modularity-based community detection","volume":"86","author":"Waltman","year":"2013","journal-title":"Eur. Phys. J. B"},{"key":"2023012808150926500_bty793-B37","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1038\/nmeth.4207","article-title":"Visualization and analysis of single-cell RNA-seq data by kernel-based similarity learning","volume":"14","author":"Wang","year":"2017","journal-title":"Nat. Methods"},{"key":"2023012808150926500_bty793-B38","doi-asserted-by":"crossref","first-page":"2800","DOI":"10.1177\/0962280215609948","article-title":"Fast clustering using adaptive density peak detection","volume":"26","author":"Wang","year":"2017","journal-title":"Stat. Methods Med. Res"},{"key":"2023012808150926500_bty793-B39","doi-asserted-by":"crossref","first-page":"15.","DOI":"10.1186\/s13059-017-1382-0","article-title":"SCANPY: large-scale single-cell gene expression data analysis","volume":"19","author":"Wolf","year":"2018","journal-title":"Genome Biol"},{"key":"2023012808150926500_bty793-B40","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1038\/nsmb.2660","article-title":"Single-cell RNA-Seq profiling of human preimplantation embryos and embryonic stem cells","volume":"20","author":"Yan","year":"2013","journal-title":"Nat. Struct. Mol. Biol"},{"key":"2023012808150926500_bty793-B41","doi-asserted-by":"crossref","first-page":"1138","DOI":"10.1126\/science.aaa1934","article-title":"Cell types in the mouse cortex and hippocampus revealed by single-cell RNA-seq","volume":"347","author":"Zeisel","year":"2015","journal-title":"Science"},{"key":"2023012808150926500_bty793-B42","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1186\/s12859-018-2092-7","article-title":"An interpretable framework for clustering single-cell RNA-Seq datasets","volume":"19","author":"Zhang","year":"2018","journal-title":"BMC Bioinformatics"},{"key":"2023012808150926500_bty793-B43","doi-asserted-by":"crossref","first-page":"14049.","DOI":"10.1038\/ncomms14049","article-title":"Massively parallel digital transcriptional profiling of single cells","volume":"8","author":"Zheng","year":"2017","journal-title":"Nat. Commun"},{"key":"2023012808150926500_bty793-B44","first-page":"206532","article-title":"A unified statistical framework for single cell and bulk RNA sequencing data","author":"Zhu","year":"2017","journal-title":"bioRxiv"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/8\/1269\/48941337\/bioinformatics_35_8_1269.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/8\/1269\/48941337\/bioinformatics_35_8_1269.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,28]],"date-time":"2023-01-28T03:38:40Z","timestamp":1674877120000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/8\/1269\/5092931"}},"subtitle":[],"editor":[{"given":"Inanc","family":"Birol","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2018,9,8]]},"references-count":44,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2019,4,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/bty793","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/215723","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,4,15]]},"published":{"date-parts":[[2018,9,8]]}}}