{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T07:41:04Z","timestamp":1781854864798,"version":"3.54.5"},"reference-count":21,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2016,12,21]],"date-time":"2016-12-21T00:00:00Z","timestamp":1482278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/about_us\/legal\/notices"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61305066"],"award-info":[{"award-number":["61305066"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61561146396"],"award-info":[{"award-number":["61561146396"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,3,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Advancements in next-generation sequencing technology have produced large amounts of reads at low cost in a short time. In metagenomics, 16S and 18S rRNA gene have been widely used as marker genes to profile diversity of microorganisms in environmental samples. Through clustering of sequencing reads we can determine both number of OTUs and their relative abundance. In many applications, clustering of very large sequencing data with high efficiency and accuracy is essential for downstream analysis.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Here, we report a scalable Dirichlet Process Means (DP-means) algorithm for clustering extremely large sequencing data, termed DACE. With an efficient random projection partition strategy for parallel clustering, DACE can cluster billions of sequences within a couple of hours. Experimental results show that DACE runs between 6 and 80 times faster than state-of-the-art programs, while maintaining overall better clustering accuracy. Using 80 cores, DACE clustered the Lake Taihu 16S rRNA gene sequencing data (\u223c316M reads, 30\u2009GB) in 25\u2009min, and the Ocean TARA Eukaryotic 18S rRNA gene sequencing data (\u223c500M reads, 88\u2009GB) into \u223c100 000 clusters within an hour. When applied to the IGC gene catalogs in human gut microbiome (\u223c10M genes), DACE produced 9.8M clusters with 52K redundant genes in 1.5\u2009hours of running time.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and Implementation<\/jats:title><jats:p>DACE is available at https:\/\/github.com\/tinglab\/DACE.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btw722","type":"journal-article","created":{"date-parts":[[2016,11,16]],"date-time":"2016-11-16T20:05:49Z","timestamp":1479326749000},"page":"834-842","source":"Crossref","is-referenced-by-count":13,"title":["DACE: a scalable DP-means algorithm for clustering extremely large sequence data"],"prefix":"10.1093","volume":"33","author":[{"given":"Linhao","family":"Jiang","sequence":"first","affiliation":[{"name":"Bioinformatics Division, Center for Synthetic and Systems Biology, TNLIST, Department of Automation, Tsinghua University, Beijing, China"},{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yichao","family":"Dong","sequence":"additional","affiliation":[{"name":"Program in Computational Biology and Bioinformatics, University of Southern California, Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ning","family":"Chen","sequence":"additional","affiliation":[{"name":"Bioinformatics Division, Center for Synthetic and Systems Biology, TNLIST, Department of Automation, Tsinghua University, Beijing, China"},{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting","family":"Chen","sequence":"additional","affiliation":[{"name":"Bioinformatics Division, Center for Synthetic and Systems Biology, TNLIST, Department of Automation, Tsinghua University, Beijing, China"},{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"},{"name":"Program in Computational Biology and Bioinformatics, University of Southern California, Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2016,12,21]]},"reference":[{"key":"2023020204522622100_btw722-B1","first-page":"gks251.","article-title":"Grinder: a versatile amplicon and shotgun sequence simulator","author":"Angly","year":"2012","journal-title":"Nucleic Acids Res"},{"key":"2023020204522622100_btw722-B2","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1214\/aos\/1176342871","article-title":"Mixture of Dirichlet processes with application to Bayesian nonparametric problems","volume":"2","author":"Antoniak","year":"1974","journal-title":"Ann. Stat"},{"key":"2023020204522622100_btw722-B3","doi-asserted-by":"crossref","first-page":"1621","DOI":"10.1038\/ismej.2012.8","article-title":"Ultra-high-throughput microbial community analysis on the illumina hiseq and miseq platforms","volume":"6","author":"Caporaso","year":"2012","journal-title":"ISME J"},{"key":"2023020204522622100_btw722-B4","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1145\/997817.997857","volume-title":"Proceedings of the Twentieth Annual Symposium on Computational Geometry","author":"Datar","year":"2004"},{"key":"2023020204522622100_btw722-B5","doi-asserted-by":"crossref","first-page":"1261605","DOI":"10.1126\/science.1261605","article-title":"Eukaryotic plankton diversity in the sunlit ocean","volume":"348","author":"De Vargas","year":"2015","journal-title":"Science"},{"key":"2023020204522622100_btw722-B6","doi-asserted-by":"crossref","first-page":"2460","DOI":"10.1093\/bioinformatics\/btq461","article-title":"Search and clustering orders of magnitude faster than BLAST","volume":"26","author":"Edgar","year":"2010","journal-title":"Bioinformatics"},{"key":"2023020204522622100_btw722-B7","doi-asserted-by":"crossref","first-page":"3150","DOI":"10.1093\/bioinformatics\/bts565","article-title":"CD-HIT: accelerated for clustering the next generation sequencing data","volume":"2828","author":"Fu","year":"2012","journal-title":"Bioinformatics"},{"key":"2023020204522622100_btw722-B8","first-page":"518","volume-title":"VLDB","author":"Gionis","year":"1999"},{"key":"2023020204522622100_btw722-B9","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1093\/bioinformatics\/btq725","article-title":"Clustering 16S rRNA for OTU prediction: a method of unsupervised Bayesian clustering","volume":"27","author":"Hao","year":"2011","journal-title":"Bioinformatics"},{"key":"2023020204522622100_btw722-B10","doi-asserted-by":"crossref","first-page":"e545","DOI":"10.7717\/peerj.545","article-title":"Subsampled open-reference clustering creates consistent, comprehensive otu definitions and scales to billions of sequences","volume":"2","author":"JR","year":"2014","journal-title":"PeerJ"},{"key":"2023020204522622100_btw722-B11","volume-title":"Proceedings of 19th International Conference on Machine Learning","author":"Kullis","year":"2012"},{"key":"2023020204522622100_btw722-B12","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1038\/nbt.2942","article-title":"An integrated catalog of reference genes in the human gut microbiome","volume":"32","author":"Li","year":"2014","journal-title":"Nat. Biotechnol"},{"key":"2023020204522622100_btw722-B13","doi-asserted-by":"crossref","first-page":"15488","DOI":"10.1038\/srep15488","article-title":"Annual periodicity in planktonic bacterial and archaeal community composition of eutrophic lake Taihu","volume":"5","author":"Li","year":"2015","journal-title":"Sci. Rep"},{"key":"2023020204522622100_btw722-B14","doi-asserted-by":"crossref","first-page":"e593.","DOI":"10.7717\/peerj.593","article-title":"Swarm: robust and fast clustering method for amplicon-based studies","volume":"2","author":"Mah\u00e9","year":"2014","journal-title":"PeerJ"},{"key":"2023020204522622100_btw722-B15","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1038\/ismej.2011.139","article-title":"An improved Greengenes taxonomy with explicit ranks for ecological and evolutionary analyses of bacteria and archaea","volume":"6","author":"McDonald","year":"2011","journal-title":"ISME J"},{"key":"2023020204522622100_btw722-B16","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1080\/10618600.2000.10474879","article-title":"Markov chain sampling methods for dirichlet process mixture models","volume":"9","author":"Neal","year":"2000","journal-title":"J. Comput. Graph. Stat"},{"key":"2023020204522622100_btw722-B17","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1038\/nbt1485","article-title":"The development and impact of 454 sequencing","volume":"26","author":"Rothberg","year":"2008","journal-title":"Nat. Biotechnol"},{"key":"2023020204522622100_btw722-B18","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.1128\/AEM.71.3.1501-1506.2005","article-title":"Introducing DOTUR, a computer program for defining operational taxonomic units and estimating species richness","volume":"71","author":"Schloss","year":"2005","journal-title":"Appl. Environ. Microbiol"},{"key":"2023020204522622100_btw722-B19","doi-asserted-by":"crossref","first-page":"7537","DOI":"10.1128\/AEM.01541-09","article-title":"Introducing mothur: open-source, platform-independent, community-supported software for describing and comparing microbial communities","volume":"75","author":"Schloss","year":"2009","journal-title":"Appl. Environ. Microbiol"},{"key":"2023020204522622100_btw722-B20","doi-asserted-by":"crossref","first-page":"e76.","DOI":"10.1093\/nar\/gkp285","article-title":"ESPRIT: estimating species richness using large collections of 16s rRNA pyrosequences","volume":"37","author":"Sun","year":"2009","journal-title":"Nucleic Acids Res"},{"key":"2023020204522622100_btw722-B21","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1038\/nature11053","article-title":"Human gut microbiome viewed across age and geography","volume":"486","author":"Yatsunenko","year":"2012","journal-title":"Nature"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/6\/834\/49038545\/bioinformatics_33_6_834.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/33\/6\/834\/49038545\/bioinformatics_33_6_834.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T16:47:04Z","timestamp":1718902024000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/33\/6\/834\/2627421"}},"subtitle":[],"editor":[{"given":"John","family":"Hancock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2016,12,21]]},"references-count":21,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2017,3,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btw722","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2017,3,15]]},"published":{"date-parts":[[2016,12,21]]}}}