{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T17:42:41Z","timestamp":1780076561968,"version":"3.54.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009086","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,7,13]],"date-time":"2021-07-13T00:00:00Z","timestamp":1626134400000}}],"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2021,6,30]],"date-time":"2021-06-30T00:00:00Z","timestamp":1625011200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012390","name":"SystemsX.ch","doi-asserted-by":"publisher","award":["HDL-X"],"award-info":[{"award-number":["HDL-X"]}],"id":[{"id":"10.13039\/501100012390","id-type":"DOI","asserted-by":"publisher"}]},{"name":"ERASysApp","award":["Rootbook"],"award-info":[{"award-number":["Rootbook"]}]},{"name":"PHRT","award":["2017-103"],"award-info":[{"award-number":["2017-103"]}]},{"name":"Swiss Data Science Center"},{"name":"PHRT","award":["2017-110"],"award-info":[{"award-number":["2017-110"]}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Clustering high-dimensional data, such as images or biological measurements, is a long-standing problem and has been studied extensively. Recently, Deep Clustering has gained popularity due to its flexibility in fitting the specific peculiarities of complex data. Here we introduce the Mixture-of-Experts Similarity Variational Autoencoder (MoE-Sim-VAE), a novel generative clustering model. The model can learn multi-modal distributions of high-dimensional data and use these to generate realistic data with high efficacy and efficiency. MoE-Sim-VAE is based on a Variational Autoencoder (VAE), where the decoder consists of a Mixture-of-Experts (MoE) architecture. This specific architecture allows for various modes of the data to be automatically learned by means of the experts. Additionally, we encourage the lower dimensional latent representation of our model to follow a Gaussian mixture distribution and to accurately represent the similarities between the data points. We assess the performance of our model on the MNIST benchmark data set and challenging real-world tasks of clustering mouse organs from single-cell RNA-sequencing measurements and defining cell subpopulations from mass cytometry (CyTOF) measurements on hundreds of different datasets. MoE-Sim-VAE exhibits superior clustering performance on all these tasks in comparison to the baselines as well as competitor methods.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009086","type":"journal-article","created":{"date-parts":[[2021,6,30]],"date-time":"2021-06-30T17:51:48Z","timestamp":1625075508000},"page":"e1009086","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":37,"title":["Mixture-of-Experts Variational Autoencoder for clustering and generating from similarity-based representations on single cell data"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5323-827X","authenticated-orcid":true,"given":"Andreas","family":"Kopf","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0640-2671","authenticated-orcid":true,"given":"Vincent","family":"Fortuin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3894-646X","authenticated-orcid":true,"given":"Vignesh Ram","family":"Somnath","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4583-9083","authenticated-orcid":true,"given":"Manfred","family":"Claassen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2021,6,30]]},"reference":[{"key":"pcbi.1009086.ref001","unstructured":"Aljalbout E., Golkov V., Siddiqui Y., Strobel M., Cremers D. Clustering with Deep Learning: Taxonomy and New Methods. arXiv, 2018."},{"key":"pcbi.1009086.ref002","doi-asserted-by":"crossref","unstructured":"Min E., Guo X., Liu Q., Zhang G., Cui J., Long J. A Survey of Clustering With Deep Learning: From the Perspective of Network Architecture. IEEE, 2018.","DOI":"10.1109\/ACCESS.2018.2855437"},{"key":"pcbi.1009086.ref003","unstructured":"Zhang D., Sun Y., Eriksson B., Balzano L. Deep Unsupervised Clustering Using Mixture of Autoencoders. arXiv, 2017."},{"key":"pcbi.1009086.ref004","doi-asserted-by":"crossref","unstructured":"Dizaji K. G., Herandi A., Deng C., Cai W., Huang H. Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization. arXiv, 2017.","DOI":"10.1109\/ICCV.2017.612"},{"key":"pcbi.1009086.ref005","unstructured":"Yang B., Fu X., Sidiropoulos N. D., Hong M. Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering. arXiv, 2017."},{"key":"pcbi.1009086.ref006","unstructured":"Fortuin V., H\u00fcser M., Locatello F., Strathmann H., R\u00e4tsch G. SOM-VAE: Interpretable Discrete Representation Learning on Time Series. Conference paper at ICLR, 2019."},{"key":"pcbi.1009086.ref007","doi-asserted-by":"crossref","unstructured":"Jiang Z., Zheng Y., Tan H., Tang B., Zhou H. Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering. arXiv, 2017.","DOI":"10.24963\/ijcai.2017\/273"},{"key":"pcbi.1009086.ref008","article-title":"Clustering Techniques and the Similarity Measures used in Clustering: A Survey","author":"J. Irani","year":"2016","journal-title":"International Journal of Computer Applications"},{"key":"pcbi.1009086.ref009","unstructured":"Chopra S., Hadsell R., LeCun Y. Learning a similarity metric discriminatively, with application to face verification. IEEE, 2005."},{"key":"pcbi.1009086.ref010","doi-asserted-by":"crossref","unstructured":"McInnes L., Healy J., Melville J. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv, 2018.","DOI":"10.21105\/joss.00861"},{"key":"pcbi.1009086.ref011","article-title":"Visualizing Data using t-SNE","author":"L. van der Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"pcbi.1009086.ref012","unstructured":"Kingma D. P., Welling M. Auto-Encoding Variational Bayes. International Conference on Learning Representations (ICLR), 2014."},{"key":"pcbi.1009086.ref013","unstructured":"Shazeer N., Mirhoseini A., Maziarz K., Davis A., Le Q, Hinton G. et al. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layers. arXiv, 2017."},{"key":"pcbi.1009086.ref014","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198538493.001.0001","volume-title":"Neural Networks for Pattern Recognition","author":"C. M Bishop","year":"1995"},{"key":"pcbi.1009086.ref015","doi-asserted-by":"crossref","unstructured":"Golik P., Doetsch P., Ney H. Cross-entropy vs. squared error training: a theoretical and experimental comparison INTERSPEECH (2013)","DOI":"10.21437\/Interspeech.2013-436"},{"key":"pcbi.1009086.ref016","volume-title":"Pattern Recognition and Machine Learning","author":"C. M Bishop","year":"2006"},{"key":"pcbi.1009086.ref017","unstructured":"Xie J., Girshick R., Farhadi A. Unsupervised deep embedding for clustering analysis. International Conference on Machine Learning (ICML), 2016."},{"key":"pcbi.1009086.ref018","doi-asserted-by":"crossref","unstructured":"Li F., Qiao H., Zhang B., Xi X. Discriminatively boosted image clustering with fully convolutional autoencoders. arXiv, 2017.","DOI":"10.1016\/j.patcog.2018.05.019"},{"key":"pcbi.1009086.ref019","unstructured":"Saito S., Tan R. T. Neural clustering: Concatenating layers for better projections. Workshop track at ICLR, 2017."},{"key":"pcbi.1009086.ref020","unstructured":"Chen D., Lv J., Yi Z. Unsupervised multi-manifold clustering by learning deep representation. Workshops at the AAAI Conference on Artificial Intelligence, 2017."},{"key":"pcbi.1009086.ref021","doi-asserted-by":"crossref","unstructured":"Mukherjee S., Asnani H., Lin E., Kannan S. ClusterGAN: Latent Space Clustering in Generative Adversarial Networks The Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19) (2019)","DOI":"10.1609\/aaai.v33i01.33014610"},{"key":"pcbi.1009086.ref022","doi-asserted-by":"crossref","unstructured":"Yang J., Parikh D., Batra D. Joint unsupervised learning of deep representations and image clusters. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016b.","DOI":"10.1109\/CVPR.2016.556"},{"key":"pcbi.1009086.ref023","doi-asserted-by":"crossref","unstructured":"Hsu C.-C., Lin C.-W. Cnn-based joint clustering and representation learning with feature drift compensation for large-scale image data. arXiv, 2017.","DOI":"10.1109\/TMM.2017.2745702"},{"key":"pcbi.1009086.ref024","doi-asserted-by":"crossref","unstructured":"Wang Z., Chang S., Zhou J., Wang M., Huang T. S. Learning a task-specific deep architecture for clustering. Proceedings of the SIAM International Conference on Data Mining (ICDM), 2016.","DOI":"10.1137\/1.9781611974348.42"},{"key":"pcbi.1009086.ref025","unstructured":"Hu W., Miyato T., Tokui S., Matsumoto E., Sugiyama M. Learning discrete representations via information maximizing self augmented training. arXiv, 2017."},{"key":"pcbi.1009086.ref026","unstructured":"Shaham U., Stanton K., Li H., Nadler B., Basri R., Kluger Y. SpectralNet: Spectral Clustering using Deep Neural Networks. Published as a conference paper at ICLR, 2018."},{"key":"pcbi.1009086.ref027","unstructured":"Chen X., Duan Y., Houthooft R., Schulman J., Sutskever I., Abbeel, P. Infogan: Interpretable representa- tion learning by information maximizing generative adversarial nets In Advances in Neural Information Processing Systems, 2172\u20132180 (2016)"},{"key":"pcbi.1009086.ref028","doi-asserted-by":"crossref","unstructured":"Gretton A., Borgwardt K., Rasch M. J., Scholkopf B., Smola A. J. A Kernel Method for the Two-Sample Problem. arXiv, 2008.","DOI":"10.7551\/mitpress\/7503.003.0069"},{"key":"pcbi.1009086.ref029","unstructured":"Sutherland D. J., Tung H.-Y., Strathmann H., De S., Ramdas A., Smola A. et al. Generative models and model criticism via optimized maximum mean discrepancy. arXiv, 2019."},{"key":"pcbi.1009086.ref030","article-title":"Dimension Reduction and Clustering Models for Single-Cell RNA Sequencing Data: A Comparative Study","author":"C. Feng","year":"2020","journal-title":"Int J Mol Sci"},{"issue":"number 11","key":"pcbi.1009086.ref031","article-title":"hdbscan: Hierarchical density based clustering","volume":"volume 2","author":"L. McInnes","year":"2017","journal-title":"Journal of Open Source Software The Open Journal"},{"key":"pcbi.1009086.ref032","author":"M. L. D Dias","year":"2019","journal-title":"fuzzy-c-means: An implementation of Fuzzy C-means clustering algorithm Zenodo"},{"key":"pcbi.1009086.ref033","article-title":"Deep Generative Modeling for Single-cell Transcriptomics","author":"R. Lopez","year":"2018","journal-title":"Nat Methods"},{"key":"pcbi.1009086.ref034","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1038\/s41586-018-0590-4","article-title":"Single-cell transcriptomics of 20 mouse organs creates a Tabula Muris","volume":"562","author":"The Tabula Muris Consortium., Overall coordination","year":"2018","journal-title":"Nature"},{"key":"pcbi.1009086.ref035","article-title":"VASC: Dimension Reduction and Visualization of Single-cell RNA-seq Data by Deep Variational Autoencoder","author":"D. Wang","year":"2018","journal-title":"Genomics Proteomics Bioinformatics"},{"key":"pcbi.1009086.ref036","doi-asserted-by":"crossref","first-page":"4200","DOI":"10.1038\/s41598-017-04520-z","article-title":"A Comprehensive Mouse Transcriptomic BodyMap across 17 Tissues by RNA-seq","volume":"7","author":"B. Li","year":"2017","journal-title":"Sci Rep"},{"key":"pcbi.1009086.ref037","volume-title":"J Lipid Res","author":"C. M. Trent","year":"2014"},{"key":"pcbi.1009086.ref038","article-title":"Lipoprotein lipase (LpL) on the surface of cardiomyocytes increases lipid uptake and produces a cardiomyopathy","author":"H. Yagyu","year":"2003","journal-title":"J Clin Invest"},{"key":"pcbi.1009086.ref039","volume-title":"Nature","author":"F. Yue","year":"2014"},{"key":"pcbi.1009086.ref040","article-title":"Application of Mass Cytometry (CyTOF) for Functional and Phenotypic Analysis of Natural Killer Cells","author":"A. W. Kay","year":"2013","journal-title":"Methods in Molecular Biology"},{"key":"pcbi.1009086.ref041","article-title":"Critical assessment of automated flow cytometry data analysis techniques","author":"FlowCAP Consortium","year":"2013","journal-title":"Nature Methods"},{"key":"pcbi.1009086.ref042","article-title":"Comparison of clustering methods for high-dimensional single-cell flow and mass cytometry data","author":"L. M. Weber","year":"2016","journal-title":"Cytometry Part A"},{"key":"pcbi.1009086.ref043","article-title":"FlowSOM: Using self-organizing maps for visualization and interpretation of cytometry data","author":"S. Van Gassen","year":"2015","journal-title":"Cytometry Part A"},{"key":"pcbi.1009086.ref044","article-title":"Data-Driven Phenotypic Dissection of AML Reveals Progenitor-like Cells that Correlate with Prognosis","author":"J. H. Levine","year":"2015","journal-title":"Cell"},{"key":"pcbi.1009086.ref045","article-title":"Automated Mapping of Phenotype Space with Single-Cell Data","author":"N. Samusik","year":"2016","journal-title":"Nature Methods"},{"key":"pcbi.1009086.ref046","article-title":"Computer programs for hierarchical polythetic classification (\u201csimilarity analysis\u201d)","author":"G. N. Lance","year":"1966","journal-title":"Computer Journal"},{"key":"pcbi.1009086.ref047","article-title":"Multiplexed mass cytometry profiling of cellular states perturbed by small-molecule regulators","author":"B. Bodenmiller","year":"2012","journal-title":"Nature Biotechnology"},{"key":"pcbi.1009086.ref048","article-title":"Extracting a cellular hierarchy from high-dimensional cytometry data with SPADE","author":"P. Qiu","year":"2011","journal-title":"Nature Biotechnology"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009086","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,7,13]],"date-time":"2021-07-13T00:00:00Z","timestamp":1626134400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009086","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,5]],"date-time":"2023-11-05T13:36:51Z","timestamp":1699191411000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009086"}},"subtitle":[],"editor":[{"given":"Qing","family":"Nie","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,6,30]]},"references-count":48,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6,30]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009086","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1009086","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,30]]}}}