{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T04:49:45Z","timestamp":1749703785870},"reference-count":77,"publisher":"MIT Press","issue":"7","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We consider the scenario of deep clustering, in which the available prior knowledge is limited. In this scenario, few existing state-of-the-art deep clustering methods can perform well for both noncomplex topology and complex topology data sets. To address the problem, we propose a constraint utilizing symmetric InfoNCE, which helps an objective of the deep clustering method in the scenario of training the model so as to be efficient for not only noncomplex topology but also complex topology data sets. Additionally, we provide several theoretical explanations of the reason that the constraint can enhances the performance of deep clustering methods. To confirm the effectiveness of the proposed constraint, we introduce a deep clustering method named MIST, which is a combination of an existing deep clustering method and our constraint. Our numerical experiments via MIST demonstrate that the constraint is effective. In addition, MIST outperforms other state-of-the-art deep clustering methods for most of the commonly used 10 benchmark data sets.<\/jats:p>","DOI":"10.1162\/neco_a_01591","type":"journal-article","created":{"date-parts":[[2023,5,15]],"date-time":"2023-05-15T22:20:45Z","timestamp":1684189245000},"page":"1288-1339","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":4,"title":["Deep Clustering With a Constraint for Topological Invariance Based on Symmetric InfoNCE"],"prefix":"10.1162","volume":"35","author":[{"given":"Yuhui","family":"Zhang","sequence":"first","affiliation":[{"name":"Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan zhang.y.av@m.titech.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuichiro","family":"Wada","sequence":"additional","affiliation":[{"name":"Fujitsu Limited, Nakahara-ku, Kawasaki, Kanagawa 211-8588, Japan"},{"name":"RIKEN AIP, Chuo-ku, Tokyo 103-0027, Japan wada.yuichiro@jp.fujitsu.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiroki","family":"Waida","sequence":"additional","affiliation":[{"name":"Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan waida.h.aa@m.titech.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaito","family":"Goto","sequence":"additional","affiliation":[{"name":"Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan kaitogoto7@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yusaku","family":"Hino","sequence":"additional","affiliation":[{"name":"Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan blueozk@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takafumi","family":"Kanamori","sequence":"additional","affiliation":[{"name":"Tokyo Institute of Technology, Meguro-ku, Tokyo 152-8552, Japan"},{"name":"RIKEN AIP, Chuo-ku, Tokyo 103-0027, Japan kanamori@c.titech.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,6,12]]},"reference":[{"issue":"6","key":"2023061316475118800_B1","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.3390\/e13061170","article-title":"Geometry of q-exponential family of probability distributions","volume":"13","author":"Amari","year":"2011","journal-title":"Entropy"},{"key":"2023061316475118800_B2","article-title":"Self-labelling via simultaneous clustering and representation learning","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Asano","year":"2019"},{"key":"2023061316475118800_B3","doi-asserted-by":"publisher","first-page":"153630","DOI":"10.1109\/ACCESS.2020.3017884","article-title":"Non-parametric clustering using deep neural networks","volume":"8","author":"Avgerinos","year":"2020","journal-title":"IEEE Access"},{"key":"2023061316475118800_B4","first-page":"4034","article-title":"Connecting language and vision for natural language\u2013based vehicle retrieval","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Bai","year":"2021"},{"key":"2023061316475118800_B5","first-page":"463","article-title":"Rademacher and Gaussian complexities: Risk bounds and structural results","volume":"3","author":"Bartlett","year":"2002","journal-title":"Journal of Machine Learning Research"},{"key":"2023061316475118800_B6","first-page":"531","article-title":"Mutual information neural estimation","volume-title":"Proceedings of Machine Learning Research","author":"Belghazi","year":"2018"},{"issue":"8","key":"2023061316475118800_B7","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"1","key":"2023061316475118800_B8","article-title":"Further reverse results for Jensen's discrete inequality and applications in information theory","volume":"3","author":"Budimir","year":"2000","journal-title":"RGMIA Research Report Collection"},{"key":"2023061316475118800_B9","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-01264-9_9","article-title":"Deep clustering for unsupervised learning of visual features","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Caron","year":"2018"},{"key":"2023061316475118800_B10","first-page":"9912","article-title":"Unsupervised learning of visual features by contrasting cluster assignments","volume-title":"Advances in neural information processing systems","author":"Caron","year":"2020"},{"key":"2023061316475118800_B11","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9780262033589.001.0001","volume-title":"Semi-supervised learning","author":"Chapelle","year":"2006"},{"key":"2023061316475118800_B12","author":"Chen","year":"2015","journal-title":"Deep learning with nonparametric clustering"},{"key":"2023061316475118800_B13","first-page":"1597","article-title":"A simple framework for contrastive learning of visual representations","volume-title":"Proceedings of Machine Learning Research","author":"Chen","year":"2020"},{"key":"2023061316475118800_B14","first-page":"215","article-title":"An analysis of single-layer networks in unsupervised feature learning","volume-title":"Proceedings of Machine Learning Research","author":"Coates","year":"2011"},{"key":"2023061316475118800_B15","volume-title":"Elements of information theory","author":"Cover","year":"1999"},{"key":"2023061316475118800_B16","first-page":"13688","article-title":"Nearest neighbor matching for deep clustering","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Dang","year":"2021"},{"issue":"3","key":"2023061316475118800_B17","doi-asserted-by":"crossref","DOI":"10.1093\/biomet\/56.3.463","article-title":"Estimating the components of a mixture of normal distributions","volume":"56","author":"Day","year":"1969","journal-title":"Biometrika"},{"key":"2023061316475118800_B18","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"ImageNet: A large-scale hierarchical image database","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng","year":"2009"},{"key":"2023061316475118800_B19","first-page":"226","article-title":"A density-based algorithm for discovering clusters in large spatial databases with noise","author":"Ester","year":"1996","journal-title":"Proceedings of the Second International Conference on Knowledge Discovery and Data Mining"},{"key":"2023061316475118800_B20","volume-title":"Hands-on machine learning with Scikit-Learn, Keras and TensorFlow: Concepts, tools, and techniques to build intelligent systems","author":"G\u00e9ron","year":"2019"},{"issue":"3","key":"2023061316475118800_B21","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1109\/TNN.2002.1000150","article-title":"Mercer kernel-based clustering in feature space","volume":"13","author":"Girolami","year":"2002","journal-title":"IEEE Transactions on Neural Networks"},{"key":"2023061316475118800_B22","first-page":"21271","article-title":"Bootstrap your own latent: A new approach to self-supervised learning","volume-title":"Advances in neural information processing systems","author":"Grill","year":"2020"},{"key":"2023061316475118800_B23","first-page":"1753","article-title":"Improved deep embedded clustering with local structure preservation","volume-title":"Proceedings of the IJCAI International Joint Conference on Artificial Intelligence","author":"Guo","year":"2017"},{"key":"2023061316475118800_B24","article-title":"Unsupervised clustering using pseudo-semi-supervised learning","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Gupta","year":"2020"},{"key":"2023061316475118800_B25","first-page":"1026","article-title":"Delving deep into rectifiers: Surpassing human-level performance on imagenet classification","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"He","year":"2015"},{"key":"2023061316475118800_B26","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"2023061316475118800_B27","author":"Hinton","year":"2012","journal-title":"Improving neural networks by preventing co-adaptation of feature detectors"},{"key":"2023061316475118800_B28","first-page":"1558","article-title":"Learning discrete representations via information maximizing self-augmented training","volume-title":"Proceedings of Machine Learning Research","author":"Hu","year":"2017"},{"key":"2023061316475118800_B29","first-page":"448","article-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","author":"Ioffe","year":"2015"},{"key":"2023061316475118800_B30","first-page":"9864","article-title":"Invariant information clustering for unsupervised image classification and segmentation","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Ji","year":"2019"},{"key":"2023061316475118800_B31","first-page":"1965","article-title":"Variational deep embedding: An unsupervised generative approach to Clustering","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence","author":"Jiang","year":"2017"},{"key":"2023061316475118800_B32","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/978-1-4842-2766-4_12","article-title":"Introduction to PyTorch","volume-title":"Deep learning with Python","author":"Ketkar","year":"2017"},{"key":"2023061316475118800_B33","article-title":"Adam: A method for stochastic optimization","volume-title":"Proceedings of the 3rd International Conference on Learning Representations","author":"Kingma","year":"2015"},{"issue":"1\u20132","key":"2023061316475118800_B34","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Naval Research Logistics Quarterly"},{"key":"2023061316475118800_B35","article-title":"One shot learning of simple visual concepts","volume-title":"Proceedings of the 33rd Annual Conference of the Cognitive Science Society","author":"Lake","year":"2011"},{"key":"2023061316475118800_B36","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/B978-1-55860-377-6.50048-7","article-title":"NewsWeeder: Learning to filter Netnews","volume-title":"Machine Learning Proceedings 1995","author":"Lang","year":"1995"},{"issue":"7553","key":"2023061316475118800_B37","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"4","key":"2023061316475118800_B38","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1162\/neco.1989.1.4.541","article-title":"Backpropagation applied to handwritten zip code recognition","volume":"1","author":"LeCun","year":"1989","journal-title":"Neural Computation"},{"issue":"11","key":"2023061316475118800_B39","doi-asserted-by":"publisher","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proceedings of the IEEE"},{"key":"2023061316475118800_B40","first-page":"361","article-title":"RCV1: A new benchmark collection for text categorization research","volume":"5","author":"Lewis","year":"2004","journal-title":"Journal of Machine Learning Research"},{"key":"2023061316475118800_B41","doi-asserted-by":"publisher","first-page":"8547","DOI":"10.1609\/aaai.v35i10.17037","article-title":"Contrastive clustering","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Li","year":"2021"},{"issue":"3","key":"2023061316475118800_B42","doi-asserted-by":"publisher","DOI":"10.1080\/00031305.2017.1419145","article-title":"Sharpening Jensen's inequality","volume":"73","author":"Liao","year":"2019","journal-title":"American Statistician"},{"key":"2023061316475118800_B43","first-page":"281","article-title":"Some methods for classification and analysis of multivariate observations","volume-title":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability","author":"MacQueen","year":"1967"},{"key":"2023061316475118800_B44","first-page":"55","article-title":"Geometry for q-exponential families","volume-title":"Recent progress in differential geometry and its related fields","author":"Matsuzoe","year":"2012"},{"key":"2023061316475118800_B45","first-page":"1258","article-title":"Deep embedded cluster tree","volume-title":"Proceedings of the 2019 IEEE International Conference on Data Mining","author":"Mautz","year":"2019"},{"key":"2023061316475118800_B46","first-page":"5145","article-title":"N2D: (not too) deep clustering via clustering the local manifold of an autoencoded embedding","volume-title":"Proceedings of the International Conference on Pattern Recognition","author":"McConville","year":"2020"},{"issue":"29","key":"2023061316475118800_B47","doi-asserted-by":"publisher","DOI":"10.21105\/joss.00861","article-title":"UMAP: Uniform manifold approximation and projection","volume":"3","author":"McInnes","year":"2018","journal-title":"Journal of Open Source Software"},{"issue":"8","key":"2023061316475118800_B48","doi-asserted-by":"publisher","first-page":"1979","DOI":"10.1109\/TPAMI.2018.2858821","article-title":"Virtual adversarial training: A regularization method for supervised and semi-supervised learning","volume":"41","author":"Miyato","year":"2019","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2023061316475118800_B49","volume-title":"Foundations of machine learning","author":"Mohri","year":"2018"},{"key":"2023061316475118800_B50","first-page":"7945","article-title":"Deep transformation-invariant clustering","volume-title":"Advances in neural information processing systems","author":"Monnier","year":"2020"},{"key":"2023061316475118800_B51","first-page":"933","article-title":"Minimax-optimal semi-supervised regression on unknown manifolds","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","author":"Moscovich","year":"2017"},{"key":"2023061316475118800_B52","first-page":"4610","article-title":"ClusterGAN: Latent space clustering in generative adversarial networks","volume-title":"Proceedings of the 33rd AAAI Conference on Artificial Intelligence","author":"Mukherjee","year":"2019"},{"key":"2023061316475118800_B53","article-title":"Rectified linear units improve restricted Boltzmann machines","volume-title":"Proceedings of the 27th International Conference on Machine Learning","author":"Nair","year":"2010"},{"issue":"3","key":"2023061316475118800_B54","first-page":"405","article-title":"The q-exponential family in statistical physics","volume":"7","author":"Naudts","year":"2009","journal-title":"Central European Journal of Physics"},{"key":"2023061316475118800_B55","volume-title":"Generalized clustering by learning to optimize expected normalized cuts","author":"Nazi","year":"2019"},{"key":"2023061316475118800_B56","article-title":"Reading digits in natural images with unsupervised feature learning","volume-title":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011","author":"Netzer","year":"2011"},{"key":"2023061316475118800_B57","first-page":"849","article-title":"On spectral clustering: Analysis and an algorithm","volume-title":"Advances in neural information processing systems","author":"Ng","year":"2002"},{"issue":"3","key":"2023061316475118800_B58","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1023\/A:1011139631724","article-title":"Modeling the shape of the scene: A holistic representation of the spatial envelope","volume":"42","author":"Oliva","year":"2001","journal-title":"International Journal of Computer Vision"},{"issue":"1","key":"2023061316475118800_B59","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1145\/1007730.1007731","article-title":"Subspace clustering for high dimensional data: a review","volume":"6","author":"Parsons","year":"2004","journal-title":"SIGKDD Explor"},{"key":"2023061316475118800_B60","first-page":"5171","article-title":"On variational bounds of mutual information","volume-title":"Proceedings of Machine Learning Research","author":"Poole","year":"2019"},{"issue":"2","key":"2023061316475118800_B61","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the dimension of a model","volume":"6","author":"Schwarz","year":"1978","journal-title":"Annals of Statistics"},{"key":"2023061316475118800_B62","article-title":"SpectralNet: Spectral clustering using deep neural networks","volume-title":"Proceedings of the 6th International Conference on Learning Representations","author":"Shaham","year":"2018"},{"key":"2023061316475118800_B63","volume-title":"Understanding machine learning: From theory to algorithms","author":"Shalev-Shwartz","year":"2013"},{"issue":"1","key":"2023061316475118800_B64","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1093\/comjnl\/16.1.30","article-title":"SLINK: An optimally efficient algorithm for the single-link cluster method","volume":"16","author":"Sibson","year":"1973","journal-title":"Computer Journal"},{"key":"2023061316475118800_B65","first-page":"1857","article-title":"Improved deep metric learning with multi-class N-pair loss objective","volume-title":"Advances in neural information processing systems","author":"Sohn","year":"2016"},{"key":"2023061316475118800_B66","author":"Springenberg","year":"2015","journal-title":"Unsupervised and semi-supervised learning with categorical generative adversarial networks"},{"issue":"11","key":"2023061316475118800_B67","doi-asserted-by":"publisher","first-page":"1958","DOI":"10.1109\/TPAMI.2008.128","article-title":"80 million tiny images: A large data set for nonparametric object and scene recognition","volume":"30","author":"Torralba","year":"2008","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2023061316475118800_B68","article-title":"On mutual information maximization for representation learning","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Tschannen","year":"2019"},{"key":"2023061316475118800_B69","volume-title":"Representation learning with contrastive predictive coding","author":"Van Den Oord","year":"2018"},{"key":"2023061316475118800_B70","first-page":"268","article-title":"Scan: Learning to classify images without labels","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Van Gansbeke","year":"2020"},{"key":"2023061316475118800_B71","first-page":"349","article-title":"Fast algorithm for approximate k-nearest neighbor graph construction","volume-title":"Proceedings of the IEEE 13th International Conference on Data Mining Workshops","author":"Wang","year":"2013"},{"key":"2023061316475118800_B72","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/978-3-540-39624-6_14","article-title":"Kernel trick embedded gaussian mixture model","volume-title":"Algorithmic learning theory","author":"Wang","year":"2003"},{"key":"2023061316475118800_B73","article-title":"Chaos is a ladder: A new understanding of contrastive learning","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Wang","year":"2022"},{"key":"2023061316475118800_B74","article-title":"Unsupervised deep embedding for clustering analysis","volume-title":"Proceedings of the 33rd International Conference on Machine Learning","author":"Xie","year":"2016"},{"key":"2023061316475118800_B75","first-page":"5147","article-title":"Joint unsupervised learning of deep representations and image clusters","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Yang","year":"2016"},{"key":"2023061316475118800_B76","first-page":"9098","article-title":"Adversarial learning for robust deep clustering","volume-title":"Advances in neural information processing systems","author":"Yang","year":"2020"},{"key":"2023061316475118800_B77","first-page":"660","article-title":"Fast kNN graph construction with locality sensitive hashing","volume-title":"Machine learning and knowledge discovery in databases","author":"Zhang","year":"2013"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/7\/1288\/2128952\/neco_a_01591.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/7\/1288\/2128952\/neco_a_01591.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T16:48:47Z","timestamp":1686674927000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/7\/1288\/115998\/Deep-Clustering-With-a-Constraint-for-Topological"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,12]]},"references-count":77,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,6,12]]},"published-print":{"date-parts":[[2023,6,12]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01591","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,7]]},"published":{"date-parts":[[2023,6,12]]}}}