{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T22:11:33Z","timestamp":1783721493343,"version":"3.55.0"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,6,18]],"date-time":"2024-06-18T00:00:00Z","timestamp":1718668800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100006754","name":"Army Research Lab","doi-asserted-by":"crossref","award":["W911NF-21-2-0272"],"award-info":[{"award-number":["W911NF-21-2-0272"]}],"id":[{"id":"10.13039\/100006754","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"crossref","award":["W911NF2410049"],"award-info":[{"award-number":["W911NF2410049"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Science Foundation","award":["CNS-2148182"],"award-info":[{"award-number":["CNS-2148182"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>Self-supervised representation learning and deep clustering are mutually beneficial to learn high-quality representations and cluster data simultaneously in centralized settings. However, it is not always feasible to gather large amounts of data at a central entity, considering data privacy requirements and computational resources. Federated Learning (FL) has been developed successfully to aggregate a global model while training on distributed local data, respecting the data privacy of edge devices. However, most FL research effort focuses on supervised learning algorithms. A fully unsupervised federated clustering scheme has not been considered in the existing literature. We present federated momentum contrastive clustering (FedMCC), a generic federated clustering framework that can not only cluster data automatically but also extract discriminative representations training from distributed local data over multiple users. In FedMCC, we demonstrate a two-stage federated learning paradigm where the first stage aims to learn differentiable instance embeddings and the second stage accounts for clustering data automatically. The experimental results show that FedMCC not only achieves superior clustering performance but also outperforms several existing federated self-supervised methods for linear evaluation and semi-supervised learning tasks. Additionally, FedMCC can easily be adapted to ordinary centralized clustering through what we call momentum contrastive clustering (MCC). We show that MCC achieves state-of-the-art clustering accuracy results in certain datasets such as STL-10 and ImageNet-10. We also present a method to reduce the memory footprint of our clustering schemes.<\/jats:p>","DOI":"10.1145\/3653981","type":"journal-article","created":{"date-parts":[[2024,3,26]],"date-time":"2024-03-26T12:31:07Z","timestamp":1711456267000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Federated Momentum Contrastive Clustering"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0140-8967","authenticated-orcid":false,"given":"Runxuan","family":"Miao","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering, University of Illinois Chicago, Chicago, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6238-0470","authenticated-orcid":false,"given":"Erdem","family":"Koyuncu","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Illinois Chicago, Chicago, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,18]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3552326.3567485"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.3390\/en10050587"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/LANMAN58293.2023.10189426"},{"key":"e_1_3_1_5_2","unstructured":"Bahman Bahmani Benjamin Moseley Andrea Vattani Ravi Kumar and Sergei Vassilvitskii. 2012. Scalable K-Means++. Retrieved from https:\/\/arxiv:1203.6402"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"e_1_3_1_7_2","series-title":"Proceedings of Machine Learning Research","first-page":"1597","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 119), Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR, 1597\u20131607. Retrieved from https:\/\/proceedings.mlr.press\/v119\/chen20j.html"},{"key":"e_1_3_1_8_2","article-title":"Improved baselines with momentum contrastive learning","author":"Chen Xinlei","year":"2020","unstructured":"Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020. Improved baselines with momentum contrastive learning. Retrieved from https:\/\/arXiv:2003.04297","journal-title":"Retrieved from https:\/\/arXiv:2003.04297"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01549"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02169"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-46502-2_13"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.02002"},{"key":"e_1_3_1_13_2","article-title":"Scaling deep contrastive learning batch size under memory limited setup","author":"Gao Luyu","year":"2021","unstructured":"Luyu Gao, Yunyi Zhang, Jiawei Han, and Jamie Callan. 2021. Scaling deep contrastive learning batch size under memory limited setup. Retrieved from https:\/\/arXiv:2101.06983","journal-title":"Retrieved from https:\/\/arXiv:2101.06983"},{"key":"e_1_3_1_14_2","unstructured":"Jean-Bastien Grill Florian Strub Florent Altch\u00e9 Corentin Tallec Pierre H. Richemond Elena Buchatskaya Carl Doersch Bernardo Avila Pires Zhaohan Daniel Guo Mohammad Gheshlaghi Azar Bilal Piot Koray Kavukcuoglu R\u00e9mi Munos and Michal Valko. 2020. Bootstrap your Own Latent: A New Approach to Self-supervised Learning. Retrieved from https:\/\/arxiv:2006.07733"},{"key":"e_1_3_1_15_2","first-page":"550","volume-title":"Proceedings of the Asian Conference on Machine Learning","author":"Guo Xifeng","year":"2018","unstructured":"Xifeng Guo, En Zhu, Xinwang Liu, and Jianping Yin. 2018. Deep embedded clustering with data augmentation. In Proceedings of the Asian Conference on Machine Learning. PMLR, 550\u2013565."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN55064.2022.9891952"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103291"},{"key":"e_1_3_1_20_2","unstructured":"Zhizhong Huang Jie Chen Junping Zhang and Hongming Shan. 2021. Exploring non-contrastive representation learning for deep clustering. Retrieved from https:\/\/arxiv.org\/abs\/2111.11821"},{"key":"e_1_3_1_21_2","series-title":"Proceedings of Machine Learning Research","first-page":"5132","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Karimireddy Sai Praneeth","year":"2020","unstructured":"Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 119), Hal Daum\u00e9 III and Aarti Singh (Eds.). PMLR, 5132\u20135143. Retrieved from https:\/\/proceedings.mlr.press\/v119\/karimireddy20a.html"},{"key":"e_1_3_1_22_2","article-title":"Adam: A method for stochastic optimization","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. Retrieved from https:\/\/arXiv:1412.6980","journal-title":"Retrieved from https:\/\/arXiv:1412.6980"},{"key":"e_1_3_1_23_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Koyuncu Erdem","year":"2023","unstructured":"Erdem Koyuncu. 2023. Memorization capacity of neural networks with conditional computation. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3183294"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.4236\/jcc.2014.211002"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/OJCOMS.2023.3280174"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/LANMAN52105.2021.9478829"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01057"},{"key":"e_1_3_1_29_2","first-page":"429","volume-title":"Proceedings of Machine Learning and Systems","volume":"2","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, I. Dhillon, D. Papailiopoulos, and V. Sze (Eds.), Vol. 2. 429\u2013450. Retrieved from https:\/\/proceedings.mlsys.org\/paper\/2020\/file\/38af86134b65d0f10fe33d30dd76442e-Paper.pdf"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i10.17037"},{"key":"e_1_3_1_31_2","first-page":"281","volume-title":"Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability","author":"MacQueen James","year":"1967","unstructured":"James MacQueen et\u00a0al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability. 281\u2013297."},{"key":"e_1_3_1_32_2","series-title":"Proceedings of Machine Learning Research","first-page":"1273","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics","volume":"54","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics(Proceedings of Machine Learning Research, Vol. 54), Aarti Singh and Jerry Zhu (Eds.). PMLR, 1273\u20131282. Retrieved from https:\/\/proceedings.mlr.press\/v54\/mcmahan17a.html"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP43922.2022.9746105"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126658"},{"key":"e_1_3_1_35_2","unstructured":"Hieu Pham Zihang Dai Golnaz Ghiasi Kenji Kawaguchi Hanxiao Liu Adams Wei Yu Jiahui Yu Yi-Ting Chen Minh-Thang Luong Yonghui Wu et\u00a0al. 2021. Combined scaling for open-vocabulary image classification. Retrieved from https:\/\/arxiv.org\/abs\/2111.10050"},{"key":"e_1_3_1_36_2","article-title":"On the pros and cons of momentum encoder in self-supervised visual representation learning","author":"Pham Trung","year":"2022","unstructured":"Trung Pham, Chaoning Zhang, Axi Niu, Kang Zhang, and Chang D. Yoo. 2022. On the pros and cons of momentum encoder in self-supervised visual representation learning. Retrieved from https:\/\/arXiv:2208.05744","journal-title":"Retrieved from https:\/\/arXiv:2208.05744"},{"key":"e_1_3_1_37_2","unstructured":"Yuming Shen Ziyi Shen Menghan Wang Jie Qin Philip H. S. Torr and Ling Shao. 2021. You never cluster alone. Retrieved from https:\/\/arxiv.org\/abs\/2106.01908"},{"key":"e_1_3_1_38_2","unstructured":"Shuai Wang and Tsung-Hui Chang. 2020. Federated clustering via matrix factorization models: From model averaging to gradient sharing. Retrieved from https:\/\/arxiv.org\/abs\/2002.04930"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2023.3265033"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3625558"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00156"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2023.3343112"},{"key":"e_1_3_1_43_2","unstructured":"Fengda Zhang Kun Kuang Zhaoyang You Tao Shen Jun Xiao Yin Zhang Chao Wu Yueting Zhuang and Xiaolin Li. 2020. Federated unsupervised representation learning. Retrieved from https:\/\/arxiv.org\/abs\/2010.08982."},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00909"},{"key":"e_1_3_1_45_2","article-title":"A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions","author":"Zhou Sheng","year":"2022","unstructured":"Sheng Zhou, Hongjia Xu, Zhuonan Zheng, Jiawei Chen, Jiajun Bu, Jia Wu, Xin Wang, Wenwu Zhu, Martin Ester et\u00a0al. 2022. A comprehensive survey on deep clustering: Taxonomy, challenges, and future directions. Retrieved from arXiv:2206.07579.","journal-title":"Retrieved from arXiv:2206.07579"},{"key":"e_1_3_1_46_2","doi-asserted-by":"crossref","unstructured":"Weiming Zhuang Xin Gan Yonggang Wen Shuai Zhang and Shuai Yi. 2021. Collaborative unsupervised visual representation learning from decentralized data. Retrieved from https:\/\/arxiv.org\/abs\/2108.06492","DOI":"10.1109\/ICCV48922.2021.00487"},{"key":"e_1_3_1_47_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Zhuang Weiming","year":"2022","unstructured":"Weiming Zhuang, Yonggang Wen, and Shuai Zhang. 2022. Divergence-aware federated self-supervised learning. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=oVE1z8NlNe"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653981","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3653981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:36Z","timestamp":1750291416000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3653981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,18]]},"references-count":46,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3653981"],"URL":"https:\/\/doi.org\/10.1145\/3653981","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,18]]},"assertion":[{"value":"2022-11-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}