{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T03:31:48Z","timestamp":1773199908299,"version":"3.50.1"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T00:00:00Z","timestamp":1731628800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Federated learning (FL) empowers a cohort of participating devices to contribute collaboratively to a global neural network model, ensuring that their training data remains private and stored locally. Despite its advantages in computational efficiency and privacy preservation, FL grapples with the challenge of non-IID (not independent and identically distributed) data from diverse clients, leading to discrepancies between local and global models and potential performance degradation. In this article, we propose FedGK, an innovative communication-efficient Group-Guided FL framework designed for heterogeneous data distributions. FedGK employs a localized-guided framework that enables the client to effectively assimilate key knowledge from teachers and peers while minimizing extraneous peer information in FL scenarios. We conduct an in-depth analysis of the dynamic similarities among clients over successive communication rounds and develop a novel clustering approach that accurately groups clients with diverse heterogeneities. We implement FedGK on public datasets with an innovative data transformation pattern called \u201ccluster-shift non-IID\u201d, which mirrors the more prevalent data distributions in real-world settings and could be grouped into clusters with similar data distributions. Extensive experimental results on public datasets demonstrate that the proposed approach FedGK improves accuracy by up to 32.89% and saves up to 53.33% communication cost over state-of-the-art baselines.<\/jats:p>","DOI":"10.1145\/3674973","type":"journal-article","created":{"date-parts":[[2024,6,26]],"date-time":"2024-06-26T11:15:56Z","timestamp":1719400556000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["FedGK: Communication-Efficient Federated Learning through Group-Guided Knowledge Distillation"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-8538-9665","authenticated-orcid":false,"given":"Wenjun","family":"Zhang","sequence":"first","affiliation":[{"name":"Computer Science, University of Helsinki, Helsinki, Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4792-2267","authenticated-orcid":false,"given":"Xiaoli","family":"Liu","sequence":"additional","affiliation":[{"name":"Computer Science, University of Helsinki, Helsinki, Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4220-3650","authenticated-orcid":false,"given":"Sasu","family":"Tarkoma","sequence":"additional","affiliation":[{"name":"Computer Science, University of Helsinki, Helsinki, Finland"}]}],"member":"320","published-online":{"date-parts":[[2024,11,15]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207469"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.07.024"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2020.3009406"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCD46524.2019.00038"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPA-BDCloud-SocialCom-SustainCom52081.2021.00042"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","unstructured":"Avishek Ghosh Jichan Chung Dong Yin and Kannan Ramchandran. 2022. An efficient framework for clustered federated learning. IEEE Transactions on Information Theory 68 12 (2022) 8076\u20138091. DOI:10.1109\/TIT.2022.3192506","DOI":"10.1109\/TIT.2022.3192506"},{"key":"e_1_3_1_8_2","unstructured":"Harshvardhan Avishek Ghosh and Arya Mazumdar. 2022. An Improved Algorithm for Clustered Federated Learning. Retrieved from https:\/\/arxiv.org\/abs\/2210.11538"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_10_2","unstructured":"Geoffrey Hinton Oriol Vinyals and Jeff Dean. 2015. Distilling the Knowledge in a Neural Network. Retrieved from https:\/\/arxiv.org\/abs\/1503.02531"},{"key":"e_1_3_1_11_2","unstructured":"Tzu-Ming Harry Hsu Hang Qi and Matthew Brown. 2019. Measuring the effects of non-identical data distribution for federated visual classification. Retrieved from https:\/\/arxiv.org\/abs\/1909.06335"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58607-2_5"},{"key":"e_1_3_1_13_2","unstructured":"Eunjeong Jeong Seungeun Oh Hyesung Kim Jihong Park Mehdi Bennis and Seong-Lyun Kim. 2023. Communication-efficient on-device machine learning: Federated distillation and augmentation under Non-IID private data. Retrieved from https:\/\/arxiv.org\/abs\/1811.11479"},{"key":"e_1_3_1_14_2","first-page":"5132","volume-title":"International Conference on Machine Learning","author":"Karimireddy Sai Praneeth","year":"2020","unstructured":"Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. Scaffold: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning. PMLR, 5132\u20135143."},{"key":"e_1_3_1_15_2","unstructured":"Jakub Kone\u010dn\u00fd H. Brendan McMahan Felix X. Yu Peter Richt\u00e1rik Ananda Theertha Suresh and Dave Bacon. 2017. Federated learning: Strategies for improving communication efficiency. Retrieved from https:\/\/arxiv.org\/abs\/1610.05492"},{"key":"e_1_3_1_16_2","unstructured":"Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical report University of Toronto Toronto Ontario. Retrieved from https:\/\/www.cs.toronto.edu\/kriz\/learning-features-2009-TR.pdf"},{"key":"e_1_3_1_17_2","unstructured":"Daliang Li and Junpu Wang. 2019. FedMD: Heterogenous Federated Learning via Model Distillation. Retrieved from https:\/\/arxiv.org\/abs\/1910.03581"},{"key":"e_1_3_1_18_2","first-page":"429","article-title":"Federated optimization in heterogeneous networks","volume":"2","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. Proceedings of Machine Learning and Systems 2 (2020), 429\u2013450.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_3_1_19_2","unstructured":"Tian Li Maziar Sanjabi Ahmad Beirami and Virginia Smith. 2020. Fair resource allocation in federated learning. Retrieved from https:\/\/arxiv.org\/abs\/1905.10497"},{"key":"e_1_3_1_20_2","unstructured":"Xiang Li Kaixuan Huang Wenhao Yang Shusen Wang and Zhihua Zhang. 2020. On the convergence of fedavg on non-IID data. Retrieved from https:\/\/arxiv.org\/abs\/1907.02189"},{"key":"e_1_3_1_21_2","unstructured":"Tao Lin Lingjing Kong Sebastian U. Stich and Martin Jaggi. 2020. Ensemble distillation for robust model fusion in federated learning. In Advances in Neural Information Processing Systems. Curran Associates Inc. 2351\u20132363. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/18df51b97ccd68128e994804f3eccc87-Paper.pdf"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-022-01046-x"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2022.05.003"},{"key":"e_1_3_1_24_2","first-page":"1273","volume-title":"Artificial Intelligence and Statistics","author":"McMahan Brendan","year":"2017","unstructured":"Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y. Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics. PMLR, 1273\u20131282."},{"key":"e_1_3_1_25_2","first-page":"4615","volume-title":"International Conference on Machine Learning","author":"Mohri Mehryar","year":"2019","unstructured":"Mehryar Mohri, Gary Sivek, and Ananda Theertha Suresh. 2019. Agnostic federated learning. In International Conference on Machine Learning. PMLR, 4615\u20134625."},{"key":"e_1_3_1_26_2","unstructured":"Sashank Reddi Zachary Charles Manzil Zaheer Zachary Garrett Keith Rush Jakub Kone\u010dn\u00fd Sanjiv Kumar and H. Brendan McMahan. 2021. Adaptive federated optimization. Retrieved from https:\/\/arxiv.org\/abs\/2003.00295"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2020.3015958"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2944481"},{"key":"e_1_3_1_29_2","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. Retrieved from https:\/\/arxiv.org\/abs\/1409.1556"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2022.3175149"},{"key":"e_1_3_1_31_2","doi-asserted-by":"crossref","unstructured":"C. Wu F. Wu L. Lyu Y. Huang and X. Xie. 2022. FedKD: Communication Efficient Federated Learning via Knowledge Distillation.NatureCommunications 13 1 (2022) 2032.","DOI":"10.1038\/s41467-022-29763-x"},{"key":"e_1_3_1_32_2","unstructured":"Han Xiao Kashif Rasul and Roland Vollgraf. 2017. Fashion-MNIST: A Novel Image Dataset for Benchmarking Machine Learning Algorithms. Retrieved from https:\/\/arxiv.org\/abs\/1708.07747"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCCS55155.2022.9846843"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1186\/s42400-023-00172-x"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","unstructured":"Yue Zhao Meng Li Liangzhen Lai Naveen Suda Damon Civin and Vikas Chandra. 2018. Federated learning with non-iid data. DOI:10.48550\/ARXIV.1806.00582","DOI":"10.48550\/ARXIV.1806.00582"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","unstructured":"Hangyu Zhu Jinjin Xu Shiqing Liu and Yaochu Jin. 2021. Federated learning on non-IID data: A survey. Neurocomputing 465 11 (2021) 371\u2013390. DOI:10.1016\/j.neucom.2021.07.098","DOI":"10.1016\/j.neucom.2021.07.098"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3674973","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3674973","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:56Z","timestamp":1750291556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3674973"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,15]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3674973"],"URL":"https:\/\/doi.org\/10.1145\/3674973","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"value":"1533-5399","type":"print"},{"value":"1557-6051","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,15]]},"assertion":[{"value":"2024-01-16","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}