{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T17:28:50Z","timestamp":1778347730024,"version":"3.51.4"},"reference-count":219,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T00:00:00Z","timestamp":1762992000000},"content-version":"vor","delay-in-days":316,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,1,1]],"date-time":"2025-01-01T00:00:00Z","timestamp":1735689600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["International Journal of Intelligent Systems"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:p>Federated learning (FL) is a distributed and privacy\u2010preserving machine learning paradigm that coordinates multiple clients to train a model while keeping the raw data localized. However, this traditional FL poses some challenges, including privacy risks, data heterogeneity, communication bottlenecks, and system heterogeneity issues. To tackle these challenges, knowledge distillation (KD) has been widely applied in FL since 2020. KD is a validated and efficacious model compression and enhancement algorithm. The core concept of KD involves facilitating knowledge transfer between models by exchanging logits at intermediate or output layers. These properties make KD an excellent solution for the long\u2010lasting challenges in FL. Up to now, there have been few reviews that summarize and analyze the current trend and methods for how KD can be applied in FL efficiently. This article aims to provide a comprehensive survey of KD\u2010based FL, focusing on addressing the above challenges. First, we provide an overview of KD\u2010based FL, including its motivation, basics, taxonomy, and a comparison with traditional FL and where KD should execute. We also analyze the critical factors in KD\u2010based FL in the Appendix, including teachers, knowledge, data, and methods. We discuss how KD can address the challenges in FL, including privacy protection, data heterogeneity, communication efficiency, and personalization. Finally, we discuss the challenges facing KD\u2010based FL algorithms and future research directions. We hope this survey can provide insights and guidance for researchers and practitioners in the FL area.<\/jats:p>","DOI":"10.1155\/int\/7406934","type":"journal-article","created":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T11:19:36Z","timestamp":1763032776000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Knowledge Distillation in Federated Learning: A Survey on Long Lasting Challenges and New Solutions"],"prefix":"10.1155","volume":"2025","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-6534-0193","authenticated-orcid":false,"given":"Laiqiao","family":"Qin","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3411-7947","authenticated-orcid":false,"given":"Tianqing","family":"Zhu","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1680-2521","authenticated-orcid":false,"given":"Wanlei","family":"Zhou","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3491-5968","authenticated-orcid":false,"given":"Philip S.","family":"Yu","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2025,11,13]]},"reference":[{"key":"e_1_2_14_1_2","first-page":"1273","volume-title":"Artificial Intelligence and Statistics","author":"McMahan B.","year":"2017"},{"key":"e_1_2_14_2_2","article-title":"Learning Differentially Private Recurrent Language Models","author":"McMahan H. B.","year":"2017","journal-title":"arXiv Preprint arXiv:1710.06963"},{"key":"e_1_2_14_3_2","article-title":"Practical One-Shot Federated Learning for Cross-Silo Setting","author":"Li Q.","year":"2020","journal-title":"arXiv Preprint arXiv:2010.01017"},{"key":"e_1_2_14_4_2","first-page":"16937","article-title":"Inverting Gradients-How Easy Is It to Break Privacy in Federated Learning?","volume":"33","author":"Geiping J.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_5_2","unstructured":"ZhaoY. LiM. LaiL. SudaN. CivinD. andChandraV. Federated Learning With Non-iid Data 2018."},{"key":"e_1_2_14_6_2","unstructured":"KarimireddyS. P. KaleS. MohriM. ReddiS. StichS. andSureshA. T. Scaffold: Stochastic Controlled Averaging for Federated Learning International Conference on Machine Learning 2020 PMLR 5132\u20135143."},{"key":"e_1_2_14_7_2","doi-asserted-by":"crossref","unstructured":"WangH. KaplanZ. NiuD. andLiB. Optimizing Federated Learning on Non-iid Data With Reinforcement Learning IEEE INFOCOM 2020-IEEE Conference on Computer Communications 2020 IEEE 1698\u20131707.","DOI":"10.1109\/INFOCOM41043.2020.9155494"},{"key":"e_1_2_14_8_2","article-title":"Fedmd: Heterogenous Federated Learning via Model Distillation","author":"Li D.","year":"2019","journal-title":"arXiv Preprint arXiv:1910.03581"},{"key":"e_1_2_14_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnnls.2022.3160699"},{"key":"e_1_2_14_10_2","first-page":"3622","article-title":"Quped: Quantized Personalization via Distillation With Applications to Federated Learning","volume":"34","author":"Ozkara K.","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_11_2","doi-asserted-by":"crossref","unstructured":"VarnoF. SaghayiM. Rafiee SevyeriL. GuptaS. MatwinS. andHavaeiM. A. Minimizing Client Drift in Federated Learning via Adaptive Bias Estimation European Conference on Computer Vision 2022 Springer 710\u2013726.","DOI":"10.1007\/978-3-031-20050-2_41"},{"key":"e_1_2_14_12_2","doi-asserted-by":"crossref","unstructured":"LiQ. DiaoY. ChenQ. andHeB. Federated Learning on Non-iid Data Silos: An Experimental Study 2022 IEEE 38th International Conference on Data Engineering (ICDE) 2022 IEEE 965\u2013978.","DOI":"10.1109\/ICDE53745.2022.00077"},{"key":"e_1_2_14_13_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i9.16960"},{"key":"e_1_2_14_14_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.10.007"},{"key":"e_1_2_14_15_2","unstructured":"HamerJ. MohriM. andSureshA. T. Fedboost: A Communication-Efficient Algorithm for Federated Learning International Conference on Machine Learning 2020 PMLR 3973\u20133983."},{"key":"e_1_2_14_16_2","article-title":"Distilling the Knowledge in a Neural Network","volume":"2","author":"Hinton G.","year":"2015","journal-title":"arXiv Preprint arXiv:1503.02531"},{"key":"e_1_2_14_17_2","first-page":"2351","article-title":"Ensemble Distillation for Robust Model Fusion in Federated Learning","volume":"33","author":"Lin T.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_18_2","unstructured":"PhuongM.andLampertC. Towards Understanding Knowledge Distillation International Conference on Machine Learning 2019 PMLR 5142\u20135151."},{"key":"e_1_2_14_19_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5963"},{"key":"e_1_2_14_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnse.2021.3081748"},{"key":"e_1_2_14_21_2","unstructured":"ChangH. ShejwalkarV. ShokriR. andHoumansadrA. Cronus: Robust and Heterogeneous Collaborative Learning With Black-Box Knowledge Transfer 2019."},{"key":"e_1_2_14_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/tsipn.2022.3205549"},{"key":"e_1_2_14_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-022-04431-1"},{"key":"e_1_2_14_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISPDC52870.2021.9521631"},{"key":"e_1_2_14_25_2","doi-asserted-by":"crossref","unstructured":"HuangY. KongL. LiQ. andZhangB. Decentralized Federated Learning via Mutual Knowledge Distillation 2023 IEEE International Conference on Multimedia and Expo (ICME) 2023 IEEE 342\u2013347.","DOI":"10.1109\/ICME55011.2023.00066"},{"key":"e_1_2_14_26_2","unstructured":"XieL. LinM. LuanT.et al. Model Heterogeneous Personalized Federated Learning via Injection and Distillation for Medical Data Analysis Proceedings of the 41st International Conference on Machine Learning 2024 54561\u201354575."},{"key":"e_1_2_14_27_2","doi-asserted-by":"crossref","unstructured":"ZhangL. ShenL. DingL. TaoD. andDuanL. Y. Fine-Tuning Global Model via Data-Free Knowledge Distillation for Non-iid Federated Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 10174\u201310183.","DOI":"10.1109\/CVPR52688.2022.00993"},{"key":"e_1_2_14_28_2","doi-asserted-by":"crossref","unstructured":"WangP. LiuB. GuoW. LiY. andGeS. Towards Personalized Federated Learning via Comprehensive Knowledge Distillation 2024 IEEE International Conference on Systems Man and Cybernetics (SMC) 2024 IEEE 3807\u20133812.","DOI":"10.1109\/SMC54092.2024.10831591"},{"key":"e_1_2_14_29_2","article-title":"Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation Under Non-iid Private Data","author":"Jeong E.","year":"2018","journal-title":"arXiv Preprint arXiv:1811.11479"},{"key":"e_1_2_14_30_2","doi-asserted-by":"crossref","unstructured":"HuangW. YeM. andDuB. Learn From Others and Be Yourself in Heterogeneous Federated Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 10143\u201310153.","DOI":"10.1109\/CVPR52688.2022.00990"},{"key":"e_1_2_14_31_2","doi-asserted-by":"crossref","unstructured":"GongX. SharmaA. KaranamS.et al. Ensemble Attention Distillation for Privacy-Preserving Federated Learning Proceedings of the IEEE\/CVF International Conference on Computer Vision 2021 15076\u201315086.","DOI":"10.1109\/ICCV48922.2021.01480"},{"key":"e_1_2_14_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2023.3276865"},{"key":"e_1_2_14_33_2","article-title":"Towards Communication-Efficient and Privacy-Preserving Federated Representation Learning","author":"Shi H.","year":"2021","journal-title":"arXiv Preprint arXiv:2109.14611"},{"key":"e_1_2_14_34_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i11.21446"},{"key":"e_1_2_14_35_2","doi-asserted-by":"crossref","unstructured":"LiuR. WuF. WuC.et al. No One Left Behind: Inclusive Federated Learning Over Heterogeneous Devices Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2022 3398\u20133406 https:\/\/doi.org\/10.1145\/3534678.3539086.","DOI":"10.1145\/3534678.3539086"},{"key":"e_1_2_14_36_2","doi-asserted-by":"publisher","DOI":"10.3390\/s23146518"},{"key":"e_1_2_14_37_2","doi-asserted-by":"crossref","unstructured":"NguyenN. H. NguyenD. L. NguyenT. B.et al. Cadis: Handling Cluster-Skewed Non-iid Data in Federated Learning With Clustered Aggregation and Knowledge Distilled Regularization 2023 IEEE\/ACM 23rd International Symposium on Cluster Cloud and Internet Computing (CCGrid) 2023 IEEE 249\u2013261.","DOI":"10.1109\/CCGrid57682.2023.00032"},{"key":"e_1_2_14_38_2","article-title":"Dataset Distillation","author":"Wang T.","year":"2018","journal-title":"arXiv Preprint arXiv:1811.10959"},{"key":"e_1_2_14_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/tcc.2022.3215520"},{"key":"e_1_2_14_40_2","doi-asserted-by":"crossref","unstructured":"SongR. LiuD. ChenD. Z.et al. Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments 2023 International Joint Conference on Neural Networks (IJCNN) 2023 IEEE 1\u201310.","DOI":"10.1109\/IJCNN54540.2023.10191879"},{"key":"e_1_2_14_41_2","doi-asserted-by":"crossref","unstructured":"ShenY. ZhouY. andYuL. Cd2-pfed: Cyclic Distillation-Guided Channel Decoupling for Model Personalization in Federated Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 10041\u201310050.","DOI":"10.1109\/CVPR52688.2022.00980"},{"key":"e_1_2_14_42_2","unstructured":"ZhouY. PuG. MaX. LiX. andWuD. Distilled One-Shot Federated Learning 2020."},{"key":"e_1_2_14_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnnls.2021.3129371"},{"key":"e_1_2_14_44_2","first-page":"38461","article-title":"Preservation of the Global Knowledge by Not-True Distillation in Federated Learning","volume":"35","author":"Lee G.","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_45_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000083"},{"key":"e_1_2_14_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpami.2021.3055564"},{"key":"e_1_2_14_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_2_14_48_2","doi-asserted-by":"crossref","unstructured":"YuanL. TayF. E. LiG. WangT. andFengJ. Revisiting Knowledge Distillation via Label Smoothing Regularization Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2020 3903\u20133911.","DOI":"10.1109\/CVPR42600.2020.00396"},{"key":"e_1_2_14_49_2","unstructured":"MoraA. TenisonI. BellavistaP. andRishI. Knowledge Distillation in Federated Learning: A Practical Guide Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence 2024 8188\u20138196."},{"key":"e_1_2_14_50_2","article-title":"Neural Networks for Machine Learning Lecture 6a Overview of Mini-Batch Gradient Descent","volume":"14","author":"Hinton G.","year":"2012","journal-title":"Cited on"},{"key":"e_1_2_14_51_2","unstructured":"TangJ. ShivannaR. ZhaoZ.et al. Understanding and Improving Knowledge Distillation 2020."},{"key":"e_1_2_14_52_2","first-page":"7232","article-title":"Evaluating Gradient Inversion Attacks and Defenses in Federated Learning","volume":"34","author":"Huang Y.","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_53_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2305.19600"},{"key":"e_1_2_14_54_2","doi-asserted-by":"publisher","DOI":"10.52202\/079017-3757"},{"key":"e_1_2_14_55_2","unstructured":"Allen-ZhuZ. LiY. andSongZ. A Convergence Theory for Deep Learning via Over-Parameterization International Conference on Machine Learning 2019 PMLR 242\u2013252."},{"key":"e_1_2_14_56_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-020-2885-6"},{"key":"e_1_2_14_57_2","article-title":"Federated Learning: Strategies for Improving Communication Efficiency","author":"Kone\u010dn\u1ef3 J.","year":"2016","journal-title":"arXiv Preprint arXiv:1610.05492"},{"key":"e_1_2_14_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/tce.2023.3327757"},{"key":"e_1_2_14_59_2","doi-asserted-by":"crossref","unstructured":"YangQ. ChenJ. YinX. XieJ. andWenQ. Fedmmd: Heterogenous Federated Learning Based on Multi-Teacher and Multi-Feature Distillation 2022 7th International Conference on Computer and Communication Systems (ICCCS) 2022 IEEE 897\u2013902.","DOI":"10.1109\/ICCCS55155.2022.9846843"},{"key":"e_1_2_14_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/jstsp.2022.3223526"},{"key":"e_1_2_14_61_2","unstructured":"SattlerF. MarbanA. RischkeR. andSamekW. Communication-Efficient Federated Distillation 2020."},{"key":"e_1_2_14_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/tim.2022.3201203"},{"key":"e_1_2_14_63_2","doi-asserted-by":"crossref","unstructured":"AhnJ. H. SimeoneO. andKangJ. Wireless Federated Distillation for Distributed Edge Learning With Heterogeneous Data 2019 IEEE 30th Annual International Symposium on Personal Indoor and Mobile Radio Communications (PIMRC) 2019 IEEE 1\u20136.","DOI":"10.1109\/PIMRC.2019.8904164"},{"key":"e_1_2_14_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/3339474"},{"key":"e_1_2_14_65_2","article-title":"Deep Leakage From Gradients","volume":"32","author":"Zhu L.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_66_2","unstructured":"LamM. WeiG. Y. BrooksD. ReddiV. J. andMitzenmacherM. Gradient Disaggregation: Breaking Privacy in Federated Learning by Reconstructing the User Participant Matrix International Conference on Machine Learning 2021 PMLR 5959\u20135968."},{"key":"e_1_2_14_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460427"},{"key":"e_1_2_14_68_2","article-title":"Communication Efficiency in Federated Learning: Achievements and Challenges","author":"Shahid O.","year":"2021","journal-title":"arXiv Preprint arXiv:2107.10996"},{"key":"e_1_2_14_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2023.3349295"},{"key":"e_1_2_14_70_2","doi-asserted-by":"crossref","unstructured":"LupingW. WeiW. andBoL. Cmfl: Mitigating Communication Overhead for Federated Learning 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) 2019 IEEE 954\u2013964.","DOI":"10.1109\/ICDCS.2019.00099"},{"key":"e_1_2_14_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2019.2944481"},{"key":"e_1_2_14_72_2","unstructured":"MansourY. MohriM. RoJ. andSureshA. T. Three Approaches for Personalization With Applications to Federated Learning 2020."},{"key":"e_1_2_14_73_2","doi-asserted-by":"crossref","unstructured":"KulkarniV. KulkarniM. andPantA. Survey of Personalization Techniques for Federated Learning 2020 Fourth World Conference on Smart Trends in Systems Security and Sustainability (WorldS4) 2020 IEEE 794\u2013797.","DOI":"10.1109\/WorldS450073.2020.9210355"},{"key":"e_1_2_14_74_2","doi-asserted-by":"crossref","unstructured":"LuoB. XiaoW. WangS. HuangJ. andTassiulasL. Tackling System and Statistical Heterogeneity for Federated Learning With Adaptive Client Sampling IEEE INFOCOM 2022-IEEE Conference on Computer Communications 2022 IEEE 1739\u20131748.","DOI":"10.1109\/INFOCOM48880.2022.9796935"},{"key":"e_1_2_14_75_2","volume-title":"Breaking Barriers of System Heterogeneity: Straggler-Tolerant Multimodal Federated Learning via Knowledge Distillation","author":"Chen J.","year":"2024"},{"key":"e_1_2_14_76_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.07.098"},{"key":"e_1_2_14_77_2","doi-asserted-by":"publisher","DOI":"10.1016\/0925-2312(93)90006-o"},{"key":"e_1_2_14_78_2","article-title":"Fairness and Accuracy in Federated Learning","author":"Huang W.","year":"2020","journal-title":"arXiv Preprint arXiv:2012.10069"},{"key":"e_1_2_14_79_2","unstructured":"LiT. HuS. BeiramiA. andSmithV. Ditto: Fair and Robust Federated Learning Through Personalization International Conference on Machine Learning 2021 PMLR 6357\u20136368."},{"key":"e_1_2_14_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnse.2022.3146399"},{"key":"e_1_2_14_81_2","doi-asserted-by":"crossref","unstructured":"HeY. ChenY. YangX. ZhangY. andZengB. Class-Wise Adaptive Self Distillation for Heterogeneous Federated Learning 22 Proceedings of the 36th AAAI Conference on Artificial Intelligence 2022 Virtual.","DOI":"10.1609\/aaai.v36i11.21620"},{"key":"e_1_2_14_82_2","unstructured":"ChoJ. H.andHariharanB. On the Efficacy of Knowledge Distillation Proceedings of the IEEE\/CVF International Conference on Computer Vision 2019 4794\u20134802."},{"key":"e_1_2_14_83_2","doi-asserted-by":"crossref","unstructured":"LiZ. HuangY. ChenD. LuoT. CaiN. andPanZ. Online Knowledge Distillation via Multi-Branch Diversity Enhancement Proceedings of the Asian Conference on Computer Vision 2020.","DOI":"10.1007\/978-3-030-69538-5_20"},{"key":"e_1_2_14_84_2","unstructured":"MenghaniG.andRaviS. Learning From a Teacher Using Unlabeled Data 2019."},{"key":"e_1_2_14_85_2","doi-asserted-by":"crossref","unstructured":"LeeK. LeeK. ShinJ. andLeeH. Overcoming Catastrophic Forgetting With Unlabeled Data in the Wild Proceedings of the IEEE\/CVF International Conference on Computer Vision 2019 312\u2013321 https:\/\/doi.org\/10.1109\/iccv.2019.00040.","DOI":"10.1109\/ICCV.2019.00040"},{"key":"e_1_2_14_86_2","article-title":"Towards Model Agnostic Federated Learning Using Knowledge Distillation","author":"Afonin A.","year":"2021","journal-title":"arXiv Preprint arXiv:2110.15210"},{"key":"e_1_2_14_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/twc.2022.3222717"},{"key":"e_1_2_14_88_2","doi-asserted-by":"crossref","unstructured":"BiegaA. J. PotashP. Daum\u00e9H. DiazF. andFinckM. Operationalizing the Legal Principle of Data Minimization for Personalization Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval 2020 399\u2013408 https:\/\/doi.org\/10.1145\/3397271.3401034.","DOI":"10.1145\/3397271.3401034"},{"key":"e_1_2_14_89_2","doi-asserted-by":"crossref","unstructured":"YamasakiY.andTakaseH. F2mkd: Fog-Enabled Federated Learning With Mutual Knowledge Distillation 2023 IEEE 20th Consumer Communications & Networking Conference (CCNC) 2023 IEEE 682\u2013683.","DOI":"10.1109\/CCNC51644.2023.10059910"},{"key":"e_1_2_14_90_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2024.125336"},{"key":"e_1_2_14_91_2","doi-asserted-by":"crossref","unstructured":"WenH. WuY. LiJ. andDuanH. Communication-Efficient Federated Data Augmentation on Non-iid Data Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 3377\u20133386.","DOI":"10.1109\/CVPRW56347.2022.00381"},{"key":"e_1_2_14_92_2","article-title":"Federated Virtual Learning on Heterogeneous Data With Local-Global Distillation","author":"Huang C. Y.","year":"2023","journal-title":"arXiv Preprint arXiv:2303.02278"},{"key":"e_1_2_14_93_2","doi-asserted-by":"crossref","unstructured":"FangX.andYeM. Robust Federated Learning With Noisy and Heterogeneous Clients Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 10072\u201310081.","DOI":"10.1109\/CVPR52688.2022.00983"},{"key":"e_1_2_14_94_2","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/3259108"},{"key":"e_1_2_14_95_2","doi-asserted-by":"crossref","unstructured":"HuangY. L. YangH. C. andLeeC. C. Federated Learning via Conditional Mutual Learning for Alzheimer\u2019s Disease Classification on t1w Mri 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) 2021 IEEE 2427\u20132432.","DOI":"10.1109\/EMBC46164.2021.9630382"},{"key":"e_1_2_14_96_2","doi-asserted-by":"publisher","DOI":"10.1109\/tmc.2021.3070013"},{"key":"e_1_2_14_97_2","first-page":"22593","article-title":"Distributed Distillation for On-Device Learning","volume":"33","author":"Bistritz I.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_98_2","doi-asserted-by":"crossref","unstructured":"K\u00e1d\u00e1rA.andHadh\u00e1ziD. F. A Client-Wise Distilled Representation Based Semi-Supervised Collaborative Multitask Learning Scheme 2022 International Joint Conference on Neural Networks (IJCNN) 2022 IEEE 1\u20138.","DOI":"10.1109\/IJCNN55064.2022.9892034"},{"key":"e_1_2_14_99_2","unstructured":"ZhuZ. HongJ. andZhouJ. Data-Free Knowledge Distillation for Heterogeneous Federated Learning International Conference on Machine Learning 2021 PMLR 12878\u201312889."},{"key":"e_1_2_14_100_2","doi-asserted-by":"crossref","unstructured":"ShangX. LuY. CheungY. M. andWangH. F. Federated Learning on Non-iid and Long-Tailed Data via Calibrated Distillation 2022 IEEE International Conference on Multimedia and Expo (ICME) 2022 IEEE 1\u20136.","DOI":"10.1109\/ICME52920.2022.9860009"},{"key":"e_1_2_14_101_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.116310"},{"key":"e_1_2_14_102_2","doi-asserted-by":"crossref","unstructured":"XiongY. WangR. ChengM. YuF. andHsiehC. J. F. Iterative Distribution Matching for Communication-Efficient Federated Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2023 16323\u201316332.","DOI":"10.1109\/CVPR52729.2023.01566"},{"key":"e_1_2_14_103_2","first-page":"28663","article-title":"Breaking the Centralized Barrier for Cross-Device Federated Learning","volume":"34","author":"Karimireddy S. P.","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_104_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.106775"},{"key":"e_1_2_14_105_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2021.3075706"},{"key":"e_1_2_14_106_2","doi-asserted-by":"publisher","DOI":"10.1109\/msp.2020.2975749"},{"key":"e_1_2_14_107_2","doi-asserted-by":"publisher","DOI":"10.1109\/mcom.005.210108"},{"key":"e_1_2_14_108_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2023.04.007"},{"key":"e_1_2_14_109_2","doi-asserted-by":"publisher","DOI":"10.1109\/tifs.2019.2929409"},{"key":"e_1_2_14_110_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2021.3074382"},{"key":"e_1_2_14_111_2","first-page":"2074","article-title":"Heterogeneous Blockchain and Ai-Driven Hierarchical Trust Evaluation for 5g-Enabled Intelligent Transportation Systems","volume":"24","author":"Wang X.","year":"2021","journal-title":"IEEE Transactions on Intelligent Transportation Systems"},{"key":"e_1_2_14_112_2","doi-asserted-by":"publisher","DOI":"10.1007\/11787006_1"},{"key":"e_1_2_14_113_2","doi-asserted-by":"crossref","unstructured":"LiW. Milletar\u00ecF. XuD.et al. Privacy-Preserving Federated Brain Tumour Segmentation Machine Learning in Medical Imaging: 10th International Workshop MLMI 2019 Held in Conjunction with MICCAI 2019 Shenzhen China October 13 2019 Proceedings 10 2019 Springer 133\u2013141.","DOI":"10.1007\/978-3-030-32692-0_16"},{"key":"e_1_2_14_114_2","doi-asserted-by":"crossref","unstructured":"TruexS. BaracaldoN. AnwarA.et al. A Hybrid Approach to Privacy-Preserving Federated Learning Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security 2019 1\u201311.","DOI":"10.1145\/3338501.3357370"},{"key":"e_1_2_14_115_2","doi-asserted-by":"crossref","unstructured":"DuW.andAtallahM. J. Secure Multi-Party Computation Problems and Their Applications: A Review and Open Problems Proceedings of the 2001 Workshop on New Security Paradigms 2001 13\u201322 https:\/\/doi.org\/10.1145\/508171.508174.","DOI":"10.1145\/508171.508174"},{"key":"e_1_2_14_116_2","first-page":"1333","article-title":"Privacy-Preserving Deep Learning via Additively Homomorphic Encryption","volume":"13","author":"Aono Y.","year":"2017","journal-title":"IEEE Transactions on Information Forensics and Security"},{"key":"e_1_2_14_117_2","unstructured":"ChenT. KornblithS. NorouziM. andHintonG. A Simple Framework for Contrastive Learning of Visual Representations International Conference on Machine Learning 2020 PMLR 1597\u20131607."},{"key":"e_1_2_14_118_2","doi-asserted-by":"publisher","DOI":"10.1109\/mis.2020.2988604"},{"key":"e_1_2_14_119_2","doi-asserted-by":"crossref","unstructured":"HaoM. LiH. XuG. LiuS. andYangH. Towards Efficient and Privacy-Preserving Federated Deep Learning ICC 2019-2019 IEEE International Conference on Communications (ICC) 2019 IEEE 1\u20136.","DOI":"10.1109\/ICC.2019.8761267"},{"key":"e_1_2_14_120_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2020.2991416"},{"key":"e_1_2_14_121_2","doi-asserted-by":"crossref","unstructured":"XieC. KoyejoO. andGuptaI. S. Secure and Efficient Distributed On-Device Machine Learning Joint European Conference on Machine Learning and Knowledge Discovery in Databases 2019 Springer 213\u2013228.","DOI":"10.1007\/978-3-030-46147-8_13"},{"key":"e_1_2_14_122_2","doi-asserted-by":"publisher","DOI":"10.3233\/FAIA200290"},{"key":"e_1_2_14_123_2","doi-asserted-by":"crossref","unstructured":"XuX. WuJ. YangM.et al. Information Leakage by Model Weights on Federated Learning Proceedings of the 2020 Workshop on Privacy-Preserving Machine Learning in Practice 2020 31\u201336.","DOI":"10.1145\/3411501.3419423"},{"key":"e_1_2_14_124_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2021.3124848"},{"key":"e_1_2_14_125_2","doi-asserted-by":"publisher","DOI":"10.1109\/tdsc.2024.3419211"},{"key":"e_1_2_14_126_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-29763-x"},{"key":"e_1_2_14_127_2","first-page":"21414","article-title":"Dense: Data-Free One-Shot Federated Learning","volume":"35","author":"Zhang J.","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_128_2","doi-asserted-by":"crossref","unstructured":"DworkC. Differential Privacy: A Survey of Results International Conference on Theory and Applications of Models of Computation 2008 Springer 1\u201319.","DOI":"10.1007\/978-3-540-79228-4_1"},{"key":"e_1_2_14_129_2","doi-asserted-by":"crossref","unstructured":"HoechH. RischkeR. M\u00fcllerK. andSamekW. Fedauxfdp: Differentially Private One-Shot Federated Distillation International Workshop on Trustworthy Federated Learning 2022 Springer 100\u2013114.","DOI":"10.1007\/978-3-031-28996-5_8"},{"key":"e_1_2_14_130_2","article-title":"Secure Multi-Party Computation","volume":"78","author":"Goldreich O.","year":"1998","journal-title":"Manuscript. Preliminary version"},{"key":"e_1_2_14_131_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-12229-8"},{"key":"e_1_2_14_132_2","doi-asserted-by":"crossref","unstructured":"MendietaM. YangT. WangP. LeeM. DingZ. andChenC. Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 8397\u20138406.","DOI":"10.1109\/CVPR52688.2022.00821"},{"key":"e_1_2_14_133_2","unstructured":"ShohamN. AvidorT. KerenA.et al. Overcoming Forgetting in Federated Learning on Non-iid Data 2019."},{"key":"e_1_2_14_134_2","doi-asserted-by":"crossref","unstructured":"DuanM. LiuD. ChenX. TanY. RenJ. andQiaoL. Others. Astraea: Self-Balancing Federated Learning for Improving Classification Accuracy of Mobile Deep Learning Applications 2019 IEEE 37th International Conference on Computer Design (ICCD) 2019 IEEE 246\u2013254.","DOI":"10.1109\/ICCD46524.2019.00038"},{"key":"e_1_2_14_135_2","first-page":"3557","article-title":"Personalized Federated Learning With Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach","volume":"33","author":"Fallah A.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_136_2","unstructured":"ArivazhaganM. G. AggarwalV. SinghA. K. andChoudharyS. Federated Learning With Personalization Layers 2019."},{"key":"e_1_2_14_137_2","first-page":"19586","article-title":"An Efficient Framework for Clustered Federated Learning","volume":"33","author":"Ghosh A.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_138_2","doi-asserted-by":"publisher","DOI":"10.1109\/ojsp.2020.3036276"},{"key":"e_1_2_14_139_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2024.121759"},{"key":"e_1_2_14_140_2","first-page":"429","article-title":"Federated Optimization in Heterogeneous Networks","volume":"2","author":"Li T.","year":"2020","journal-title":"Proceedings of Machine learning and systems"},{"key":"e_1_2_14_141_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-99-4749-2_30"},{"key":"e_1_2_14_142_2","doi-asserted-by":"publisher","DOI":"10.1109\/tpds.2022.3225185"},{"key":"e_1_2_14_143_2","doi-asserted-by":"crossref","unstructured":"ZhangY. XiangT. HospedalesT. M. andLuH. Deep Mutual Learning Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018 4320\u20134328.","DOI":"10.1109\/CVPR.2018.00454"},{"key":"e_1_2_14_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2023.3271678"},{"key":"e_1_2_14_145_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-statistics-030718-104938"},{"key":"e_1_2_14_146_2","doi-asserted-by":"publisher","DOI":"10.3390\/info14040234"},{"key":"e_1_2_14_147_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2023.3243622"},{"key":"e_1_2_14_148_2","doi-asserted-by":"crossref","unstructured":"WangH. LiY. XuW. LiR. ZhanY. andZengZ. D. Domain-Aware Federated Knowledge Distillation Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2023 20412\u201320421.","DOI":"10.1109\/CVPR52729.2023.01955"},{"key":"e_1_2_14_149_2","doi-asserted-by":"crossref","unstructured":"ZhengX. YingS. ZhengF.et al. ArdagnaC. A. BenatallahB. BianH.et al. Federated Learning on Non-iid Data via Local and Global Distillation IEEE International Conference on Web Services ICWS 2023 Chicago IL USA July 2-8 2023 2023 IEEE 647\u2013657.","DOI":"10.1109\/ICWS60048.2023.00083"},{"key":"e_1_2_14_150_2","doi-asserted-by":"crossref","unstructured":"HanS. ParkS. WuF.et al. Fedx: Unsupervised Federated Learning With Cross Knowledge Distillation European Conference on Computer Vision 2022 Springer 691\u2013707.","DOI":"10.1007\/978-3-031-20056-4_40"},{"key":"e_1_2_14_151_2","doi-asserted-by":"crossref","unstructured":"DongJ. WangL. FangZ.et al. Federated Class-Incremental Learning Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2022 10164\u201310173.","DOI":"10.1109\/CVPR52688.2022.00992"},{"key":"e_1_2_14_152_2","doi-asserted-by":"crossref","unstructured":"DongJ. ZhangD. CongY. CongW. DingH. andDaiD. Federated Incremental Semantic Segmentation Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2023 3934\u20133943 https:\/\/doi.org\/10.1109\/cvpr52729.2023.00383.","DOI":"10.1109\/CVPR52729.2023.00383"},{"key":"e_1_2_14_153_2","doi-asserted-by":"crossref","unstructured":"SuS. LiB. ZhangC. YangM. andXueX. Cross-Domain Federated Object Detection 2023 IEEE International Conference on Multimedia and Expo (ICME) 2023 IEEE 1469\u20131474.","DOI":"10.1109\/ICME55011.2023.00254"},{"key":"e_1_2_14_154_2","doi-asserted-by":"crossref","unstructured":"MouT. JiangX. LiJ.et al. Fedtam: Decentralized Federated Learning With a Feature Attention Based Multi-Teacher Knowledge Distillation for Healthcare 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS) 2023 IEEE 1246\u20131253.","DOI":"10.1109\/ICPADS60453.2023.00180"},{"key":"e_1_2_14_155_2","article-title":"Hepco: Data-Free Heterogeneous Prompt Consolidation for Continual Federated Learning","author":"Halbe S.","year":"2023","journal-title":"arXiv Preprint arXiv:2306.09970"},{"key":"e_1_2_14_156_2","doi-asserted-by":"crossref","unstructured":"MaY. XieZ. WangJ. ChenK. andShouL. Continual Federated Learning Based on Knowledge Distillation 3 Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence 2022.","DOI":"10.24963\/ijcai.2022\/303"},{"key":"e_1_2_14_157_2","doi-asserted-by":"publisher","DOI":"10.1109\/tbdata.2022.3189703"},{"key":"e_1_2_14_158_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2024.109216"},{"key":"e_1_2_14_159_2","unstructured":"KimJ. KimG. andHanB. Multi-Level Branched Regularization for Federated Learning International Conference on Machine Learning 2022 PMLR 11058\u201311073."},{"key":"e_1_2_14_160_2","doi-asserted-by":"crossref","unstructured":"ChenY. ChenD. WangH. HanK. andZhaoM. Confidence-Based Federated Distillation for Vision-Based Lane-Centering IEEE International Conference on Acoustics Speech and Signal Processing ICASSP 2023-Workshops Rhodes Island Greece June 4-10 2023 2023 IEEE 1\u20135.","DOI":"10.1109\/ICASSPW59220.2023.10193741"},{"key":"e_1_2_14_161_2","doi-asserted-by":"crossref","unstructured":"LuoY. ZhengL. GuanT. YuJ. andYangY. Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2019 2507\u20132516.","DOI":"10.1109\/CVPR.2019.00261"},{"key":"e_1_2_14_162_2","doi-asserted-by":"crossref","unstructured":"SankaranarayananS. BalajiY. JainA. LimS. N. andChellappaR. Learning From Synthetic Data: Addressing Domain Shift for Semantic Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2018 3752\u20133761 https:\/\/doi.org\/10.1109\/cvpr.2018.00395 2-s2.0-85055515209.","DOI":"10.1109\/CVPR.2018.00395"},{"key":"e_1_2_14_163_2","first-page":"7449","volume-title":"Findings of the Association for Computational Linguistics","author":"Wang R.","year":"2023"},{"key":"e_1_2_14_164_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2024789118"},{"key":"e_1_2_14_165_2","doi-asserted-by":"publisher","DOI":"10.1109\/comst.2020.2986024"},{"key":"e_1_2_14_166_2","doi-asserted-by":"crossref","unstructured":"KangM. ChikontweP. KimS.et al. One-Shot Federated Learning on Medical Data Using Knowledge Distillation With Image Synthesis and Client Model Adaptation International Conference on Medical Image Computing and Computer-Assisted Intervention 2023 Springer 521\u2013531.","DOI":"10.1007\/978-3-031-43895-0_49"},{"key":"e_1_2_14_167_2","doi-asserted-by":"publisher","DOI":"10.1109\/BMSB58369.2023.10211313"},{"key":"e_1_2_14_168_2","doi-asserted-by":"publisher","DOI":"10.1109\/tccn.2023.3286665"},{"key":"e_1_2_14_169_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.02.011"},{"key":"e_1_2_14_170_2","unstructured":"ChenH. WangC. andVikaloH. The Best of Both Worlds: Accurate Global and Personalized Models Through Federated Learning With Data-Free Hyper-Knowledge Distillation The Eleventh International Conference on Learning Representations ICLR 2023 Kigali Rwanda May 1-5 2023 2023 OpenReview.net."},{"key":"e_1_2_14_171_2","doi-asserted-by":"crossref","unstructured":"LuoK. WangS. FuY.et al. DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learning IEEE International Conference on Data Mining ICDM 2024 Abu Dhabi United Arab Emirates December 9-12 2024 2024 IEEE.","DOI":"10.1109\/ICDM59182.2024.00035"},{"key":"e_1_2_14_172_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106627"},{"key":"e_1_2_14_173_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2022.102690"},{"key":"e_1_2_14_174_2","doi-asserted-by":"publisher","DOI":"10.3390\/s23010006"},{"key":"e_1_2_14_175_2","unstructured":"DengY. KamaniM. M. andMahdaviM. Adaptive Personalized Federated Learning 2020."},{"key":"e_1_2_14_176_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2024.107594"},{"key":"e_1_2_14_177_2","doi-asserted-by":"publisher","DOI":"10.1109\/access.2023.3294812"},{"key":"e_1_2_14_178_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.1496"},{"key":"e_1_2_14_179_2","doi-asserted-by":"crossref","unstructured":"WangS. XieJ. LuM. andXiongN. N. Fedgraph-Kd: An Effective Federated Graph Learning Scheme Based on Knowledge Distillation 2023 IEEE 9th Intl Conference on Big Data Security on Cloud (BigDataSecurity) IEEE Intl Conference on High Performance and Smart Computing (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS) 2023 IEEE 130\u2013134.","DOI":"10.1109\/BigDataSecurity-HPSC-IDS58521.2023.00032"},{"key":"e_1_2_14_180_2","unstructured":"ZhuZ. HongJ. DrewS. andZhouJ. Resilient and Communication Efficient Learning for Heterogeneous Federated Systems Proceedings of Thirty-Ninth International Conference on Machine Learning (ICML 2022) 2022."},{"key":"e_1_2_14_181_2","doi-asserted-by":"crossref","unstructured":"ZhangL. SongJ. GaoA. ChenJ. BaoC. andMaK. Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation Proceedings of the IEEE\/CVF International Conference on Computer Vision 2019 3713\u20133722.","DOI":"10.1109\/ICCV.2019.00381"},{"key":"e_1_2_14_182_2","article-title":"Metafed: Federated Learning Among Federations With Cyclic Knowledge Distillation for Personalized Healthcare","author":"Chen Y.","year":"2022","journal-title":"arXiv Preprint arXiv:2206.08516"},{"key":"e_1_2_14_183_2","doi-asserted-by":"crossref","unstructured":"LiuL. ZhangJ. SongS. andLetaiefK. B. Communication-Efficient Federated Distillation With Active Data Sampling ICC 2022-IEEE International Conference on Communications 2022 IEEE 201\u2013206.","DOI":"10.1109\/ICC45855.2022.9839214"},{"key":"e_1_2_14_184_2","doi-asserted-by":"crossref","unstructured":"HuangJ. ZhangY. BiR. LinJ. andXiongJ. Knowledge Distillation Enables Federated Learning: A Data-Free Federated Aggregation Scheme 2024 International Joint Conference on Neural Networks (IJCNN) 2024 IEEE 1\u20137.","DOI":"10.1109\/IJCNN60899.2024.10650725"},{"key":"e_1_2_14_185_2","doi-asserted-by":"publisher","DOI":"10.1145\/3422622"},{"key":"e_1_2_14_186_2","first-page":"6840","article-title":"Denoising Diffusion Probabilistic Models","volume":"33","author":"Ho J.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_187_2","doi-asserted-by":"publisher","DOI":"10.1109\/tifs.2019.2897874"},{"key":"e_1_2_14_188_2","article-title":"Selective Knowledge Sharing for Privacy-Preserving Federated Distillation Without a Good Teacher","author":"Shao J.","year":"2023","journal-title":"arXiv Preprint arXiv:2304.01731"},{"key":"e_1_2_14_189_2","doi-asserted-by":"crossref","unstructured":"HuangC. J. WangL. andHanX. Vertical Federated Knowledge Transfer via Representation Distillation for Healthcare Collaboration Networks Proceedings of the ACM Web Conference 2023 2023 4188\u20134199 https:\/\/doi.org\/10.1145\/3543507.3583874.","DOI":"10.1145\/3543507.3583874"},{"key":"e_1_2_14_190_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i18.29958"},{"key":"e_1_2_14_191_2","article-title":"One Teacher Is Enough? Pre-Trained Language Model Distillation From Multiple Teachers","author":"Wu C.","year":"2021","journal-title":"arXiv Preprint arXiv:2106.01023"},{"key":"e_1_2_14_192_2","doi-asserted-by":"crossref","unstructured":"LiB. ShiY. GuoY. KongQ. andJiangY. Incentive and Knowledge Distillation Based Federated Learning for Cross-Silo Applications IEEE INFOCOM 2022-IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) 2022 IEEE 1\u20136.","DOI":"10.1109\/INFOCOMWKSHPS54753.2022.9798320"},{"key":"e_1_2_14_193_2","article-title":"Exploring the Distributed Knowledge Congruence in Proxy-Data-Free Federated Distillation","author":"Wu Z.","year":"2022","journal-title":"arXiv Preprint arXiv:2204.07028"},{"key":"e_1_2_14_194_2","doi-asserted-by":"crossref","unstructured":"LeH. Q. ShinJ. H. NguyenM. N. andHongC. S. Distilling Knowledge in Federated Learning 2021 22nd Asia-Pacific Network Operations and Management Symposium (APNOMS) 2021 IEEE 196\u2013201.","DOI":"10.23919\/APNOMS52696.2021.9562670"},{"key":"e_1_2_14_195_2","doi-asserted-by":"crossref","unstructured":"ZhaoR. YangL. WangY. XueZ. GuiG. andOhtsukiT. A Semi-Supervised Federated Learning Scheme via Knowledge Distillation for Intrusion Detection ICC 2022-IEEE International Conference on Communications 2022 IEEE 2688\u20132693.","DOI":"10.1109\/ICC45855.2022.9838256"},{"key":"e_1_2_14_196_2","doi-asserted-by":"crossref","unstructured":"MaJ. YonetaniR. andIqbalZ. Adaptive Distillation for Decentralized Learning From Heterogeneous Clients 2020 25th International Conference on Pattern Recognition (ICPR) 2021 IEEE 7486\u20137492.","DOI":"10.1109\/ICPR48806.2021.9412356"},{"key":"e_1_2_14_197_2","unstructured":"YaoD. PanW. DaiY.et al. Local-Global Knowledge Distillation in Heterogeneous Federated Learning With Non-iid Data 2021."},{"key":"e_1_2_14_198_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2022.05.079"},{"key":"e_1_2_14_199_2","doi-asserted-by":"crossref","unstructured":"ChenP. LiuS. ZhaoH. andJiaJ. Distilling Knowledge via Knowledge Review Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition 2021b 5008\u20135017.","DOI":"10.1109\/CVPR46437.2021.00497"},{"key":"e_1_2_14_200_2","article-title":"Fitnets: Hints for Thin Deep Nets","author":"Romero A.","year":"2014","journal-title":"arXiv Preprint arXiv:1412.6550"},{"key":"e_1_2_14_201_2","doi-asserted-by":"crossref","unstructured":"LeH. Q. NguyenL. X. ParkS. B. andHongC. S. Layer-Wise Knowledge Distillation for Cross-Device Federated Learning 2023 International Conference on Information Networking (ICOIN) 2023 IEEE 526\u2013529.","DOI":"10.1109\/ICOIN56518.2023.10049011"},{"key":"e_1_2_14_202_2","doi-asserted-by":"crossref","unstructured":"YimJ. JooD. BaeJ. andKimJ. A Gift From Knowledge Distillation: Fast Optimization Network Minimization and Transfer Learning Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017 4133\u20134141.","DOI":"10.1109\/CVPR.2017.754"},{"key":"e_1_2_14_203_2","doi-asserted-by":"crossref","unstructured":"WuC. ZhangF. andWuF. Distributed Modelling Approaches for Data Privacy Preserving 2019 IEEE Fifth International Conference on Multimedia Big Data (BigMM) 2019 IEEE 357\u2013365.","DOI":"10.1109\/BigMM.2019.00016"},{"key":"e_1_2_14_204_2","doi-asserted-by":"crossref","unstructured":"SunC. JiangT. ZonouzS. andPompiliD. Fed2kd: Heterogeneous Federated Learning for Pandemic Risk Assessment via Two-Way Knowledge Distillation 2022 17th Wireless On-Demand Network Systems and Services Conference (WONS) 2022 IEEE 1\u20138.","DOI":"10.23919\/WONS54113.2022.9764443"},{"key":"e_1_2_14_205_2","doi-asserted-by":"crossref","unstructured":"SunL.andLyuL. ZhouZ. Federated Model Distillation With Noise-Free Differential Privacy Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence IJCAI 2021 Virtual Event\/Montreal Canada 19-27 August 2021 2021 ijcai.org 1563\u20131570.","DOI":"10.24963\/ijcai.2021\/216"},{"key":"e_1_2_14_206_2","article-title":"Personalized Federated Learning for Heterogeneous Clients With Clustered Knowledge Transfer","author":"Cho Y. J.","year":"2021","journal-title":"arXiv Preprint arXiv:2109.08119"},{"key":"e_1_2_14_207_2","article-title":"Feddtg: Federated Data-Free Knowledge Distillation via Three-Player Generative Adversarial Networks","author":"Zhang Z.","year":"2022","journal-title":"arXiv Preprint arXiv:2201.03169"},{"key":"e_1_2_14_208_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnnls.2018.2876865"},{"key":"e_1_2_14_209_2","doi-asserted-by":"crossref","unstructured":"PengC. GuoY. ChenY. RuiQ. YangZ. andXuC. Fedgm: Heterogeneous Federated Learning via Generative Learning and Mutual Distillation European Conference on Parallel Processing 2023 Springer 339\u2013351.","DOI":"10.1007\/978-3-031-39698-4_23"},{"key":"e_1_2_14_210_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics11203262"},{"key":"e_1_2_14_211_2","doi-asserted-by":"crossref","unstructured":"UsmanovaA. PortetF. LalandaP. andVegaG. Federated Continual Learning Through Distillation in Pervasive Computing 2022 IEEE International Conference on Smart Computing (SMARTCOMP) 2022 IEEE 86\u201391.","DOI":"10.1109\/SMARTCOMP55677.2022.00027"},{"key":"e_1_2_14_212_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_2_14_213_2","first-page":"30","volume-title":"2023 24st Asia-Pacific Network Operations and Management Symposium (APNOMS)","author":"Zou L.","year":"2023"},{"key":"e_1_2_14_214_2","first-page":"14068","article-title":"Group Knowledge Transfer: Federated Learning of Large Cnns at the Edge","volume":"33","author":"He C.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_14_215_2","article-title":"Split Learning for Health: Distributed Deep Learning Without Sharing Raw Patient Data","author":"Vepakomma P.","year":"2018","journal-title":"arXiv Preprint arXiv:1812.00564"},{"key":"e_1_2_14_216_2","article-title":"Label Driven Knowledge Distillation for Federated Learning With Non-iid Data","author":"Nguyen M. D.","year":"2022","journal-title":"arXiv Preprint arXiv:2209.14520"},{"key":"e_1_2_14_217_2","doi-asserted-by":"publisher","DOI":"10.1109\/tac.1980.1102314"},{"key":"e_1_2_14_218_2","doi-asserted-by":"publisher","DOI":"10.1109\/jiot.2021.3078543"},{"key":"e_1_2_14_219_2","article-title":"Knowledge Distillation in Federated Learning: A Survey on Long Lasting Challenges and New Solutions","author":"Qin L.","year":"2024","journal-title":"arXiv Preprint arXiv:2406.10861"}],"container-title":["International Journal of Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/int\/7406934","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1155\/int\/7406934","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/int\/7406934","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,8]],"date-time":"2026-03-08T17:59:54Z","timestamp":1772992794000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/int\/7406934"}},"subtitle":[],"editor":[{"given":"Richard","family":"Murray","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2025,1]]},"references-count":219,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["10.1155\/int\/7406934"],"URL":"https:\/\/doi.org\/10.1155\/int\/7406934","archive":["Portico"],"relation":{},"ISSN":["0884-8173","1098-111X"],"issn-type":[{"value":"0884-8173","type":"print"},{"value":"1098-111X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1]]},"assertion":[{"value":"2025-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"7406934"}}