{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T18:17:12Z","timestamp":1782929832418,"version":"3.54.5"},"reference-count":43,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T00:00:00Z","timestamp":1767830400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China Youth Fund","award":["61902165"],"award-info":[{"award-number":["61902165"]}]},{"name":"Liaoning Provincial Department of Education Fund","award":["203070132279"],"award-info":[{"award-number":["203070132279"]}]},{"name":"Liaoning Provincial Department of Education Fund","award":["JYTMS2023104"],"award-info":[{"award-number":["JYTMS2023104"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Federated learning with data free knowledge distillation enables effective and privacy-preserving knowledge aggregation by employing generators to produce local pseudo samples during client-side model migration. However, in practical applications, data distributions across different institutions are often non-independent and identically distributed (Non-IID), which introduces bias in local models and consequently impedes the effective transfer of knowledge to the global model. In addition, insufficient local training can further exacerbate model bias, undermining overall performance. To address these challenges, we propose a heterogeneous federated learning framework that enhances knowledge transfer through guidance from global proxy data. Specifically, a noise filter is incorporated into the training of local generators to mitigate the negative impact of low-quality pseudo proxy samples on local knowledge distillation. Furthermore, a global generator is introduced to produce global pseudo proxy samples, which, together with local pseudo proxy data, are used to construct a cross attention matrix. This design effectively alleviates overfitting and underfitting issues in local models caused by data heterogeneity. Extensive experiments on publicly available datasets with heterogeneous data distributions demonstrate the superiority of the proposed framework. Results show that when the Dirichlet distribution coefficient is 0.05, our method achieves an average accuracy improvement of 5.77% over popular baselines; when the coefficient is 0.1, the improvement reaches 6.54%. Even under uniformly distributed sample classes, our model still achieves an average accuracy improvement of 7.07% compared to other methods.<\/jats:p>","DOI":"10.3390\/fi18010036","type":"journal-article","created":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T09:01:29Z","timestamp":1767862889000},"page":"36","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Heterogeneous Federated Learning via Knowledge Transfer Guided by Global Pseudo Proxy Data"],"prefix":"10.3390","volume":"18","author":[{"given":"Wenhao","family":"Sun","sequence":"first","affiliation":[{"name":"Information Center, Dalian Party Institute of Communist Party of China, No. 75 Binhai West Road, Xigang District, Dalian 116016, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7642-3905","authenticated-orcid":false,"given":"Xiaoxuan","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Liaoning Normal University, No. 1 Liushu South Street, Ganjingzi District, Dalian 116081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenjun","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Liaoning Normal University, No. 1 Liushu South Street, Ganjingzi District, Dalian 116081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fang","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computer Science and Artificial Intelligence, Liaoning Normal University, No. 1 Liushu South Street, Ganjingzi District, Dalian 116081, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"357","DOI":"10.22214\/ijraset.2024.61000","article-title":"Privacy-Centric AI: Navigating the Landscape with Federated Learning","volume":"12","author":"Shrivastava","year":"2024","journal-title":"Int. J. Res. Appl. Sci. Eng. Technol. (IJRASET)"},{"key":"ref_2","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017, January 20\u201322). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) 2017, Fort Lauderdale, FL, USA. PMLR."},{"key":"ref_3","unstructured":"Smith, V., Chiang, C.K., Sanjabi, M., and Talwalkar, A.S. (2017). Federated multi-task learning. Adv. Neural Inf. Process. Syst. 30 (NIPS), 30."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2031","DOI":"10.1109\/COMST.2020.2986024","article-title":"Federated learning in mobile edge networks: A comprehensive survey","volume":"22","author":"Lim","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","first-page":"8","article-title":"Heterogeneous data and big data analytics","volume":"3","author":"Wang","year":"2017","journal-title":"Autom. Control Inf. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"280","DOI":"10.2307\/3545921","article-title":"On definition and quantification of heterogeneity","volume":"73","author":"Li","year":"1995","journal-title":"Oikos"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Huang, W., Ye, M., and Du, B. (2022, January 18\u201324). Learn from others and be yourself in heterogeneous federated learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00990"},{"key":"ref_8","unstructured":"Chen, H., Wang, Y., Xu, C., Yang, Z., Liu, C., Shi, B., Xu, C., Xu, C., and Tian, Q. (November, January 27). Data-free learning of student networks. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_9","unstructured":"Pan, H., Wang, C., Qiu, M., Zhang, Y., Li, Y., and Huang, J. (2020). Meta-KD: A meta knowledge distillation framework for language model compression across domains. arXiv."},{"key":"ref_10","unstructured":"Zhu, Z., Hong, J., and Zhou, J. (2020, January 18\u201324). Data-free knowledge distillation for heterogeneous federated learning. Proceedings of the 38 th International Conference on Machine Learning, Virtual. PMLR."},{"key":"ref_11","unstructured":"Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., and Khazaeni, Y. (2019, January 9\u201315). Bayesian nonparametric federated learning of neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA. PMLR."},{"key":"ref_12","unstructured":"Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. (2020). Federated learning with matched averaging. arXiv."},{"key":"ref_13","unstructured":"Chen, H.Y., and Chao, W.L. (2020). Fedbe: Making bayesian model ensemble applicable to federated learning. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yu, F., Zhang, W., Qin, Z., Xu, Z., Wang, D., Liu, C., Tian, Z., and Chen, X. (2021, January 14\u201318). Fed2: Feature-aligned federated learning. Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, Singapore.","DOI":"10.1145\/3447548.3467309"},{"key":"ref_15","unstructured":"Corinzia, L., Beuret, A., and Buhmann, J.M. (2019). Variational federated multi-task learning. arXiv."},{"key":"ref_16","unstructured":"Jiang, Y., Kone\u010dn\u00fd, J., Rush, K., and Kannan, S. (2019). Improving federated learning personalization via model agnostic meta learning. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Duan, M., Liu, D., Chen, X., Tan, Y., Ren, J., Qiao, L., and Liang, L. (2019, January 17\u201320). Astraea: Self-balancing federated learning for improving classification accuracy of mobile deep learning applications. Proceedings of the 2019 IEEE 37th International Conference on Computer Design (ICCD), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICCD46524.2019.00038"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"10951","DOI":"10.1007\/s10115-025-02556-6","article-title":"Emerging AI threats in cybercrime: A review of zero-day attacks via machine, deep, and federated learning","volume":"67","author":"Alansary","year":"2025","journal-title":"Knowl. Inf. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3877","DOI":"10.1109\/TIFS.2025.3555193","article-title":"Enhanced model poisoning attack and multi-strategy defense in federated learning","volume":"20","author":"Yang","year":"2025","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_20","unstructured":"Acar, D.A.E., Zhao, Y., Zhu, R., Matas, R., Mattina, M., Whatmough, P., and Saligrama, V. (2021, January 18\u201324). Debiasing model updates for improving personalized federated training. Proceedings of the International Conference on Machine Learning, Virtual. PMLR."},{"key":"ref_21","unstructured":"Shoham, N., Avidor, T., Keren, A., Israel, N., Benditkis, D., Mor-Yosef, L., and Zeitak, I. (2019). Overcoming forgetting in federated learning on non-iid data. arXiv."},{"key":"ref_22","first-page":"429","article-title":"Federated optimization in heterogeneous networks","volume":"2","author":"Li","year":"2020","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_23","first-page":"21394","article-title":"Personalized federated learning with moreau envelopes","volume":"33","author":"Tran","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, Q., He, B., and Song, D. (2021, January 20\u201325). Model-contrastive federated learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01057"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.future.2023.01.019","article-title":"Fedproc: Prototypical contrastive federated learning on non-iid data","volume":"143","author":"Mu","year":"2023","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_26","unstructured":"Yoon, T., Shin, S., Hwang, S.J., and Yang, E. (2021). Fedmix: Approximation of mixup under mean augmented federated learning. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1655","DOI":"10.1109\/TII.2022.3195896","article-title":"Safe: Synergic data filtering for federated learning in cloud-edge computing","volume":"19","author":"Xu","year":"2022","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, L., Zhang, J., Song, S.H., and Letaief, K.B. (2022, January 16\u201320). Communication-efficient federated distillation with active data sampling. Proceedings of the ICC 2022-IEEE International Conference on Communications, Seoul, Republic of Korea.","DOI":"10.1109\/ICC45855.2022.9839214"},{"key":"ref_29","first-page":"14068","article-title":"Group knowledge transfer: Federated learning of large cnns at the edge","volume":"33","author":"He","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Fang, X., and Ye, M. (2022, January 18\u201324). Robust federated learning with noisy and heterogeneous clients. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00983"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Huang, W., Ye, M., Du, B., and Gao, X. (2022, January 10\u201314). Few-shot model agnostic federated learning. Proceedings of the 30th ACM International Conference on Multimedia, Lisboa, Portugal.","DOI":"10.1145\/3503161.3548764"},{"key":"ref_32","unstructured":"Li, D., and Wang, J. (2019). Fedmd: Heterogenous federated learning via model distillation. arXiv."},{"key":"ref_33","unstructured":"Chang, H., Shejwalkar, V., Shokri, R., and Houmansadr, A. (2019). Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer. arXiv."},{"key":"ref_34","first-page":"3622","article-title":"Quped: Quantized personalization via distillation with applications to federated learning","volume":"34","author":"Ozkara","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","first-page":"2351","article-title":"Ensemble distillation for robust model fusion in federated learning","volume":"33","author":"Lin","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhang, L., Shen, L., Ding, L., Tao, D., and Duan, L.Y. (2022, January 18\u201324). Fine-tuning global model via data-free knowledge distillation for non-iid federated learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00993"},{"key":"ref_37","first-page":"21414","article-title":"Dense: Data-free one-shot federated learning","volume":"35","author":"Zhang","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","unstructured":"Heinbaugh, C.E., Luz-Ricca, E., and Shao, H. (2023, January 1\u20135). Data-free one-shot federated learning under very high statistical heterogeneity. Proceedings of the Eleventh International Conference on Learning Representations, Kigali, Rwanda."},{"key":"ref_39","first-page":"1","article-title":"Exploring the distributed knowledge congruence in proxy-data-free federated distillation","volume":"15","author":"Wu","year":"2024","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhang, J., Chen, C., Zhuang, W., and Lyu, L. (2023, January 1\u20136). Target: Federated class-continual learning via exemplar-free distillation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Paris, France.","DOI":"10.1109\/ICCV51070.2023.00441"},{"key":"ref_41","unstructured":"Zhu, Y., Li, X., Wu, Z., Wu, D., Hu, M., and Li, R.H. (2024). FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning. arXiv."},{"key":"ref_42","first-page":"17854","article-title":"DFRD: Data-Free Robustness Distillation for Heterogeneous Federated Learning","volume":"36","author":"Wang","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"106627","DOI":"10.1016\/j.neunet.2024.106627","article-title":"Data-free knowledge distillation via generator-free data generation for Non-IID federated learning","volume":"179","author":"Zhao","year":"2024","journal-title":"Neural Netw."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/1\/36\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T09:13:49Z","timestamp":1767863629000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/18\/1\/36"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,8]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1]]}},"alternative-id":["fi18010036"],"URL":"https:\/\/doi.org\/10.3390\/fi18010036","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,8]]}}}