{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T00:57:56Z","timestamp":1760057876409,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2025,2,25]],"date-time":"2025-02-25T00:00:00Z","timestamp":1740441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007129","name":"Natural Science Foundation of Shandong Province","doi-asserted-by":"publisher","award":["ZR2023MF041","62072469","306607\/2023-9"],"award-info":[{"award-number":["ZR2023MF041","62072469","306607\/2023-9"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["ZR2023MF041","62072469","306607\/2023-9"],"award-info":[{"award-number":["ZR2023MF041","62072469","306607\/2023-9"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shandong Data Open Innovative Application Laboratory","award":["ZR2023MF041","62072469","306607\/2023-9"],"award-info":[{"award-number":["ZR2023MF041","62072469","306607\/2023-9"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>Data heterogeneity poses a significant challenge in federated learning (FL), which has become a central focus of contemporary research in artificial intelligence. Personalized federated learning (pFL), a specialized branch of FL, seeks to address this issue by tailoring models to the unique data distributions of individual clients. Despite its potential, current pFL frameworks face critical limitations, particularly in handling client training discontinuity. When clients are unable to engage in every training round, the resulting models tend to diverge from their local knowledge, leading to catastrophic forgetting. Moreover, existing frameworks often separate the model from the local classifier used for personalization, keeping the classifier local for extended periods. This inherent characteristic of classifiers frequently leads to overfitting on local training data, thereby impairing the generalization capability of the local models. To tackle these challenges, we propose a novel personalized federated learning framework, PFPS-LWC (Personalized Federated Learning with a Progressive Local Training Strategy and a Lightweight Classifier). Our approach introduces local knowledge recall and employs regularized classifiers to mitigate the effects of local knowledge forgetting and enhance the generalization of the models. We evaluated the performance of PFPS-LWC under varying degrees of data heterogeneity using the CIFAR10 and CIFAR100 datasets. Our method outperformed the state-of-the-art approach by up to 4.22% and consistently achieved the best performance across various heterogeneous environments, further demonstrating its effectiveness and robustness.<\/jats:p>","DOI":"10.3390\/app15052481","type":"journal-article","created":{"date-parts":[[2025,2,25]],"date-time":"2025-02-25T12:01:17Z","timestamp":1740484877000},"page":"2481","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Personalized Federated Learning with Progressive Local Training Strategy and Lightweight Classifier"],"prefix":"10.3390","volume":"15","author":[{"given":"Jianhao","family":"Liu","sequence":"first","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7805-3629","authenticated-orcid":false,"given":"Wenjuan","family":"Gong","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}]},{"given":"Ziyi","family":"Fang","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8033-0306","authenticated-orcid":false,"given":"Jordi","family":"Gonz\u00e0lez","sequence":"additional","affiliation":[{"name":"Computer Vision Center, Universitat Aut\u00f2noma de Barcelona, 08193 Barcelona, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8657-3800","authenticated-orcid":false,"given":"Joel","family":"Rodrigues","sequence":"additional","affiliation":[{"name":"Higher School of Technology, Amazonas State University, Manaus 69000, Brazil"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3501813","article-title":"Federated learning for healthcare: Systematic review and architecture proposal","volume":"13","author":"Antunes","year":"2022","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1145\/3633202","article-title":"Privacy-Preserving Individual-Level COVID-19 Infection Prediction via Federated Graph Learning","volume":"42","author":"Fu","year":"2024","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"101006","DOI":"10.1016\/j.patter.2024.101006","article-title":"Recent methodological advances in federated learning for healthcare","volume":"5","author":"Zhang","year":"2024","journal-title":"Patterns"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3951","DOI":"10.1007\/s10462-022-10271-9","article-title":"Securing federated learning with blockchain: A systematic literature review","volume":"56","author":"Qammar","year":"2023","journal-title":"Artif. Intell. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhuang, W., Wen, Y., Zhang, X., Gan, X., Yin, D., Zhou, D., Zhang, S., and Yi, S. (2020, January 12\u201316). Performance optimization of federated person re-identification via benchmark analysis. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3413814"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fantauzzo, L., Fan\u00ec, E., Caldarola, D., Tavera, A., Cermelli, F., Ciccone, M., and Caputo, B. (2022, January 23\u201327). Feddrive: Generalizing federated learning to semantic segmentation in autonomous driving. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981098"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3259","DOI":"10.1109\/TITS.2023.3324962","article-title":"Federated learning in intelligent transportation systems: Recent applications and open problems","volume":"25","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3560486","article-title":"ReFRS: Resource-efficient Federated Recommender System for Dynamic and Diversified User Preferences","volume":"43","author":"Imran","year":"2023","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Le, Q., Diao, E., Wang, X., Anwar, A., Tarokh, V., and Ding, J. (November, January 31). Personalized federated recommender systems with private and partially federated autoencoders. Proceedings of the 2022 56th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA.","DOI":"10.1109\/IEEECONF56349.2022.10051918"},{"key":"ref_10","first-page":"1","article-title":"A Generic Federated Recommendation Framework via Fake Marks and Secret Sharing","volume":"41","author":"Lin","year":"2022","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1145\/3653448","article-title":"Privacy-preserving Cross-domain Recommendation with Federated Graph Learning","volume":"42","author":"Tian","year":"2024","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_12","unstructured":"Karimireddy, S.P., Kale, S., Mohri, M., Reddi, S., Stich, S., and Suresh, A.T. (2020, January 13\u201318). Scaffold: Stochastic controlled averaging for federated learning. Proceedings of the International Conference on Machine Learning., PMLR, Virtual."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Li, Q., He, B., and Song, D. (2021, January 19\u201325). Model-contrastive federated learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01057"},{"key":"ref_14","first-page":"429","article-title":"Federated optimization in heterogeneous networks","volume":"2","author":"Li","year":"2020","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_15","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014). Generative adversarial nets. Adv. Neural Inf. Process. Syst., 27."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hao, W., El-Khamy, M., Lee, J., Zhang, J., Liang, K.J., Chen, C., and Duke, L.C. (2021, January 19\u201325). Towards fair federated learning with zero-shot data augmentation. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00369"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9352","DOI":"10.1109\/TMC.2024.3365295","article-title":"Feature matching data synthesis for non-iid federated learning","volume":"23","author":"Li","year":"2024","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_18","unstructured":"Rasouli, M., Sun, T., and Rajagopal, R. (2020). Fedgan: Federated generative adversarial networks for distributed data. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"22530","DOI":"10.1109\/JIOT.2023.3303889","article-title":"Data augmentation based federated learning","volume":"10","author":"Zhang","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_20","unstructured":"Arivazhagan, M.G., Aggarwal, V., Singh, A.K., and Choudhary, S. (2019). Federated learning with personalization layers. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"13426","DOI":"10.1109\/TNNLS.2023.3269062","article-title":"Fedtp: Federated learning by transformer personalization","volume":"35","author":"Li","year":"2023","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","unstructured":"Vaswani, A. (2017). Attention is all you need. srXiv."},{"key":"ref_23","unstructured":"Li, X., Jiang, M., Zhang, X., Kamp, M., and Dou, Q. (2021). Fedbn: Federated learning on non-iid features via local batch normalization. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, J., Hua, Y., Wang, H., Song, T., Xue, Z., Ma, R., and Guan, H. (2023, January 7\u201314). Fedala: Adaptive local aggregation for personalized federated learning. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v37i9.26330"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3560485","article-title":"Federated User Modeling from Hierarchical Information","volume":"41","author":"Liu","year":"2023","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Han, S., Park, S., Wu, F., Kim, S., Wu, C., Xie, X., and Cha, M. (2022, January 23\u201327). Fedx: Unsupervised federated learning with cross knowledge distillation. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-20056-4_40"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jiang, D., Shan, C., and Zhang, Z. (2020, January 23\u201325). Federated learning algorithm based on knowledge distillation. Proceedings of the 2020 International Conference on Artificial Intelligence and Computer Engineering (ICAICE), Beijing, China.","DOI":"10.1109\/ICAICE51518.2020.00038"},{"key":"ref_28","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_29","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017, January 20\u201322). Communication-efficient learning of deep networks from decentralized data. Proceedings of the Artificial Intelligence and Statistics, PMLR, Fort Lauderdale, FL, USA."},{"key":"ref_30","unstructured":"Acar, D.A.E., Zhao, Y., Navarro, R.M., Mattina, M., Whatmough, P.N., and Saligrama, V. (2021). Federated learning based on dynamic regularization. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gao, L., Fu, H., Li, L., Chen, Y., Xu, M., and Xu, C.Z. (2022, January 18\u201324). Feddc: Federated learning with non-iid data via local drift decoupling and correction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00987"},{"key":"ref_32","unstructured":"Shi, Y., Liang, J., Zhang, W., Tan, V.Y., and Bai, S. (2022). Towards understanding and mitigating dimensional collapse in heterogeneous federated learning. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zheng, X., Xie, H., Guo, Y., and Bie, R. (2023, January 14\u201316). FedIR: Learning Invariant Representations from Heterogeneous Data in Federated Learning. Proceedings of the 2023 19th International Conference on Mobility, Sensing and Networking (MSN), Nanjing, China.","DOI":"10.1109\/MSN60784.2023.00095"},{"key":"ref_34","unstructured":"Tan, J., Zhou, Y., Liu, G., Wang, J.H., and Yu, S. (2023). pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning. arXiv."},{"key":"ref_35","unstructured":"Ye, R., Xu, M., Wang, J., Xu, C., Chen, S., and Wang, Y. (2023, January 23\u201329). Feddisco: Federated learning with discrepancy-aware collaboration. Proceedings of the International Conference on Machine Learning, PMLR, Honolulu, HI, USA."},{"key":"ref_36","first-page":"6596925","article-title":"A data augmentation method for vertical federated learning","volume":"2022","author":"Zhang","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_37","first-page":"21394","article-title":"Personalized federated learning with moreau envelopes","volume":"33","author":"Dinh","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","unstructured":"Li, T., Hu, S., Beirami, A., and Smith, V. (2021, January 18\u201324). Ditto: Fair and robust federated learning through personalization. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_39","unstructured":"Collins, L., Hassani, H., Mokhtari, A., and Shakkottai, S. (2021, January 18\u201324). Exploiting shared representations for personalized federated learning. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_40","unstructured":"Zhang, M., Sapra, K., Fidler, S., Yeung, S., and Alvarez, J.M. (2020). Personalized federated learning with first order model optimization. arXiv."},{"key":"ref_41","unstructured":"Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J., and Zhang, C. (March, January 22). Fedproto: Federated prototype learning across heterogeneous clients. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada."},{"key":"ref_42","unstructured":"Chen, H.Y., and Chao, W.L. (2021). On bridging generic and personalized federated learning for image classification. arXiv."},{"key":"ref_43","unstructured":"Liang, P.P., Liu, T., Ziyin, L., Allen, N.B., Auerbach, R.P., Brent, D., Salakhutdinov, R., and Morency, L.P. (2020). Think locally, act globally: Federated learning with local and global representations. arXiv."},{"key":"ref_44","unstructured":"Scott, C. (2025, January 10). Rademacher Complexity. Available online: https:\/\/web.eecs.umich.edu\/~cscott\/past_courses\/eecs598w14\/notes\/10_rademacher.pdf."},{"key":"ref_45","unstructured":"Yin, D., Kannan, R., and Bartlett, P. (2019, January 9\u201315). Rademacher complexity for adversarially robust generalization. Proceedings of the International Conference on Machine Learning, PMLR, Long Beach, CA, USA."},{"key":"ref_46","first-page":"19290","article-title":"Rethinking the value of labels for improving class-imbalanced learning","volume":"33","author":"Yang","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_47","first-page":"1","article-title":"Learning Multiple Layers of Features from Tiny Images","volume":"1","author":"Krizhevsky","year":"2009","journal-title":"Handbook of Systemic Autoimmune Diseases"}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/15\/5\/2481\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:42:23Z","timestamp":1760028143000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/15\/5\/2481"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,25]]},"references-count":47,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["app15052481"],"URL":"https:\/\/doi.org\/10.3390\/app15052481","relation":{},"ISSN":["2076-3417"],"issn-type":[{"type":"electronic","value":"2076-3417"}],"subject":[],"published":{"date-parts":[[2025,2,25]]}}}