{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T17:30:17Z","timestamp":1776879017108,"version":"3.51.2"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T00:00:00Z","timestamp":1670803200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T00:00:00Z","timestamp":1670803200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["World Wide Web"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In conventional federated learning, each device is restricted to train a network model of the same structure. This greatly hinders the application of federated learning where the data and devices are quite heterogeneous because of their different hardware equipment and communication networks. At the same time, existing studies have shown that transmitting all of the model parameters not only has heavy communication costs, but also increases risk of privacy leakage. We propose a general framework for personalized federated learning (PerHeFed), which enables the devices to design their local model structures autonomously and share sub-models without structural restrictions. In PerHeFed, a simple-but-effective mapping relation and a novel personalized sub-model aggregation method are proposed for heterogeneous sub-models to be aggregated. By dividing the aggregations into two primitive types (i.e., inter-layer and intra-layer), PerHeFed is applicable to any combination of heterogeneous convolutional neural networks, and we believe that this can satisfy the personalized requirements of heterogeneous models. Experiments show that, compared to the state-of-the-art method (e.g., FLOP), in non-IID data sets our method compress \u2248\u200950<jats:italic>%<\/jats:italic> of the shared sub-model parameters with only a 4.38% drop in accuracy on SVHN dataset and on CIFAR-10, PerHeFed even achieves a 0.3% improvement in accuracy. To the best of our knowledge, our work is the first general personalized federated learning framework for heterogeneous convolutional networks, even cross different networks, addressing model structure unity in conventional federated learning.<\/jats:p>","DOI":"10.1007\/s11280-022-01119-x","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T09:03:21Z","timestamp":1670835801000},"page":"2027-2049","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["PerHeFed: A general framework of personalized federated learning for heterogeneous convolutional neural networks"],"prefix":"10.1007","volume":"26","author":[{"given":"Le","family":"Ma","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"YuYing","family":"Liao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bin","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wen","family":"Xi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,12,12]]},"reference":[{"key":"1119_CR1","unstructured":"Kone\u010dn\u00fd, J., McMahan, H.B., Yu, F.X., Richt\u00e1rik, P., Suresh, A.T., Bacon, D.: Federated learning: Strategies for improving communication efficiency. arXiv:abs\/1610.05492 (2016)"},{"key":"1119_CR2","unstructured":"Frankle, J., Carbin, M.: The lottery ticket hypothesis: Finding sparse, trainable neural networks. In: International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rJl-b3RcF7 (2019)"},{"key":"1119_CR3","unstructured":"McMahan, H. B., Moore, E., Ramage, D., Arcas, B. A.: Federated learning of deep networks using model averaging. arXiv:abs\/1602.05629 (2016)"},{"key":"1119_CR4","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., Arcas, B.A.y.: Communication-efficient learning of deep networks from decentralized data. In: Singh, A., Zhu, J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Proceedings of Machine Learning Research, vol. 54, pp. 1273\u20131282. PMLR. https:\/\/proceedings.mlr.press\/v54\/mcmahan17a.html (2017)"},{"key":"1119_CR5","unstructured":"Yang, Q., Liu, Y., Chen, T., Tong, Y.: Federated machine learning: Concept and applications. arXiv:abs\/1902.04885 (2019)"},{"key":"1119_CR6","doi-asserted-by":"publisher","unstructured":"Shokri, R., Stronati, M., Song, C., Shmatikov, V.: Membership inference attacks against machine learning models. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 3\u201318. https:\/\/doi.org\/10.1109\/SP.2017.41 (2017)","DOI":"10.1109\/SP.2017.41"},{"key":"1119_CR7","doi-asserted-by":"publisher","unstructured":"Melis, L., Song, C., De Cristofaro, E., Shmatikov, V.: Exploiting unintended feature leakage in collaborative learning, pp. 691\u2013706. https:\/\/doi.org\/10.1109\/SP.2019.00029 (2019)","DOI":"10.1109\/SP.2019.00029"},{"key":"1119_CR8","doi-asserted-by":"crossref","unstructured":"Hitaj, B., Ateniese, G., P\u00e9rez-cruz, F.: Deep models under the GAN: information leakage from collaborative deep learning. arXiv:abs\/1702.07464 (2017)","DOI":"10.1145\/3133956.3134012"},{"key":"1119_CR9","unstructured":"Collins, L., Hassani, H., Mokhtari, A., Shakkottai, S.: Exploiting shared representations for personalized federated learning. arXiv:abs\/2102.07078 (2021)"},{"key":"1119_CR10","unstructured":"Achituve, I., Shamsian, A., Navon, A., Chechik, G., Fetaya, E.: Personalized federated learning with gaussian processes. In: Ranzato, M., Beygelzimer, A., Dauphin, Y.N., Liang, P., Vaughan, J.W. (eds.) Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, Virtual, pp. 8392\u20138406 . https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/46d0671dd4117ea366031f87f3aa0093-Abstract.html (2021)"},{"key":"1119_CR11","doi-asserted-by":"publisher","first-page":"7865","DOI":"10.1609\/aaai.v35i9.16960","volume":"35","author":"H Yutao","year":"2021","unstructured":"Yutao, H., Chu, L., Zhou, Z., Wang, L., Liu, J., Pei, J., Zhang, Y.: Personalized cross-silo federated learning on non-iid data. Proceedings of the AAAI Conference on Artificial Intelligence 35, 7865\u20137873 (2021). https:\/\/doi.org\/10.1609\/aaai.v35i9.16960","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"1119_CR12","doi-asserted-by":"publisher","unstructured":"Kulkarni, V., Kulkarni, M., Pant, A.: Survey of personalization techniques for federated learning. In: 2020 Fourth World Conference on Smart Trends in Systems, Security and Sustainability (Worlds4), pp. 794\u2013797. https:\/\/doi.org\/10.1109\/WorldS450073.2020.9210355 (2020)","DOI":"10.1109\/WorldS450073.2020.9210355"},{"key":"1119_CR13","doi-asserted-by":"publisher","unstructured":"Yin, X., Zhu, Y., Hu, J.: A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions. ACM Comput. Surv 54(6). https:\/\/doi.org\/10.1145\/3460427 (2021)","DOI":"10.1145\/3460427"},{"key":"1119_CR14","unstructured":"Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., Shmatikov, V.: How to backdoor federated learning. In: Chiappa, S., Calandra, R. (eds.) The 23rd International Conference on Artificial Intelligence and Statistics, AISTATS 2020, 26-28 August 2020, Online [Palermo, Sicily, Italy]. Proceedings of Machine Learning Research, vol. 108, pp. 2938\u20132948. PMLR. http:\/\/proceedings.mlr.press\/v108\/bagdasaryan20a.html(2020)"},{"key":"1119_CR15","doi-asserted-by":"publisher","unstructured":"Zhao, C., Wen, Y., Li, S., Liu, F., Meng, D.: Federatedreverse: a Detection and Defense Method against Backdoor Attacks in Federated Learning. In: Borghys, D., Bas, P., Verdoliva, L., Pevn\u00fd, T., Li, B., Newman, J. (eds.) IH&MMSec \u201921: ACM Workshop on Information Hiding and Multimedia Security, Virtual Event, Belgium, June, 22-25, 2021, pp. 51\u201362. ACM. https:\/\/doi.org\/10.1145\/3437880.3460403 (2021)","DOI":"10.1145\/3437880.3460403"},{"key":"1119_CR16","doi-asserted-by":"publisher","unstructured":"Li, Y., Chu, S., Zhao, H., Rong, F., Liu, C., Zhao, S., Wang, Z., Xiong, Z.: Rapid prediction and evaluation of COVID-19 epidemic in the United States based on feature selection and improved ARIMAX model. In: ICAIIS 2021: 2021 2Nd International Conference on Artificial Intelligence and Information Systems, Chongqing, China, May 28 - 30, 2021, pp. 299\u201312998. ACM. https:\/\/doi.org\/10.1145\/3469213.3471327 (2021)","DOI":"10.1145\/3469213.3471327"},{"key":"1119_CR17","doi-asserted-by":"publisher","unstructured":"Tran, T.Q., Sakuma, J.: Seasonal-adjustment based feature selection method for predicting epidemic with large-scale search engine logs. In: Teredesai, A., Kumar, V., Li, Y., Rosales, R., Terzi, E., Karypis, G. (eds.) Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4-8, 2019, pp. 2857\u20132866. ACM. https:\/\/doi.org\/10.1145\/3292500.3330766 (2019)","DOI":"10.1145\/3292500.3330766"},{"key":"1119_CR18","doi-asserted-by":"publisher","unstructured":"Veron, M., Ghannay, S., Ligozat, A., Rosset, S.: Lifelong learning and task-oriented dialogue system: What does it mean?. In: Marchi, E., Siniscalchi, S.M., Cumani, S., Salerno, V.M., Li, H. (eds.) Increasing Naturalness and Flexibility in Spoken Dialogue Interaction - 10th International Workshop on Spoken Dialogue Systems, IWSDS 2019, Syracuse, Sicily, Italy, 24-26 April 2019. Lecture Notes in Electrical Engineering, vol. 714, pp. 347\u2013356. Springer. https:\/\/doi.org\/10.1007\/978-981-15-9323-9_32 (2019)","DOI":"10.1007\/978-981-15-9323-9_32"},{"key":"1119_CR19","doi-asserted-by":"publisher","unstructured":"Li, A., Sun, J., Li, P., Pu, Y., Li, H., Chen, Y.: Hermes: An efficient federated learning framework for heterogeneous mobile clients. In: ACM Mobicom \u201921: the 27Th Annual International Conference on Mobile Computing and Networking, New Orleans, Louisiana, USA, October 25-29, 2021, pp. 420\u2013437. ACM. https:\/\/doi.org\/10.1145\/3447993.3483278 (2021)","DOI":"10.1145\/3447993.3483278"},{"key":"1119_CR20","doi-asserted-by":"publisher","unstructured":"Yang, Q., Zhang, J., Hao, W., Spell, G.P., Carin, L.: FLOP: Federated learning on medical datasets using partial networks. In: Zhu, F., Ooi, B.C., Miao, C. (eds.) KDD \u201921: the 27Th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, Singapore, August 14-18, 2021, pp. 3845\u20133853. ACM. https:\/\/doi.org\/10.1145\/3447548.3467185 (2021)","DOI":"10.1145\/3447548.3467185"},{"issue":"1-2","key":"1119_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000083","volume":"14","author":"P Kairouz","year":"2021","unstructured":"Kairouz, P., McMahan, H.B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A.N., Bonawitz, K.A., Charles, Z., Cormode, G., Cummings, R., D\u2019Oliveira, R.G.L., Eichner, H., Rouayheb, S.E., Evans, D., Gardner, J., Garrett, Z., Gasc\u00f3n, A., Ghazi, B., Gibbons, P. B., Gruteser, M., Harchaoui, Z., He, C., He, L., Huo, Z., Hutchinson, B., Hsu, J., Jaggi, M., Javidi, T., Joshi, G., Khodak, M., Kone\u010dn\u00fd, J., Korolova, A., Koushanfar, F., Koyejo, S., Lepoint, T., Liu, Y., Mittal, P., Mohri, M., Nock, R., \u00d6zg\u00fcr, A., Pagh, R., Qi, H., Ramage, D., Raskar, R., Raykova, M., Song, D., Song, W., Stich, S.U., Sun, Z., Suresh, A.T., Tram\u00e8r, F., Vepakomma, P., Wang, J., Xiong, L., Xu, Z., Yang, Q., Yu, F.X., Yu, H., Zhao, S.: Advances and open problems in federated learning. Found. Trends Mach. Learn. 14 (1-2), 1\u2013210 (2021). https:\/\/doi.org\/10.1561\/2200000083","journal-title":"Found. Trends Mach. Learn."},{"key":"1119_CR22","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1109\/OJCS.2020.2993259","volume":"1","author":"Q Wu","year":"2020","unstructured":"Wu, Q., He, K., Chen, X.: Personalized federated learning for intelligent iot applications: A cloud-edge based framework. IEEE Open J. Comput. Soc. 1, 35\u201344 (2020). https:\/\/doi.org\/10.1109\/OJCS.2020.2993259","journal-title":"IEEE Open J. Comput. Soc."},{"issue":"23","key":"1119_CR23","doi-asserted-by":"publisher","first-page":"16853","DOI":"10.1109\/JIOT.2020.3033806","volume":"8","author":"WYB Lim","year":"2021","unstructured":"Lim, W.Y.B., Garg, S., Xiong, Z., Niyato, D., Leung, C., Miao, C., Guizani, M.: Dynamic contract design for federated learning in smart healthcare applications. IEEE Internet Things J. 8(23), 16853\u201316862 (2021). https:\/\/doi.org\/10.1109\/JIOT.2020.3033806","journal-title":"IEEE Internet Things J."},{"issue":"1","key":"1119_CR24","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1109\/TII.2018.2843169","volume":"15","author":"P Pace","year":"2019","unstructured":"Pace, P., Aloi, G., Gravina, R., Caliciuri, G., Fortino, G., Liotta, A.: An edge-based architecture to support efficient applications for healthcare industry 4.0. IEEE Trans. Ind. Inform. 15(1), 481\u2013489 (2019). https:\/\/doi.org\/10.1109\/TII.2018.2843169","journal-title":"IEEE Trans. Ind. Inform."},{"key":"1119_CR25","doi-asserted-by":"publisher","unstructured":"Yan, Z., Yi, Y.Z., Zhang, J., Zhao, N., Ren, Y., Wan, J., Yu, J.: Federated learning model training method based on data features perception aggregation. In: 94Th IEEE Vehicular Technology Conference, VTC Fall 2021, Norman, OK, USA, September 27-30, 2021, pp. 1\u20137. IEEE. https:\/\/doi.org\/10.1109\/VTC2021-Fall52928.2021.9625291(2021)","DOI":"10.1109\/VTC2021-Fall52928.2021.9625291"},{"key":"1119_CR26","unstructured":"Chen, F., Dong, Z., Li, Z., He, X.: Federated meta-learning for recommendation. arXiv:abs\/1802.07876 (2018)"},{"key":"1119_CR27","doi-asserted-by":"publisher","unstructured":"Wang, L., Wang, Y., Bai, Y., Liu, P., Li, X.: POI Recommendation with Federated Learning and Privacy Preserving in Cross Domain Recommendation. In: 2021 IEEE Conference on Computer Communications Workshops, INFOCOM Workshops 2021, Vancouver, BC, Canada, May 10-13, 2021, pp. 1\u20136. IEEE. https:\/\/doi.org\/10.1109\/INFOCOMWKSHPS51825.2021.9484510 (2021)","DOI":"10.1109\/INFOCOMWKSHPS51825.2021.9484510"},{"key":"1119_CR28","unstructured":"Vanhaesebrouck, P., Bellet, A., Tommasi, M.: Decentralized collaborative learning of personalized models over networks. In: Singh, A., Zhu, X.J. (eds.) Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Lauderdale, FL, USA. Proceedings of Machine Learning Research, vol. 54, pp. 509\u2013517. PMLR. http:\/\/proceedings.mlr.press\/v54\/vanhaesebrouck17a.html (2017)"},{"key":"1119_CR29","unstructured":"Bellet, A., Guerraoui, R., Taziki, M., Tommasi, M.: Personalized and private peer-to-peer machine learning. In: Storkey, A.J., P\u00e9rez-Cruz, F. (eds.) International Conference on Artificial Intelligence and Statistics, AISTATS 2018, 9-11 April 2018, Playa Blanca, Lanzarote, Canary Islands, Spain. Proceedings of Machine Learning Research, vol. 84, pp. 473\u2013481. PMLR. http:\/\/proceedings.mlr.press\/v84\/bellet18a.html(2018)"},{"key":"1119_CR30","unstructured":"Diao, E., Ding, J., Tarokh, V.: Heterofl: Computation and communication efficient federated learning for heterogeneous clients. In: 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. https:\/\/openreview.net\/forum?id=TNkPBBYFkXg (2021)"},{"issue":"8","key":"1119_CR31","doi-asserted-by":"publisher","first-page":"6058","DOI":"10.1109\/JIOT.2021.3110908","volume":"9","author":"X Lu","year":"2022","unstructured":"Lu, X., Liao, Y., Liu, C., Li\u00f2, P., Hui, P.: Heterogeneous model fusion federated learning mechanism based on model mapping. IEEE Internet Things J. 9(8), 6058\u20136068 (2022). https:\/\/doi.org\/10.1109\/JIOT.2021.3110908","journal-title":"IEEE Internet Things J."},{"key":"1119_CR32","unstructured":"Arivazhagan, M.G., Aggarwal, V., Singh, A.K., Choudhary, S.: Federated learning with personalization layers. arXiv:abs\/1912.00818 (2019)"},{"key":"1119_CR33","unstructured":"Lin, T., Kong, L., Stich, S.U., Jaggi, M.: Ensemble distillation for robust model fusion in federated learning. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, Virtual. https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/18df51b97ccd68128e994804f3eccc87-Abstract.html (2020)"},{"key":"1119_CR34","unstructured":"Li, D., Wang, J.: Fedmd: Heterogenous federated learning via model distillation. arXiv:abs\/abs\/1910.03581 (2019)"},{"key":"1119_CR35","unstructured":"Hinton, G.E., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv:abs\/1503.02531 (2015)"},{"key":"1119_CR36","unstructured":"Krizhevsky, A.: Learning multiple layers of features from tiny images (2009)"},{"key":"1119_CR37","unstructured":"Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.: Reading digits in natural images with unsupervised feature learning (2011)"},{"key":"1119_CR38","doi-asserted-by":"publisher","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016, pp. 770\u2013778. IEEE Computer Society. https:\/\/doi.org\/10.1109\/CVPR.2016.90 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"1119_CR39","doi-asserted-by":"publisher","unstructured":"Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pp. 2261\u20132269. IEEE Computer Society. https:\/\/doi.org\/10.1109\/CVPR.2017.243 (2017)","DOI":"10.1109\/CVPR.2017.243"}],"container-title":["World Wide Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11280-022-01119-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11280-022-01119-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11280-022-01119-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T14:45:30Z","timestamp":1690382730000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11280-022-01119-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,12]]},"references-count":39,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["1119"],"URL":"https:\/\/doi.org\/10.1007\/s11280-022-01119-x","relation":{},"ISSN":["1386-145X","1573-1413"],"issn-type":[{"value":"1386-145X","type":"print"},{"value":"1573-1413","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,12]]},"assertion":[{"value":"22 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 October 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 November 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 December 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}}]}}