{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T03:39:53Z","timestamp":1771904393161,"version":"3.50.1"},"reference-count":64,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T00:00:00Z","timestamp":1716854400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T00:00:00Z","timestamp":1716854400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In this paper, a framework based on a sparse Mixture of Experts (MoE) architecture is proposed for the federated learning and application of a distributed classification model in domains (like cybersecurity and healthcare) where different parties of the federation store different subsets of features for a number of data instances. The framework is designed to limit the risk of information leakage and computation\/communication costs in both model training (through data sampling) and application (leveraging the conditional-computation abilities of sparse MoEs). Experiments on real data have shown the proposed approach to ensure a better balance between efficiency and model accuracy, compared to other VFL-based solutions. Notably, in a real-life cybersecurity case study focused on malware classification (the KronoDroid dataset), the proposed method surpasses competitors even though it utilizes only 50% and 75% of the training set, which is fully utilized by the other approaches in the competition. This method achieves reductions in the rate of false positives by 16.9% and 18.2%, respectively, and also delivers satisfactory results on the other evaluation metrics. These results showcase our framework\u2019s potential to significantly enhance cybersecurity threat detection and prevention in a collaborative yet secure manner.<\/jats:p>","DOI":"10.1186\/s40537-024-00933-6","type":"journal-article","created":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T21:06:19Z","timestamp":1716930379000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Efficiently approaching vertical federated learning by combining data reduction and conditional computation techniques"],"prefix":"10.1186","volume":"11","author":[{"given":"Francesco","family":"Folino","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gianluigi","family":"Folino","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesco Sergio","family":"Pisani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luigi","family":"Pontieri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pietro","family":"Sabatino","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,5,28]]},"reference":[{"key":"933_CR1","unstructured":"Yousefpour A, et al. Green federated learning. arXiv preprint arXiv:2303.14604 2023."},{"key":"933_CR2","unstructured":"Huba D, other: Papaya: practical, private, and scalable federated learning. arxiv:2111.04877 2021."},{"key":"933_CR3","unstructured":"Wu C-J, et al. Sustainable AI: environmental implications, challenges and opportunities. CoRR abs\/2111.00364 2022."},{"issue":"129","key":"933_CR4","first-page":"1","volume":"24","author":"X Qiu","year":"2023","unstructured":"Qiu X. A first look into the carbon footprint of federated learning. Jo Mach Learning Res. 2023;24(129):1\u201323.","journal-title":"Jo Mach Learning Res"},{"key":"933_CR5","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1186\/s40537-021-00419-9","volume":"8","author":"A Adadi","year":"2021","unstructured":"Adadi A. A survey on data-efficient algorithms in big data era. J Big Data. 2021;8:24.","journal-title":"J Big Data"},{"key":"933_CR6","doi-asserted-by":"crossref","unstructured":"Albelaihi R, Yu L, Craft WD, Sun X, Wang C, Gazda R. Green federated learning via energy-aware client selection. In: GLOBECOM 2022-2022 IEEE Global Communications Conference, 2022;13\u201318. IEEE","DOI":"10.1109\/GLOBECOM48099.2022.10001569"},{"key":"933_CR7","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2023.101804","volume":"92","author":"F De Rango","year":"2023","unstructured":"De Rango F, Guerrieri A, Raimondo P, Spezzano G. Hed-fl: A hierarchical, energy efficient, and dynamic approach for edge federated learning. Pervasive Mobile Comput. 2023;92: 101804.","journal-title":"Pervasive Mobile Comput"},{"issue":"1","key":"933_CR8","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","volume":"3","author":"RA Jacobs","year":"1991","unstructured":"Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE. Adaptive mixtures of local experts. Neural Comput. 1991;3(1):79\u201387.","journal-title":"Neural Comput"},{"key":"933_CR9","unstructured":"Bengio Y, L\u00e9onard N, Courville A. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432 2013."},{"key":"933_CR10","unstructured":"Shazeer N, Mirhoseini A, Maziarz K, Davis A, Le Q, Hinton G, Dean J. Outrageously large neural networks: the sparsely-gated mixture-of-experts layer. In: ICLR, 2017;1\u201317."},{"key":"933_CR11","unstructured":"Lepikhin D, et al. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668 2021."},{"issue":"1","key":"933_CR12","first-page":"5232","volume":"23","author":"W Fedus","year":"2022","unstructured":"Fedus W, Zoph B, Shazeer N. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. J Mach Learn Res. 2022;23(1):5232\u201370.","journal-title":"J Mach Learn Res"},{"issue":"12","key":"933_CR13","doi-asserted-by":"publisher","first-page":"4006","DOI":"10.1109\/TPDS.2022.3178443","volume":"33","author":"J Zhang","year":"2022","unstructured":"Zhang J, Guo S, Qu Z, Zeng D, Wang H, Liu Q, Zomaya AY. Adaptive vertical federated learning on unbalanced features. IEEE Trans Parallel Distribut Syst. 2022;33(12):4006\u201318.","journal-title":"IEEE Trans Parallel Distribut Syst"},{"key":"933_CR14","doi-asserted-by":"crossref","unstructured":"Tran NH. Federated learning over wireless networks: optimization model design and analysis. In: Proc. of IEEE conference on computer communications (INFOCOM), 2019;1387\u20131395.","DOI":"10.1109\/INFOCOM.2019.8737464"},{"issue":"3","key":"933_CR15","doi-asserted-by":"publisher","first-page":"1935","DOI":"10.1109\/TWC.2020.3037554","volume":"20","author":"Z Yang","year":"2020","unstructured":"Yang Z. Energy efficient federated learning over wireless communication networks. IEEE Trans Wireless Commun. 2020;20(3):1935\u201349.","journal-title":"IEEE Trans Wireless Commun"},{"issue":"3","key":"933_CR16","doi-asserted-by":"publisher","first-page":"2120","DOI":"10.1109\/TWC.2020.3039309","volume":"20","author":"G Zhu","year":"2020","unstructured":"Zhu G, Du Y, G\u00fcnd\u00fcz D, Huang K. One-bit over-the-air aggregation for communication-efficient federated edge learning: design and convergence analysis. IEEE Trans Wireless Commun. 2020;20(3):2120\u201335.","journal-title":"IEEE Trans Wireless Commun"},{"issue":"9","key":"933_CR17","doi-asserted-by":"publisher","first-page":"5902","DOI":"10.1109\/TCOMM.2021.3087125","volume":"69","author":"C Feng","year":"2021","unstructured":"Feng C. On the design of federated learning in the mobile edge computing systems. IEEE Trans Commun. 2021;69(9):5902\u201316.","journal-title":"IEEE Trans Commun"},{"issue":"8","key":"933_CR18","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1631\/FITEE.2100538","volume":"23","author":"P Liu","year":"2022","unstructured":"Liu P. Training time minimization for federated edge learning with optimized gradient quantization and bandwidth allocation. Front Inf Technol Electron Eng. 2022;23(8):1247\u201363.","journal-title":"Front Inf Technol Electron Eng"},{"key":"933_CR19","doi-asserted-by":"crossref","unstructured":"Luo B. Cost-effective federated learning design. In: proc. of IEEE conference on computer communications (INFOCOM), 2021;1\u201310.","DOI":"10.1109\/INFOCOM42981.2021.9488679"},{"issue":"12","key":"933_CR20","doi-asserted-by":"publisher","first-page":"7947","DOI":"10.1109\/TWC.2021.3088910","volume":"20","author":"Q Zeng","year":"2021","unstructured":"Zeng Q, Du Y, Huang K, Leung KK. Energy-efficient resource management for federated edge learning with cpu-gpu heterogeneous computing. IEEE Trans Wireless Commun. 2021;20(12):7947\u201362.","journal-title":"IEEE Trans Wireless Commun"},{"key":"933_CR21","doi-asserted-by":"crossref","unstructured":"Kim YG, Wu C-J. Fedgpo: Heterogeneity-aware global parameter optimization for efficient federated learning. In: Proc of 2022 IEEE Intl. Symp. on workload characterization (IISWC), 2022;117\u2013129.","DOI":"10.1109\/IISWC55918.2022.00020"},{"key":"933_CR22","doi-asserted-by":"crossref","unstructured":"Kim YG, Wu C-J. Autofl: enabling heterogeneity-aware energy efficient federated learning. In: Proc of 54th Annual IEEE\/ACM Intl. Symp. on Microarchitecture, 2021;183\u2013198.","DOI":"10.1145\/3466752.3480129"},{"key":"933_CR23","doi-asserted-by":"crossref","unstructured":"Abdelmoniem AM, Sahu AN, Canini M, Fahmy SA. Refl: resource-efficient federated learning. In: Proc. of 18th Europ. Conf. on Computer Systems, 2023;215\u2013232.","DOI":"10.1145\/3552326.3567485"},{"key":"933_CR24","doi-asserted-by":"crossref","unstructured":"Kim M, Saad W, Mozaffari M, Debbah M. Green, quantized federated learning over wireless networks: an energy-efficient design. IEEE transactions on wireless communications 2023.","DOI":"10.1109\/TWC.2023.3289177"},{"key":"933_CR25","doi-asserted-by":"crossref","unstructured":"Xu R. Fedv: Privacy-preserving federated learning over vertically partitioned data. In: Proc. of 14th ACM Workshop on artificial intelligence and security (AISec), New York, NY, USA, 2021;181\u2013192.","DOI":"10.1145\/3474369.3486872"},{"key":"933_CR26","unstructured":"Peterson DW, Kanani P, Marathe VJ. Private federated learning with domain adaptation. CoRR abs\/1912.06733 2019. arXiv:1912.06733"},{"key":"933_CR27","doi-asserted-by":"crossref","unstructured":"Guo B. Pfl-moe: personalized federated learning based on mixture of experts. In: Web and Big Data, pp. 480\u2013486. Springer, Cham 2021.","DOI":"10.1007\/978-3-030-85896-4_37"},{"key":"933_CR28","unstructured":"Reisser M, Louizos C, Gavves E, Welling M. Federated mixture of experts. arXiv preprint arXiv:2107.06724 2021."},{"key":"933_CR29","unstructured":"Zec EL, Mogren O, Martinsson J, S\u00fctfeld LR, Gillblad D. Specialized federated learning using a mixture of experts. arXiv preprint arXiv:2010.02056 2020."},{"key":"933_CR30","doi-asserted-by":"crossref","unstructured":"Folino F, Folino G, Pisani FS, Pontieri L, Sabatino P. A scalable vertical federated learning framework for analytics in the cybersecurity domain. In: Proc. of 32nd Euromicro Intl. Conf. on parallel, distributed, and network-based processing (PDP), p. 2024.","DOI":"10.1109\/PDP62718.2024.00041"},{"key":"933_CR31","doi-asserted-by":"crossref","unstructured":"Dwork C. Differential privacy: a survey of results. In: Proc. of Intl. Conf. on theory and applications of models of computation, 2008;1\u201319.","DOI":"10.1007\/978-3-540-79228-4_1"},{"key":"933_CR32","doi-asserted-by":"crossref","unstructured":"Gentry C. Fully homomorphic encryption using ideal lattices. In: Proc. of 41st ACM Sympo. on Theory of Computing, 2009;169\u2013178.","DOI":"10.1145\/1536414.1536440"},{"key":"933_CR33","unstructured":"Okanovic P, et al. Repeated random sampling for minimizing the time-to-accuracy of learning. arXiv preprint arXiv:2305.18424 2023."},{"key":"933_CR34","doi-asserted-by":"crossref","unstructured":"Hellmeier M, Pampus J, Qarawlus H, Howar F. Implementing data sovereignty: Requirements & challenges from practice. Proceedings of the 18th international conference on availability, reliability and security 2023.","DOI":"10.1145\/3600160.3604995"},{"issue":"1","key":"933_CR35","doi-asserted-by":"publisher","first-page":"205395172098201","DOI":"10.1177\/2053951720982012","volume":"8","author":"P Hummel","year":"2021","unstructured":"Hummel P, Braun M, Tretter M, Dabrock P. Data sovereignty: a review. Big Data Soc. 2021;8(1):2053951720982012.","journal-title":"Big Data Soc"},{"issue":"1","key":"933_CR36","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1109\/MCC.2016.18","volume":"3","author":"C Esposito","year":"2016","unstructured":"Esposito C, Castiglione A, Choo K-KR. Encryption-based solution for data sovereignty in federated clouds. IEEE Cloud Comput. 2016;3(1):12\u20137.","journal-title":"IEEE Cloud Comput"},{"issue":"1","key":"933_CR37","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1007\/s10207-021-00542-x","volume":"21","author":"M Sheikhalishahi","year":"2022","unstructured":"Sheikhalishahi M, Saracino A, Martinelli F, Marra AL. Privacy preserving data sharing and analysis for edge-based architectures. Int J Inf Sec. 2022;21(1):79\u2013101.","journal-title":"Int J Inf Sec"},{"issue":"2","key":"933_CR38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3298981","volume":"10","author":"Q Yang","year":"2019","unstructured":"Yang Q, Liu Y, Chen T, Tong Y. Federated machine learning: concept and applications. ACM Trans Intell Syst Technol. 2019;10(2):1\u201319.","journal-title":"ACM Trans Intell Syst Technol"},{"key":"933_CR39","unstructured":"Yang L, et al. Vertical federated learning: concepts, advances and challenges. arXiv preprint arXiv:2211.12814v4 2023."},{"key":"933_CR40","unstructured":"Romanini D, et al. PyVertical: a vertical federated learning framework for multi-headed SplitNN 2021."},{"key":"933_CR41","unstructured":"Kaplan J, et al. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 2020."},{"key":"933_CR42","unstructured":"Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. In: Proc. of 27th Intl. Conf. on neural information processing systems - 2014;2:3104\u20133112."},{"issue":"6","key":"933_CR43","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","volume":"60","author":"A Krizhevsky","year":"2017","unstructured":"Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Commun ACM. 2017;60(6):84\u201390.","journal-title":"Commun ACM"},{"issue":"6","key":"933_CR44","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","volume":"29","author":"G Hinton","year":"2012","unstructured":"Hinton G. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag. 2012;29(6):82\u201397.","journal-title":"IEEE Signal Process Mag"},{"issue":"2","key":"933_CR45","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1162\/neco.1994.6.2.181","volume":"6","author":"MI Jordan","year":"1994","unstructured":"Jordan MI, Jacobs RA. Hierarchical mixtures of experts and the em algorithm. Neural Comput. 1994;6(2):181\u2013214.","journal-title":"Neural Comput"},{"key":"933_CR46","unstructured":"Han S, Mao H, Dally WJ. Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 2015."},{"key":"933_CR47","unstructured":"Eigen D, Ranzato M, Sutskever I. Learning factored representations in a deep mixture of experts. arXiv preprint arXiv:1312.4314 2013."},{"key":"933_CR48","unstructured":"Roller S, Sukhbaatar S, Weston J. Hash layers for large sparse models. In: advances in neural information processing systems, 2021;34:17555\u201317566."},{"key":"933_CR49","unstructured":"Liu L, Gao J, Chen W. Sparse backpropagation for MoE training. In: arXiv:2310.00811 [cs] 2023."},{"key":"933_CR50","unstructured":"Ismail AA, Arik S\u00d6, Yoon J, Taly A, Feizi S, Pfister T. Interpretable mixture of experts for structured data. arXiv preprint arXiv:2206.02107 2022."},{"issue":"1","key":"933_CR51","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","volume":"3","author":"RA Jacobs","year":"1991","unstructured":"Jacobs RA, Jordan MI, Nowlan SJ, Hinton GE. Adaptive mixtures of local experts. Neural Comput. 1991;3(1):79\u201387.","journal-title":"Neural Comput"},{"key":"933_CR52","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2021.102399","volume":"110","author":"A Guerra-Manzanares","year":"2021","unstructured":"Guerra-Manzanares A, Bahsi H, N\u00f5mm S. Kronodroid: time-based hybrid-featured dataset for effective android malware detection and characterization. Comput Sec. 2021;110: 102399.","journal-title":"Comput Sec"},{"key":"933_CR53","doi-asserted-by":"crossref","unstructured":"Platt JC. Fast training of support vector machines using sequential minimal optimization 1998.","DOI":"10.7551\/mitpress\/1130.003.0016"},{"issue":"12","key":"933_CR54","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/3381831","volume":"63","author":"R Schwartz","year":"2020","unstructured":"Schwartz R, Dodge J, Smith NA, Etzioni O. Green AI. Commun ACM. 2020;63(12):54\u201363.","journal-title":"Commun ACM"},{"issue":"2","key":"933_CR55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3485128","volume":"55","author":"D Rolnick","year":"2022","unstructured":"Rolnick D. Tackling climate change with machine learning. ACM Comput Surv (CSUR). 2022;55(2):1\u201396.","journal-title":"ACM Comput Surv (CSUR)"},{"key":"933_CR56","doi-asserted-by":"crossref","unstructured":"Sevilla J, Compute trends across three eras of machine learning. In: Proc. of 2022 Intl. Joint conference on neural networks (IJCNN), 2022;1\u20138.","DOI":"10.1109\/IJCNN55064.2022.9891914"},{"key":"933_CR57","unstructured":"Vogels T, Karimireddy SP, Jaggi, M. Powersgd: Practical low-rank gradient compression for distributed optimization. Adv Neural Inf Process Syst 2019;32."},{"key":"933_CR58","unstructured":"Rothchild D. Fetchsgd: Communication-efficient federated learning with sketching. In: Proc. of Intl. Conf. on Machine Learning (ICML), 2020;8253\u20138265."},{"key":"933_CR59","unstructured":"Chen T, Jin X, Sun Y, Yin W. Vafl: a method of vertical asynchronous federated learning. arXiv preprint arXiv:2007.06081 2020."},{"key":"933_CR60","doi-asserted-by":"publisher","first-page":"4277","DOI":"10.1109\/TSP.2022.3198176","volume":"70","author":"Y Liu","year":"2022","unstructured":"Liu Y. Fedbcd: a communication-efficient collaborative learning framework for distributed features. IEEE Trans Signal Process. 2022;70:4277\u201390.","journal-title":"IEEE Trans Signal Process"},{"issue":"13","key":"933_CR61","doi-asserted-by":"publisher","first-page":"10922","DOI":"10.1109\/JIOT.2021.3051382","volume":"8","author":"L Su","year":"2021","unstructured":"Su L, Lau VKN. Hierarchical federated learning for hybrid data partitioning across multitype sensors. IEEE Int Things J. 2021;8(13):10922\u201339.","journal-title":"IEEE Int Things J"},{"key":"933_CR62","unstructured":"Parsaeefard S, Etesami SE, Leon-Garcia A. Robust federated learning by mixture of experts. CoRR abs\/2104.11700 2021. arXiv:2104.11700"},{"key":"933_CR63","unstructured":"Liu L, Dong C, Liu X, Yu B, Gao J. Bridging discrete and backpropagation: Straight-through and beyond. In: Proc. of 36th Intl. Conf. on Advances in Neural Information Processing Systems 2024."},{"key":"933_CR64","unstructured":"Kool W, Maddison CJ, Mnih A. Unbiased gradient estimation with balanced assignments for mixtures of experts. 2021."}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-024-00933-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40537-024-00933-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-024-00933-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,28]],"date-time":"2024-05-28T21:12:05Z","timestamp":1716930725000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-024-00933-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,28]]},"references-count":64,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["933"],"URL":"https:\/\/doi.org\/10.1186\/s40537-024-00933-6","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,28]]},"assertion":[{"value":"31 January 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"77"}}