{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:35:12Z","timestamp":1783010112901,"version":"3.54.5"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,9,22]],"date-time":"2022-09-22T00:00:00Z","timestamp":1663804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Union\u2019s Horizon 2020 research and innovation programme","award":["824988"],"award-info":[{"award-number":["824988"]}]},{"name":"FEDER\/Ministerio de Ciencia, Innovaci\u00f3n y Universidades - Agencia Estatal de Investigaci\u00f3n","award":["TEC2017-83838-R"],"award-info":[{"award-number":["TEC2017-83838-R"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2022,12,31]]},"abstract":"<jats:p>In recent years, there has been remarkable growth in Federated Learning (FL) approaches because they have proven to be very effective in training large Machine Learning (ML) models and also serve to preserve data confidentiality, as recommended by the GDPR or other business confidentiality restrictions that may apply. Despite the success of FL, performance is greatly reduced when data is not distributed identically (non-ID) across participants, as local model updates tend to diverge from the optimal global solution and thus the model averaging procedure in the aggregator is less effective. Kernel methods such as Support Vector Machines (SVMs) have not seen an equivalent evolution in the area of privacy preserving edge computing because they suffer from inherent computational, privacy and scalability issues. Furthermore, non-linear SVMs do not naturally lead to federated schemes, since locally trained models cannot be passed to the aggregator because they reveal training data (they are built on Support Vectors), and the global model cannot be updated at every worker using gradient descent. In this article, we explore the use of a particular controlled complexity (\u201cBudget\u201d) Distributed SVM (BDSVM) in the FL scenario with non-ID data, which is the least favorable situation, but very common in practice. The proposed BDSVM algorithm is as follows: model weights are broadcasted to workers, which locally update some kernel Gram matrices computed according to a common architectural base and send them back to the aggregator, which finally combines them, updates the global model, and repeats the procedure until a convergence criterion is met. Experimental results using synthetic 2D datasets show that the proposed method can obtain maximal margin decision boundaries even when the data is non-ID distributed. Further experiments using real-world datasets with non-ID data distribution show that the proposed algorithm provides better performance with less communication requirements than a comparable Multilayer Perceptron (MLP) trained using FedAvg. The advantage is more remarkable for a larger number of edge devices. We have also demonstrated the robustness of the proposed method against information leakage, membership inference attacks, and situations with dropout or straggler participants. Finally, in experiments run on separate processes\/machines interconnected via the cloud messaging service developed in the context of the EU-H2020 MUSKETEER project, BDSVM is able to train better models than FedAvg in about half the time.<\/jats:p>","DOI":"10.1145\/3539734","type":"journal-article","created":{"date-parts":[[2022,5,31]],"date-time":"2022-05-31T07:36:51Z","timestamp":1653982611000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Budget Distributed Support Vector Machine for Non-ID Federated Learning Scenarios"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6453-2378","authenticated-orcid":false,"given":"A.","family":"Navia-V\u00e1zquez","sequence":"first","affiliation":[{"name":"Universidad Carlos III of Madrid, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6203-3655","authenticated-orcid":false,"given":"R.","family":"D\u00edaz-Morales","sequence":"additional","affiliation":[{"name":"Tree Technology, Llanera, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9326-0386","authenticated-orcid":false,"given":"M.","family":"Fern\u00e1ndez-D\u00edaz","sequence":"additional","affiliation":[{"name":"Tree Technology, Llanera, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,9,22]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Technologies and Applications for Big Data Value","author":"Bonura S.","year":"2020","unstructured":"S. Bonura, D. Dalle Carbonare, R. D\u00edaz-Morales, M. Fern\u00e1ndez-D\u00edaz, L. Morabito, L. Mu\u00f1oz-Gonz\u00e1lez, C. Napione, A. Navia-V\u00e1zquez, and M. Purcell. 2020. Privacy preserving technologies for trusted data spaces. Technologies and Applications for Big Data Value. Big Data Value Association (BDVA)."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2018.01.007"},{"key":"e_1_3_2_4_2","volume-title":"Proc. 20th National Conference on Artificial Intelligence","author":"Caragea C.","year":"2005","unstructured":"C. Caragea, D. Caragea, and V. Honavar. 2005. Learning support vector machine classifiers from distributed data sources. In Proc. 20th National Conference on Artificial Intelligence."},{"key":"e_1_3_2_5_2","article-title":"Privacy-Preserved Federated Learning: A Survey of Applicable Machine Learning Algorithms in a Federated Environment","author":"Carlsson R.","year":"2020","unstructured":"R. Carlsson. 2020. Privacy-Preserved Federated Learning: A Survey of Applicable Machine Learning Algorithms in a Federated Environment. Ph.D. Dissertation, Uppsala University.","journal-title":"Ph.D. Dissertation, Uppsala University"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1162\/089976602753633402"},{"key":"e_1_3_2_7_2","unstructured":"Council of European Union. 2014. Council regulation (EU) no. 269\/2014. http:\/\/eur-lex.europa.eu\/legal-content\/EN\/TXT\/?qid=1416170084502&uri=CELEX:32014R0269."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781107337756"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.2018.2858778"},{"key":"e_1_3_2_10_2","first-page":"59","volume-title":"Proc. International Conference on Research, Innovation and Vision for the Future","author":"Do T. N.","year":"2006","unstructured":"T. N. Do and F. Poulet. 2006. Classifying one billion data with a new distributed SVM algorithm. In Proc. International Conference on Research, Innovation and Vision for the Future. 59\u201366."},{"key":"e_1_3_2_11_2","unstructured":"D. Dua and C. Graff. 2019. MUCI Machine Learning Repository University of California School of Information and Computer Science Irvine CA."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-5906-5_752"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1162\/0899766052530875"},{"key":"e_1_3_2_14_2","volume-title":"Proc. EUSIPCO\u201908","author":"Flouri K.","year":"2006","unstructured":"K. Flouri, B. Beferull-Lozano, and P. Tsakalides. 2006. Distributed consensus algorithms for SVM training in wireless sensor networks. In Proc. EUSIPCO\u201908."},{"key":"e_1_3_2_15_2","volume-title":"Proc. of the 14th European Signal Processing Conference","author":"Flouri K.","year":"2006","unstructured":"K. Flouri, B. Beferull-Lozano, and P. Tsakalides. 2006. Training a support-vector machine-based classifier in distributed sensor networks. In Proc. of the 14th European Signal Processing Conference."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1791212.1791218"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1859906"},{"key":"e_1_3_2_18_2","first-page":"521","volume-title":"Advances in Neural Information Processing Systems","author":"Graf H. P.","year":"2005","unstructured":"H. P. Graf, E. Cosatto, L. Bottou, I. Dourdanovic, and V. Vapnik. 2005. Parallel support vector machines: The cascade SVM. Advances in Neural Information Processing Systems, Vol. 17. 521\u2013528."},{"key":"e_1_3_2_19_2","volume-title":"Proc. International Conference on Machine Learning (ICML\u201916)","author":"Hardt M.","year":"2016","unstructured":"M. Hardt, B. Recht, and Y. Singer. 2016. Train faster, generalize better: Stability of stochastic gradient descent. In Proc. International Conference on Machine Learning (ICML\u201916)."},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412051"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2008.4587354"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2021\/8261663","article-title":"Federated learning: A distributed shared machine learning method","author":"Hu K.","year":"2021","unstructured":"K. Hu, Y. Li, M. Xia, J. Wu, M. Lu, S. Zhang, and L. Weng. 2021. Federated learning: A distributed shared machine learning method. Complexity (2021), 1\u201320.","journal-title":"Complexity"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12559-015-9333-0"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2005.12.126"},{"key":"e_1_3_2_25_2","article-title":"Advances and open problems in federated learning","author":"Kairouz P.","year":"2019","unstructured":"P. Kairouz, H. B. McMahan, B. Avent, A. Bellet, M. Bennis, A. N. Bhagoji, K. Bonawitz, Z. Charles, G. Cormode, R. Cummings, et\u00a0al. 2019. Advances and open problems in federated learning. arXiv:1912.04977. https:\/\/arxiv.org\/pdf\/1912.04977.pdf.","journal-title":"arXiv:1912.04977"},{"key":"e_1_3_2_26_2","first-page":"5132","volume-title":"International Conference on Machine Learning","author":"Karimireddy S. P.","year":"2020","unstructured":"S. P. Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh. 2020. SCAFFOLD: Stochastic controlled averaging for federated learning. In International Conference on Machine Learning. PMLR, 5132\u20135143."},{"key":"e_1_3_2_27_2","unstructured":"J. Kone\u010dn\u00fd H. B. McMahan D. Ramage and P. Richt\u00e1rik. 2016. Federated optimization: Distributed machine learning for on-device intelligence.arxiv:1610.02527 [cs.LG]. https:\/\/arxiv.org\/pdf\/1610.02527.pdf."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_2_29_2","article-title":"Federated optimization in heterogeneous networks","author":"Li T.","year":"2018","unstructured":"T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith. 2018. Federated optimization in heterogeneous networks. arXiv:1812.06127. https:\/\/arxiv.org\/abs\/1812.06127.","journal-title":"arXiv:1812.06127"},{"key":"e_1_3_2_30_2","article-title":"FedBN: Federated learning on non-IID features via local batch normalization","author":"Li X.","year":"2021","unstructured":"X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou. 2021. FedBN: Federated learning on non-IID features via local batch normalization. arXiv:2102.07623. https:\/\/arxiv.org\/abs\/2102.07623.","journal-title":"arXiv:2102.07623"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1409620.1409622"},{"key":"e_1_3_2_32_2","article-title":"Communication-efficient learning of deep networks from decentralized data","author":"McMahan H. B.","year":"2017","unstructured":"H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Ag\u00fcera-Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. arXiv:1602.05629v2. https:\/\/arxiv.org\/pdf\/1602.05629.pdf.","journal-title":"arXiv:1602.05629v2"},{"key":"e_1_3_2_33_2","first-page":"492","volume-title":"Proc. International Conference for Internet Technology and Secured Transactions (ICITST\u201912)","author":"Mohammad R.","year":"2012","unstructured":"R. Mohammad, F. Thabtah, and T. Mccluskey. 2012. An assessment of features related to phishing websites using an automated technique. In Proc. International Conference for Internet Technology and Secured Transactions (ICITST\u201912). 492\u2013497."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2006.875968"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/72.950134"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(02)00351-5"},{"key":"e_1_3_2_37_2","volume-title":"Proc. NIPS 2011 Workshop on Big Learning: Algorithms, Systems and Tools for Learning at Scale","author":"Pechyony D.","year":"2011","unstructured":"D. Pechyony, L. Shen, and R. Jones. 2011. Solving large scale linear SVM with distributed block minimization. In Proc. NIPS 2011 Workshop on Big Learning: Algorithms, Systems and Tools for Learning at Scale."},{"key":"e_1_3_2_38_2","unstructured":"D. Prokhorov. 2001. IJCNN 2001 neural network competition Ford Research Laboratory."},{"key":"e_1_3_2_39_2","unstructured":"F. P\u00e9rez-Cruz. 2000. M\u00e1quina de Vectores Soporte Adaptativa y Compacta . Ph.D. thesis Universidad Polit\u00e9cnica de Madrid (in Spanish)."},{"key":"e_1_3_2_40_2","first-page":"734","volume-title":"Advances in Neural Information Processing Systems (NIPS\u201900)","author":"P\u00e9rez-Cruz F.","year":"2000","unstructured":"F. P\u00e9rez-Cruz, P. L. Alarc\u00f3n-Diana, A. Navia-V\u00e1zquez, and A. Art\u00e9s-Rodr\u00edguez. 2000. Fast training of support vector machines. In Advances in Neural Information Processing Systems (NIPS\u201900), Vol. 13. 734\u2013740."},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2013.51"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0042519"},{"key":"e_1_3_2_43_2","volume-title":"Proc. of the 26th Annual Network and Distributed System Security Symposium (NDSS\u201919)","author":"Salem A.","year":"2019","unstructured":"A. Salem, Y. Zhang, M. Humbert, P. Berrang, M. Fritz, and M. Backes. 2019. ML-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Proc. of the 26th Annual Network and Distributed System Security Symposium (NDSS\u201919)."},{"key":"e_1_3_2_44_2","volume-title":"Proc. of the 26th Annual Network and Distributed System Security Symposium (NDSS\u201919)","author":"Salem A.","year":"2019","unstructured":"A. Salem, Y. Zhang, M. Humbert, P. Berrang, and M. Fritz, and M. Backes. 2019. ML-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Proc. of the 26th Annual Network and Distributed System Security Symposium (NDSS\u201919)."},{"key":"e_1_3_2_45_2","unstructured":"B. Scholkopf C. Burges and A. Smola. (Eds.) 1998. Advances in Kernel Methods - Support Vector Learning. The MIT Press."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2901840"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.29012\/jpc.684"},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/SP.2017.41","volume-title":"2017 IEEE Symposium on Security and Privacy (SP\u201917)","author":"Shokri R.","year":"2017","unstructured":"R. Shokri, M. Stronati, C. Song, and V. Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP\u201917). 3\u201318."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2018.8622478"},{"key":"e_1_3_2_50_2","unstructured":"M. Sinn R. D\u00edaz L. Mu\u00f1oz D. Dalle Carbonare A. Navia L. Nagel S. Rosello G. Fecondo L. Morabitto J. Correia and P. Papachristou. 2018. MUSKETEER: Machine learning to augment shared knowledge in federated privacy-preserving scenarios. European Union\u2019s Horizon 2020 Research and Innovation Programme under Grant Agreement No. 824988.http:\/\/www.musketeer.eu."},{"key":"e_1_3_2_51_2","article-title":"Federated multi-task learning","author":"Smith V.","year":"2017","unstructured":"V. Smith, C.-K. Chiang, M. Sanjabi, and A. Talwalkar. 2017. Federated multi-task learning. arXiv:1705.10467. https:\/\/arxiv.org\/abs\/1705.10467.","journal-title":"arXiv:1705.10467"},{"key":"e_1_3_2_52_2","volume-title":"Proc. of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Syed N. A.","year":"1999","unstructured":"N. A. Syed, S. Huan, L. Kah, and K. Sung. 1999. Incremental learning with support vector machines. In Proc. of the 5th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining."},{"key":"e_1_3_2_53_2","unstructured":"T. Timan and Mann Z. A. (Eds.). 2019. Data Protection in the Era of Artificial Intelligence. Trends Existing Solutions and Recommendations for Privacy-Preserving Technologies . Big Data Value Association (BDVA) position paper."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-007-0073-7"},{"key":"e_1_3_2_55_2","volume-title":"Proc. of the 3rd Conference on Email and Anti-Spam (CEAS\u201906)","author":"Webb S.","year":"2006","unstructured":"S. Webb, J. Caverlee, and C. Pu. 2006. Introducing the webb spam corpus: Using email spam to identify web spam automatically. In Proc. of the 3rd Conference on Email and Anti-Spam (CEAS\u201906)."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2016.7798964"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2019.2961673"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.011.2000045"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3339474"},{"key":"e_1_3_2_60_2","first-page":"2444","volume-title":"Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201916)","author":"Yang Z.","year":"2016","unstructured":"Z. Yang and W. U. Bajwa. 2016. A resilient distributed support vector machine. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201916). 2444\u20132448."},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1145\/1141277.1141415","volume-title":"Proc. 2006 ACM Symposium on Applied computing (SAC\u201906)","author":"Yu H.","year":"2006","unstructured":"H. Yu, X. Jiang, and J. Vaidya. 2006. Privacy-preserving SVM using nonlinear kernels on horizontally partitioned data. In Proc. 2006 ACM Symposium on Applied computing (SAC\u201906). 603\u2013610."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICNC.2009.120"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8191(03)00021-8"},{"key":"e_1_3_2_64_2","unstructured":"V. Zantedeschi A. Bellet and M. Tommasi. 2019. Communication-efficient and decentralized multi-task boosting while learning the collaboration graph. arXiv:1901.08460. https:\/\/aps.arxiv.org\/abs\/1901.08460v1."},{"key":"e_1_3_2_65_2","article-title":"Federated learning with non-iid data","author":"Zhao Y.","year":"2018","unstructured":"Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra. 2018. Federated learning with non-iid data. arXiv:1806.00582. https:\/\/arxiv.org\/abs\/1806.00582.","journal-title":"arXiv:1806.00582"},{"key":"e_1_3_2_66_2","volume-title":"Proc. of the Advances in Neural Information Processing Systems 32 (NeurIPS\u201919).","author":"Zhu L.","year":"2019","unstructured":"L. Zhu, Z. Liu, and S. Han. 2019. Deep leakage from gradients. In Proc. of the Advances in Neural Information Processing Systems 32 (NeurIPS\u201919)."}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539734","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3539734","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:02Z","timestamp":1750183802000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539734"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,22]]},"references-count":65,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12,31]]}},"alternative-id":["10.1145\/3539734"],"URL":"https:\/\/doi.org\/10.1145\/3539734","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,22]]},"assertion":[{"value":"2021-03-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}