{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T11:58:55Z","timestamp":1782907135825,"version":"3.54.5"},"reference-count":57,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T00:00:00Z","timestamp":1723161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Federated learning (FL) is a decentralized machine learning approach whereby each device is allowed to train local models, eliminating the requirement for centralized data collecting and ensuring data privacy. Unlike typical typical centralized machine learning, collaborative model training in FL involves aggregating updates from various devices without sending raw data. This ensures data privacy and security while collecting a collective learning from distributed data sources. These devices in FL models exhibit high efficacy in terms of privacy protection, scalability, and robustness, which is contingent upon the success of communication and collaboration. This paper explore the various topologies of both decentralized or centralized in the context of FL. In this respect, we investigated and explored in detail the evaluation of four widly used end-to-end FL frameworks: FedML, Flower, Flute, and PySyft. We specifically focused on vertical and horizontal FL systems using a logistic regression model that aggregated by the FedAvg algorithm. specifically, we conducted experiments on two images datasets, MNIST and Fashion-MNIST, to evaluate their efficiency and performance. Our paper provides initial findings on how to effectively combine horizontal and vertical solutions to address common difficulties, such as managing model synchronization and communication overhead. Our research indicates the trade-offs that exist in the performance of several simulation frameworks for federated learning.<\/jats:p>","DOI":"10.3390\/s24165149","type":"journal-article","created":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T07:59:07Z","timestamp":1723190347000},"page":"5149","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Evaluating Federated Learning Simulators: A Comparative Analysis of Horizontal and Vertical Approaches"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-4897-1445","authenticated-orcid":false,"given":"Ismail M.","family":"Elshair","sequence":"first","affiliation":[{"name":"Information Systems Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1617-4403","authenticated-orcid":false,"given":"Tariq Jamil Saifullah","family":"Khanzada","sequence":"additional","affiliation":[{"name":"Information Systems Department, Faculty of Computing and Information Technology, King Abdulaziz University, Jeddah 21589, Saudi Arabia"},{"name":"Computer Systems Engineering Department, Mehran University of Engineering and Technology, Jamshoro 76062, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8787-1868","authenticated-orcid":false,"given":"Muhammad Farrukh","family":"Shahid","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National University of Computer and Emerging Sciences, Karachi 75030, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shahbaz","family":"Siddiqui","sequence":"additional","affiliation":[{"name":"Department of Computer Science, National University of Computer and Emerging Sciences, Karachi 75030, Pakistan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,9]]},"reference":[{"key":"ref_1","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and y Arcas, B.A. (2017). Communication-efficient learning of deep networks from decentralized data. Artificial Intelligence and Statistics, PMLR."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, Q., He, B., and Song, D. (2020). Practical one-shot federated learning for cross-silo setting. arXiv.","DOI":"10.24963\/ijcai.2021\/205"},{"key":"ref_3","unstructured":"Ani Petrosyan (2023, April 20). Total Annual Number of Data Compromises in the United States Healthcare Sector from 2005 to 2022. Available online: https:\/\/www.statista.com\/statistics\/798417\/health-and-medical-data-compromises-united-states\/."},{"key":"ref_4","unstructured":"Rydning, D., Reinsel, J., and Gantz, J. (2018). The Digitization of The World from Edge to Core, International Data Corporation."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1109\/TSIPN.2022.3205549","article-title":"Decentralized and model-free federated learning: Consensus-based distillation in function space","volume":"8","author":"Taya","year":"2022","journal-title":"IEEE Trans. Signal Inf. Process. Over Networks"},{"key":"ref_6","first-page":"374","article-title":"Towards federated learning at scale: System design","volume":"1","author":"Bonawitz","year":"2019","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_7","unstructured":"Hardy, S., Henecka, W., Ivey-Law, H., Nock, R., Patrini, G., Smith, G., and Thorne, B. (2017). Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption. arXiv."},{"key":"ref_8","unstructured":"He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., and Qiu, H. (2020). Fedml: A research library and benchmark for federated machine learning. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, L., Wang, J., and Xu, C. (2020, January 28\u201329). FLSim: An Extensible and Reusable Simulation Framework for Federated Learning. Proceedings of the International Conference on Simulation Tools and Techniques, Guiyang, China.","DOI":"10.1007\/978-3-030-72792-5_30"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1646","DOI":"10.1109\/COMST.2020.2988293","article-title":"A survey of machine and deep learning methods for internet of things (IoT) security","volume":"22","author":"Mohamed","year":"2020","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"77518","DOI":"10.1109\/ACCESS.2023.3296311","article-title":"5G Aviation Networks Using Novel AI Approach for DDoS Detection","volume":"11","author":"Whitworth","year":"2023","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"209802","DOI":"10.1109\/ACCESS.2020.3036728","article-title":"A holistic review of cybersecurity and reliability perspectives in smart airports","volume":"8","author":"Koroniotis","year":"2020","journal-title":"IEEE Access"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wu, Z., Pan, S., Long, G., Jiang, J., and Zhang, C. (2019). Graph wavenet for deep spatial-temporal graph modeling. arXiv.","DOI":"10.24963\/ijcai.2019\/264"},{"key":"ref_14","unstructured":"Hipolito Garcia, M., Manoel, A., Diaz, D.M., Mireshghallah, F., Sim, R., and Dimitriadis, D. (2022). Flute: A scalable, extensible framework for high-performance federated learning simulations. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000083","article-title":"Advances and open problems in federated learning","volume":"14","author":"Kairouz","year":"2021","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Arouj, A., and Abdelmoniem, A.M. (2022, January 17). Towards energy-aware federated learning on battery-powered clients. Proceedings of the 1st ACM Workshop on Data Privacy and Federated Learning Technologies for Mobile Edge Network, Sydney, Australia.","DOI":"10.1145\/3556557.3557952"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wang, Z., Hu, Y., Xiao, J., and Wu, C. (2021). Efficient ring-topology decentralized federated learning with deep generative models for industrial artificial intelligent. arXiv.","DOI":"10.3390\/electronics11101548"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/OJCS.2020.2992630","article-title":"Federated learning for vehicular internet of things: Recent advances and open issues","volume":"1","author":"Du","year":"2020","journal-title":"IEEE Open J. Comput. Soc."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zantalis, F., Koulouras, G., Karabetsos, S., and Kandris, D. (2019). A review of machine learning and IoT in smart transportation. Future Internet, 11.","DOI":"10.3390\/fi11040094"},{"key":"ref_20","first-page":"13172","article-title":"Fedvision: An online visual object detection platform powered by federated learning","volume":"34","author":"Liu","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"8464","DOI":"10.1109\/TII.2021.3055283","article-title":"FASTGNN: A topological information protected federated learning approach for traffic speed forecasting","volume":"17","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yu, B., Yin, H., and Zhu, Z. (2017). Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv.","DOI":"10.24963\/ijcai.2018\/505"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"9980","DOI":"10.1109\/TVT.2022.3178808","article-title":"Efficient federated learning with spike neural networks for traffic sign recognition","volume":"71","author":"Xie","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Mun, H., and Lee, Y. (2020). Internet traffic classification with federated learning. Electronics, 10.","DOI":"10.3390\/electronics10010027"},{"key":"ref_25","unstructured":"Mahembe, B.K., and Nyirenda, C. (2022). A Modified UDP for Federated Learning Packet Transmissions. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, L., Zhang, J., Song, S.H., and Letaief, K.B. (2020, January 7\u201311). Client-edge-cloud hierarchical federated learning. Proceedings of the ICC 2020\u20132020 IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9148862"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/TNSE.2021.3053588","article-title":"Optimal user-edge assignment in hierarchical federated learning based on statistical properties and network topology constraints","volume":"9","author":"Mhaisen","year":"2021","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3805","DOI":"10.1109\/TMC.2022.3147792","article-title":"Accelerating federated learning with cluster construction and hierarchical aggregation","volume":"22","author":"Wang","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1038\/s41746-020-00323-1","article-title":"The future of digital health with federated learning","volume":"3","author":"Rieke","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_30","unstructured":"Vanhaesebrouck, P., Bellet, A., and Tommasi, M. (2017, January 20\u201322). Decentralized collaborative learning of personalized models over networks. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, FL, USA."},{"key":"ref_31","first-page":"50","article-title":"Federated learning: Challenges, methods, and future directions","volume":"37","author":"Li","year":"2020","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_32","unstructured":"Roy, A.G., Siddiqui, S., P\u00f6lsterl, S., Navab, N., and Wachinger, C. (2019). Braintorrent: A peer-to-peer environment for decentralized federated learning. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"120526","DOI":"10.1016\/j.apenergy.2022.120526","article-title":"Federated reinforcement learning for smart building joint peer-to-peer energy and carbon allowance trading","volume":"333","author":"Qiu","year":"2023","journal-title":"Appl. Energy"},{"key":"ref_34","unstructured":"Lee, J.-W., Oh, J., Lim, S., Yun, S.-Y., and Lee, J.-G. (2020). Tornadoaggregate: Accurate and scalable federated learning via the ring-based architecture. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Yu, G., Wang, X., Sun, C., Wang, Q., Yu, P., Ni, W., and Liu, R.P. (2023). IronForge: An open, secure, fair, decentralized federated learning. IEEE Trans. Neural Networks Learn. Syst., 1\u201315.","DOI":"10.1109\/TNNLS.2023.3329249"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ekaireb, E., Yu, X., Ergun, K., Zhao, Q., Lee, K., Huzaifa, M., and Rosing, T. (2022, January 22\u201323). ns3-fl: Simulating Federated Learning with ns-3. Proceedings of the 2022 Workshop on ns-3, Virtual.","DOI":"10.1145\/3532577.3532591"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, S., Ngai, E.C.-H., Ye, F., Ju, L., Zhang, T., and Voigt, T. (2024, January 13\u201314). Blades: A unified benchmark suite for byzantine attacks and defenses in federated learning. Proceedings of the 2024 IEEE\/ACM Ninth International Conference on Internet-of-Things Design and Implementation (IoTDI), Hong Kong, China.","DOI":"10.1109\/IoTDI61053.2024.00018"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Karimireddy, S.P., Veeraragavan, N.R., Elvatun, S., and Nygard, J.F. (2023, January 18\u201320). Federated Learning Showdown: The Comparative Analysis of Federated Learning Frameworks. Proceedings of the Eighth International Conference on Fog and Mobile Edge Computing (FMEC), Tartu, Estonia.","DOI":"10.1109\/FMEC59375.2023.10305961"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Solanki, T., Rai, B.K., and Sharma, S. (2022). Federated Learning using tensor flow. Federated Learning for IoT Applications, Springer International Publishing.","DOI":"10.1007\/978-3-030-85559-8_10"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ziller, A., Trask, A., Lopardo, A., Szymkow, B., Wagner, B., Bluemke, E., Nounahon, J.M., Passerat-Palmbach, J., Prakash, K., and Rose, N. (2021). Pysyft: A library for easy federated learning. Federated Learning Systems: Towards Next-Generation AI, Springer.","DOI":"10.1007\/978-3-030-70604-3_5"},{"key":"ref_41","unstructured":"Beutel, D.J., Topal, T., Mathur, A., Qiu, X., Fernandez-Marques, J., Gao, Y., Sani, L., Li, K.H., Parcollet, T., and de Gusmao, P.P.B. (2020). Flower: A friendly federated learning research framework. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, J., and Li, J. (2024, August 01). Federated Dual Averaging Learning Algorithm with Delayed Gradients for Composite Optimization. Available online: https:\/\/ssrn.com\/abstract=4507875.","DOI":"10.2139\/ssrn.4507875"},{"key":"ref_43","unstructured":"Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. (2020). Federated learning with matched averaging. arXiv."},{"key":"ref_44","first-page":"604","article-title":"Neural network gain scheduling design for large envelope curve flight control law","volume":"31","author":"Zhang","year":"2005","journal-title":"J. Beijing Univ. Aeronaut. Astronaut."},{"key":"ref_45","first-page":"429","article-title":"Federated optimization in heterogeneous networks","volume":"2","author":"Li","year":"2020","journal-title":"Proc. Mach. Learn. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lyu, L., Yu, H., and Yang, Q. (2020). Threats to federated learning: A survey. arXiv.","DOI":"10.1007\/978-3-030-63076-8_1"},{"key":"ref_47","unstructured":"Gen\u00e7t\u00fcrk, M. (2024, August 01). Privacy-Preserving Horizontal Federated Learning Methodology through a Novel Boosting-Based Federated Random Forest Algorithm. Available online: https:\/\/hdl.handle.net\/11511\/101867."},{"key":"ref_48","unstructured":"Han, Y., Du, P., and Yang, K. (2022). FedGBF: An efficient vertical federated learning framework via gradient boosting and bagging. arXiv."},{"key":"ref_49","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., Friedman, J.H., and Friedman, J.H. (2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer.","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Banerjee, K., Gupta, R.R., Vyas, K., and Mishra, B. (2020). Exploring alternatives to softmax function. arXiv.","DOI":"10.5220\/0010502000002996"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3339474","article-title":"Federated machine learning: Concept and applications","volume":"10","author":"Yang","year":"2019","journal-title":"Acm Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST database of handwritten digit images for machine learning research","volume":"29","author":"Liu","year":"2012","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_54","unstructured":"Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3523061","article-title":"An efficient learning framework for federated XGBoost using secret sharing and distributed optimization","volume":"13","author":"Xie","year":"2022","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"100742","DOI":"10.1016\/j.iot.2023.100742","article-title":"Communication and computation efficiency in federated learning: A survey","volume":"22","author":"Almanifi","year":"2023","journal-title":"Internet Things"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"012015","DOI":"10.1088\/1742-6596\/2711\/1\/012015","article-title":"Comparative analysis of various models for image classification on Cifar-100 dataset","volume":"2711","author":"Zheng","year":"2024","journal-title":"J. Phys. Conf. Ser."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/16\/5149\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:32:43Z","timestamp":1760110363000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/16\/5149"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,9]]},"references-count":57,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["s24165149"],"URL":"https:\/\/doi.org\/10.3390\/s24165149","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,9]]}}}