{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T18:25:51Z","timestamp":1783189551032,"version":"3.54.6"},"reference-count":45,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T00:00:00Z","timestamp":1677110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institute of Information and Communications Technology"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Federated learning (FL) is a technique that allows multiple clients to collaboratively train a global model without sharing their sensitive and bandwidth-hungry data. This paper presents a joint early client termination and local epoch adjustment for FL. We consider the challenges of heterogeneous Internet of Things (IoT) environments including non-independent and identically distributed (non-IID) data as well as diverse computing and communication capabilities. The goal is to strike the best tradeoff among three conflicting objectives, namely global model accuracy, training latency and communication cost. We first leverage the balanced-MixUp technique to mitigate the influence of non-IID data on the FL convergence rate. A weighted sum optimization problem is then formulated and solved via our proposed FL double deep reinforcement learning (FedDdrl) framework, which outputs a dual action. The former indicates whether a participating FL client is dropped, whereas the latter specifies how long each remaining client needs to complete its local training task. Simulation results show that FedDdrl outperforms the existing FL scheme in terms of overall tradeoff. Specifically, FedDdrl achieves higher model accuracy by about 4% while incurring 30% less latency and communication costs.<\/jats:p>","DOI":"10.3390\/s23052494","type":"journal-article","created":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T02:03:26Z","timestamp":1677204206000},"page":"2494","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["FedDdrl: Federated Double Deep Reinforcement Learning for Heterogeneous IoT with Adaptive Early Client Termination and Local Epoch Adjustment"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4598-2653","authenticated-orcid":false,"given":"Yi Jie","family":"Wong","sequence":"first","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang 43000, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4600-9839","authenticated-orcid":false,"given":"Mau-Luen","family":"Tham","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang 43000, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7094-8612","authenticated-orcid":false,"given":"Ban-Hoe","family":"Kwan","sequence":"additional","affiliation":[{"name":"Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Kajang 43000, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yasunori","family":"Owada","sequence":"additional","affiliation":[{"name":"Resilient ICT Research Center, Network Research Institute, National Institute of Information and Communications Technology (NICT), Tokyo 184-8795, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Al-Maslamani, N., Abdallah, M., and Ciftler, B.S. (June, January 30). Secure Federated Learning for IoT Using DRL-Based Trust Mechanism. Proceedings of the 2022 International Wireless Communications and Mobile Computing, IWCMC 2022, Dubrovnik, Croatia.","DOI":"10.1109\/IWCMC55113.2022.9824672"},{"key":"ref_2","first-page":"16","article-title":"The Digitization of the World from Edge to Core","volume":"16","author":"Reinsel","year":"2018","journal-title":"Fram. Int. Data Corp."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"12598","DOI":"10.1038\/s41598-020-69250-1","article-title":"Federated Learning in Medicine: Facilitating Multi-Institutional Collaborations without Sharing Patient Data","volume":"10","author":"Sheller","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_4","unstructured":"McMahan, B.H., Moore, E., Ramage, D., Hampson, S., and Ag\u00fcera y Arcas, B. (2017, January 20\u201322). Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, Fort Lauderdale, FL, USA."},{"key":"ref_5","unstructured":"Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., and Ramage, D. (2018). Federated Learning for Mobile Keyboard Prediction. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"208518","DOI":"10.1109\/ACCESS.2020.3038676","article-title":"Active Learning Based Federated Learning for Waste and Natural Disaster Image Classification","volume":"8","author":"Ahmed","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"115930","DOI":"10.1109\/ACCESS.2022.3218655","article-title":"An Optimized Multi-Task Learning Model for Disaster Classification and Victim Detection in Federated Learning Environments","volume":"10","author":"Wong","year":"2022","journal-title":"IEEE Access"},{"key":"ref_8","unstructured":"Reina, G.A., Gruzdev, A., Foley, P., Perepelkina, O., Sharma, M., Davidyuk, I., Trushkin, I., Radionov, M., Mokrov, A., and Agapov, D. (2021). OpenFL: An Open-Source Framework for Federated Learning. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chen, X., Li, Z., Ni, W., Wang, X., Zhang, S., Xu, S., and Pei, Q. (2022, January 11\u201313). Two-Phase Deep Reinforcement Learning of Dynamic Resource Allocation and Client Selection for Hierarchical Federated Learning. Proceedings of the 2022 IEEE\/CIC International Conference on Communications in China, ICCC 2022, Foshan, China.","DOI":"10.1109\/ICCC55456.2022.9880724"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1884","DOI":"10.1109\/TII.2022.3183465","article-title":"Optimizing Federated Learning with Deep Reinforcement Learning for Digital Twin Empowered Industrial IoT","volume":"19","author":"Yang","year":"2022","journal-title":"IEEE Trans. Industr. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3688","DOI":"10.1109\/JSAC.2021.3118352","article-title":"Optimizing Federated Learning in Distributed Industrial IoT: A Multi-Agent Approach","volume":"39","author":"Zhang","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liu, L., Zhang, J., Song, S.H., and Letaief, K.B. (2020, January 7\u201311). Client-Edge-Cloud Hierarchical Federated Learning. Proceedings of the IEEE International Conference on Communications 2020, Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9148862"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Song, Q., Lei, S., Sun, W., and Zhang, Y. (April, January 29). Adaptive Federated Learning for Digital Twin Driven Industrial Internet of Things; Adaptive Federated Learning for Digital Twin Driven Industrial Internet of Things. Proceedings of the 2021 IEEE Wireless Communications and Networking Conference (WCNC), Nanjing, China.","DOI":"10.1109\/WCNC49053.2021.9417370"},{"key":"ref_14","first-page":"9091","article-title":"A Multi-Agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning","volume":"36","author":"Zhang","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5476","DOI":"10.1109\/JIOT.2020.3030072","article-title":"A Survey on Federated Learning: The Journey from Centralized to Distributed on-Site Learning and Beyond","volume":"8","author":"Abdulrahman","year":"2021","journal-title":"IEEE Internet Things J."},{"key":"ref_16","unstructured":"Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V. (2018). Federated Learning with Non-IID Data. arXiv."},{"key":"ref_17","unstructured":"Wang, J., Liu, Q., Liang, H., Joshi, G., and Vincent Poor, H. (2020, January 6\u201312). Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Vancouver, BC, Canada."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.jpdc.2022.07.009","article-title":"AMBLE: Adjusting Mini-Batch and Local Epoch for Federated Learning with Heterogeneous Devices","volume":"170","author":"Park","year":"2022","journal-title":"J. Parallel. Distrib. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"98423","DOI":"10.1109\/ACCESS.2021.3095915","article-title":"Adaptive Client Selection in Resource Constrained Federated Learning Systems: A Deep Reinforcement Learning Approach","volume":"9","author":"Zhang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8475","DOI":"10.1109\/TII.2021.3064351","article-title":"Deep Reinforcement Learning Assisted Federated Learning Algorithm for Data Management of IIoT","volume":"17","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Industr. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wang, H., Kaplan, Z., Niu, D., and Li, B. (2020, January 6\u20139). Optimizing Federated Learning on Non-IID Data with Reinforcement Learning. Proceedings of the IEEE INFOCOM 2020-IEEE Conference on Computer Communications, Toronto, ON, Canada.","DOI":"10.1109\/INFOCOM41043.2020.9155494"},{"key":"ref_22","first-page":"323","article-title":"Balanced-MixUp for Highly Imbalanced Medical Image Classification","volume":"Volume 12905","author":"Galdran","year":"2021","journal-title":"Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021, Proceedings of the 24th International Conference, Strasbourg, France, 27 September\u20131 October 2021"},{"key":"ref_23","unstructured":"Li, T., Sahu, A.K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. (2018). Federated Optimization in Heterogeneous Networks. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Nishio, T., and Yonetani, R. (2019, January 20\u201324). Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge. Proceedings of the ICC 2019\u20142019 IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761315"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zheng, J., Li, K., Tovar, E., and Guizani, M. (July, January 28). Federated Learning for Energy-Balanced Client Selection in Mobile Edge Computing. Proceedings of the 2021 International Wireless Communications and Mobile Computing, IWCMC 2021, Harbin, China.","DOI":"10.1109\/IWCMC51323.2021.9498853"},{"key":"ref_26","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing Atari with Deep Reinforcement Learning. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1038\/s41586-019-1724-z","article-title":"Grandmaster Level in StarCraft II Using Multi-Agent Reinforcement Learning","volume":"575","author":"Vinyals","year":"2019","journal-title":"Nature"},{"key":"ref_28","unstructured":"Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., and Graepel, T. (2017). Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Han, M., Sun, X., Zheng, S., Wang, X., and Tan, H. (2021, January 26\u201328). Resource Rationing for Federated Learning with Reinforcement Learning. Proceedings of the 2021 Computing, Communications and IoT Applications (ComComAp), Shenzhen, China.","DOI":"10.1109\/ComComAp53641.2021.9653111"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Xiong, Z., Cheng, Z., Xu, C., Lin, X., Liu, X., Wang, D., Luo, X., Zhang, Y., Qiao, N., and Zheng, M. (2020). Facing Small and Biased Data Dilemma in Drug Discovery with Federated Learning. bioRxiv.","DOI":"10.1101\/2020.03.19.998898"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jallepalli, D., Ravikumar, N.C., Badarinath, P.V., Uchil, S., and Suresh, M.A. (2021, January 23\u201326). Federated Learning for Object Detection in Autonomous Vehicles. Proceedings of the IEEE 7th International Conference on Big Data Computing Service and Applications, BigDataService, Oxford, UK.","DOI":"10.1109\/BigDataService52369.2021.00018"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, Q., Diao, Y., Chen, Q., and He, B. (2021, January 9\u201312). Federated Learning on Non-IID Data Silos: An Experimental Study. Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE), Virtual.","DOI":"10.1109\/ICDE53745.2022.00077"},{"key":"ref_33","unstructured":"Sunehag, P., Lever, G., Gruslys, A., Marian Czarnecki, W., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J.Z., and Tuyls, K. (2017). Value-Decomposition Networks for Cooperative Multi-Agent Learning. arXiv."},{"key":"ref_34","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (May, January 30). Mixup: Beyond Empirical Risk Minimization. Proceedings of the 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada. Conference Track Proceedings 2017."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Thulasidasan, S., Chennupati, G., Bilmes, J.A., Bhattacharya, T., and Michalak, S. (2019). On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks. Adv. Neural. Inf. Process. Syst., 32.","DOI":"10.2172\/1525811"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhou, Z., Qi, L., and Shi, Y. (2022, January 23\u201327). Generalizable Medical Image Segmentation via Random Amplitude Mixup and Domain-Specific Image Restoration. Proceedings of the 17th European Conference, Computer Vision\u2013ECCV 2022, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19803-8_25"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sun, L., Xia, C., Yin, W., Liang, T., Yu, P.S., and He, L. (2020, January 8\u201313). Mixup-Transformer: Dynamic Data Augmentation for NLP Tasks. Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain.","DOI":"10.18653\/v1\/2020.coling-main.305"},{"key":"ref_38","unstructured":"Guo, H., Mao, Y., and Zhang, R. (2019). Augmenting Data with Mixup for Sentence Classification: An Empirical Study. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chou, H.P., Chang, S.C., Pan, J.Y., Wei, W., and Juan, D.C. (2020, January 23\u201328). Remix: Rebalanced Mixup. Proceedings of the Computer Vision\u2013ECCV 2020 Workshops, Glasgow, UK. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics).","DOI":"10.1007\/978-3-030-65414-6_9"},{"key":"ref_40","unstructured":"Alam, F., Alam, T., Ofli, F., and Imran, M. (2021). Social Media Images Classification Models for Real-Time Disaster Response. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.-C. (2018, January 18\u201322). MobileNetV2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_42","unstructured":"(2022, December 25). Tensorflow Federated Using TFF for Federated Learning Research|TensorFlow Federated. Available online: https:\/\/www.tensorflow.org\/federated\/tff_for_research."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1746","DOI":"10.4208\/cicp.OA-2020-0085","article-title":"Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks","volume":"28","author":"Xu","year":"2019","journal-title":"Commun. Comput. Phys."},{"key":"ref_44","unstructured":"Li, X., Huang, K., Yang, W., Wang, S., and Zhang, Z. (2019). On the Convergence of FedAvg on Non-IID Data. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1109\/TWC.2020.3024860","article-title":"Deep Reinforcement Learning Based Intelligent Reflecting Surface for Secure Wireless Communications","volume":"20","author":"Yang","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2494\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:40:41Z","timestamp":1760121641000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2494"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,23]]},"references-count":45,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052494"],"URL":"https:\/\/doi.org\/10.3390\/s23052494","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,23]]}}}