{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T17:43:36Z","timestamp":1781631816564,"version":"3.54.5"},"reference-count":35,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T00:00:00Z","timestamp":1760054400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute of Information & Communications Technology Planning and Evaluation","award":["RS-2022-00167197"],"award-info":[{"award-number":["RS-2022-00167197"]}]},{"name":"BK21 FOUR","award":["5199990914048"],"award-info":[{"award-number":["5199990914048"]}]},{"name":"Soonchunhyang University Research Fund"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Federated Learning (FL) has been proposed as a new machine learning paradigm to ensure data privacy by training the model in a decentralized manner. However, FL is challenged by device heterogeneity, asymmetric data contribution, and imbalanced datasets, which complicate system control and hinder performance due to long waiting times for aggregation. To tackle the FL challenges, we propose Adaptive Time Threshold Client Selection using DRL (ATCS-FL) to adjust the time threshold (\u03b1) in each communication round based on computing and resource capacity of each device and the volume of data updates. The Double Deep Q-Network (DDQN) model determines the appropriate \u03b1, according to the variations in local training time that achieves performance improvement alongside latency reduction. Based on the \u03b1, the server selects a subset of clients with adequate resources that can finish training within the \u03b1 for participating in the training process. Our approach dynamically adjusts the \u03b1 and adaptively selects the number of clients, effectively mitigates the impact of heterogeneous training speeds and significantly enhances communication efficiency. Our experiment utilizes CIFAR-10 and MNIST benchmarked datasets for image classification training with convolutional neural networks across non-IID distributed levels in FL. Specifically, ATCS-FL demonstrates performance improvement and latency reduction of 77% and 75%, respectively, compared to FedProx and FLASH-RL.<\/jats:p>","DOI":"10.3390\/sym17101700","type":"journal-article","created":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T09:45:01Z","timestamp":1760089501000},"page":"1700","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["DRL-Based Adaptive Time Threshold Client Selection FL"],"prefix":"10.3390","volume":"17","author":[{"given":"Sreyleak","family":"Sam","sequence":"first","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taikuong","family":"Iv","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-3542-0548","authenticated-orcid":false,"given":"Rothny","family":"Mom","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4832-3995","authenticated-orcid":false,"given":"Seungwoo","family":"Kang","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-9325-8518","authenticated-orcid":false,"given":"Inseok","family":"Song","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seyha","family":"Ros","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sovanndoeur","family":"Riel","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seokhoon","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Soonchunhyang University, Asan 31538, Republic of Korea"},{"name":"Department of Computer Software Engineering, Soonchunhyang University, Asan 31538, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,10]]},"reference":[{"key":"ref_1","unstructured":"McMahan, B., Moore, E., Ramage, D., Hampson, S., and Arcas, B.A. (2017, January 20\u201322). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_2","first-page":"50","article-title":"Federated learning: Challenges, methods, and future directions","volume":"37","author":"Li","year":"2020","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1109\/MNET.101.2100328","article-title":"Federated analytics: Opportunities and challenges","volume":"36","author":"Wang","year":"2021","journal-title":"IEEE Netw."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Berkani, M.R., Chouchane, A., Himeur, Y., Ouamane, A., Miniaoui, S., Atalla, S., Mansoor, W., and Al-Ahmad, H. (2025). Advances in federated learning: Applications and challenges in smart building environments and beyond. Computers, 14.","DOI":"10.3390\/computers14040124"},{"key":"ref_5","unstructured":"Tang, Z., Zhang, Y., Shi, S., He, X., Han, B., and Chu, X. (2022, January 17\u201323). Virtual homogeneity learning: Defending against data heterogeneity in federated learning. Proceedings of the 39th International Conference on Machine Learning, Baltimore, ML, USA."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1996","DOI":"10.1109\/TPDS.2021.3134647","article-title":"AUCTION: Automated and quality-aware client selection framework for efficient federated learning","volume":"33","author":"Deng","year":"2021","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2031","DOI":"10.1109\/COMST.2020.2986024","article-title":"Federated learning in mobile edge networks: A comprehensive survey","volume":"22","author":"Lim","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","first-page":"1","article-title":"Heterogeneous federated learning: State-of-the-art and research challenges","volume":"56","author":"Ye","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"110663","DOI":"10.1016\/j.comnet.2024.110663","article-title":"A comprehensive survey on client selection strategies in federated learning","volume":"251","author":"Li","year":"2024","journal-title":"Comput. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"21811","DOI":"10.1109\/JIOT.2023.3299573","article-title":"Client selection in federated learning: Principles, challenges, and opportunities","volume":"10","author":"Fu","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1007\/s11831-023-10011-4","article-title":"A review of client selection methods in federated learning","volume":"31","author":"Mayhoub","year":"2024","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bouaziz, S., Benmeziane, H., Imine, Y., Hamdad, L., Niar, S., and Ouarnoughi, H. (2023, January 6\u20138). Flash-rl: Federated learning addressing system and static heterogeneity using reinforcement learning. Proceedings of the 2023 IEEE 41st International Conference on Computer Design (ICCD), Washington, DC, USA.","DOI":"10.1109\/ICCD58817.2023.00074"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nishio, T., and Yonetani, R. (2019, January 20). Client selection for federated learning with heterogeneous resources in mobile edge. Proceedings of the ICC 2019\u20142019 IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761315"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, X., Chen, T., Qian, F., Guo, Z., Lin, F.X., Wang, X., and Chen, K. (2017, January 16). Characterizing smartwatch usage in the wild. Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, Niagara Falls, NY, USA.","DOI":"10.1145\/3081333.3081351"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/IOTM.004.2100182","article-title":"Federated learning for the internet of things: Applications, challenges, and opportunities","volume":"5","author":"Zhang","year":"2022","journal-title":"IEEE Internet Things Mag."},{"key":"ref_16","first-page":"5296","article-title":"Federated learning framework with straggling mitigation and privacy-awareness for AI-based mobile application services","volume":"22","author":"Saputra","year":"2022","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_17","unstructured":"Hard, A., Girgis, A.M., Amid, E., Augenstein, S., McConnaughey, L., Mathews, R., and Anil, R. (2024). Learning from straggler clients in federated learning. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3333","DOI":"10.1109\/TCOMM.2024.3486979","article-title":"Stragglers-aware low-latency synchronous federated learning via layer-wise model updates","volume":"73","author":"Lang","year":"2024","journal-title":"IEEE Trans. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7108","DOI":"10.1109\/TWC.2020.3008091","article-title":"Multi-armed bandit-based client scheduling for federated learning","volume":"19","author":"Xia","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3367","DOI":"10.1109\/TVT.2021.3136308","article-title":"Adaptive deadline determination for mobile device selection in federated learning","volume":"71","author":"Lee","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, W., Cui, T., Shen, B., Huang, X., and Chen, Q. (2023, January 2\u20134). Adaptive waiting time asynchronous federated learning in edge computing. Proceedings of the 2023 International Conference on Wireless Communications and Signal Processing (WCSP), Hangzhou, China.","DOI":"10.1109\/WCSP58612.2023.10404949"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10400","DOI":"10.1109\/ACCESS.2021.3050172","article-title":"Dynamic federated learning for GMEC with time-varying wireless link","volume":"9","author":"Zhai","year":"2021","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Chen, B., Ivanov, N., Wang, G., and Yan, Q. (2023, January 11\u201314). DynamicFL: Balancing communication dynamics and client manipulation for federated learning. Proceedings of the 2023 20th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON), Madrid, Spain.","DOI":"10.1109\/SECON58729.2023.10287430"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"8475","DOI":"10.1109\/TII.2021.3064351","article-title":"Deep reinforcement learning assisted federated learning algorithm for data management of IIoT","volume":"17","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Li, A., Li, R., Yang, L., and Xu, X. (2025). FedPPO: Reinforcement learning-based client selection for federated learning with heterogeneous data. IEEE Trans. Cogn. Commun. Netw., 1.","DOI":"10.1109\/TCCN.2025.3547751"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"674","DOI":"10.1109\/TMC.2021.3096846","article-title":"Adaptive asynchronous federated learning in resource-constrained edge computing","volume":"22","author":"Liu","year":"2021","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shi, Y., Liu, Z., Shi, Z., and Yu, H. (2023, January 10\u201314). Fairness-aware client selection for federated learning. Proceedings of the 2023 IEEE International Conference on Multimedia and Expo (ICME), Brisbane, Australia.","DOI":"10.1109\/ICME55011.2023.00063"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, M., Zhao, H., Ebron, S., Xie, R., and Yang, K. (2024, January 23\u201326). Ensuring fairness in federated learning services: Innovative Approaches to Client Selection, Scheduling, and Rewards. Proceedings of the 2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS), Jersey City, NJ, USA.","DOI":"10.1109\/ICDCS60910.2024.00076"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"2002","journal-title":"Proc. IEEE"},{"key":"ref_30","unstructured":"Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., and Chandra, V. (2018). Federated learning with non-iid data. arXiv."},{"key":"ref_31","unstructured":"O\u2019shea, K., and Nash, R. (2015). An introduction to convolutional neural networks. arXiv."},{"key":"ref_32","unstructured":"Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K., Hoang, N., and Khazaeni, Y. (2019, January 9\u201315). Bayesian nonparametric federated learning of neural networks. Proceedings of the International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_33","unstructured":"Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. (2020). Federated learning with matched averaging. arXiv."},{"key":"ref_34","unstructured":"Steiner, B., DeVito, Z., Chintala, S., Gross, S., Paske, A., Massa, F., Lerer, A., Chanan, G., Lin, Z., and Yang, E. (2019, January 8\u201314). Pytorch: An imperative style, high-performance deep learning library. Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_35","first-page":"1","article-title":"Fedlab: A flexible federated learning framework","volume":"24","author":"Zeng","year":"2023","journal-title":"J. Mach. Learn. Res."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/10\/1700\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T10:01:37Z","timestamp":1760090497000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/10\/1700"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,10]]},"references-count":35,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["sym17101700"],"URL":"https:\/\/doi.org\/10.3390\/sym17101700","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,10]]}}}