{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T13:23:52Z","timestamp":1780406632615,"version":"3.54.1"},"reference-count":30,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T00:00:00Z","timestamp":1674604800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the advent of the Internet of Things (IoT) era, a wide array of wireless sensors supporting the IoT have proliferated. As key elements for enabling the IoT, wireless sensor nodes require minimal energy consumption and low device complexity. In particular, energy-efficient resource scheduling is critical in maintaining a network of wireless sensor nodes, since the energy-intensive processing of wireless sensor nodes and their interactions is too complicated to control. In this study, we present a practical deep Q-network (DQN)-based packet scheduling algorithm that coordinates the transmissions of multiple IoT devices. The scheduling algorithm dynamically adjusts the connection interval (CI) and the number of packets transmitted by each node within the interval. Through various experiments, we verify how effectively the proposed scheduler improves energy efficiency and handles the time-varying nature of the network environment. Moreover, we attempt to gain insight into the optimized packet scheduler by analyzing the policy of the DQN scheduler. The experimental results show that the proposed scheduling algorithm can further prolong a network\u2019s lifetime in a dynamic network environment in comparison with that in other alternative schemes while ensuring the quality of service (QoS).<\/jats:p>","DOI":"10.3390\/s23031339","type":"journal-article","created":{"date-parts":[[2023,1,25]],"date-time":"2023-01-25T04:54:41Z","timestamp":1674622481000},"page":"1339","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Deep-Q-Network-Based Packet Scheduling in an IoT Environment"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8374-072X","authenticated-orcid":false,"given":"Xing","family":"Fu","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, College of Electronics and Information, Kyung Hee University, Yongin 17104, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3142-2014","authenticated-orcid":false,"given":"Jeong Geun","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, College of Electronics and Information, Kyung Hee University, Yongin 17104, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2787","DOI":"10.1016\/j.comnet.2010.05.010","article-title":"The Internet of Things: A survey","volume":"54","author":"Atzori","year":"2010","journal-title":"Comput. Netw."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sethi, P., and Sarangi, S.R. (2017). Internet of Things: Architectures, protocols, and applications. J. Electr. Comput. Eng., 2017.","DOI":"10.1155\/2017\/9324035"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1007\/s11277-019-06407-w","article-title":"IoT technology, applications and challenges: A contemporary survey","volume":"108","author":"Balaji","year":"2019","journal-title":"Wirel. Pers. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2292","DOI":"10.1016\/j.comnet.2008.04.002","article-title":"Wireless sensor network survey","volume":"52","author":"Yick","year":"2008","journal-title":"Comput. Netw."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kandris, D., Nakas, C., Vomvas, D., and Koulouras, G. (2020). Applications of wireless sensor networks: An up-to-date survey. Appl. Syst. Innov., 3.","DOI":"10.3390\/asi3010014"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1109\/SURV.2011.060710.00094","article-title":"Energy harvesting sensor nodes: Survey and implications","volume":"13","author":"Sudevalayam","year":"2010","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","first-page":"100504","article-title":"Energy efficient scheme for better connectivity in sustainable mobile wireless sensor networks","volume":"30","author":"Sachan","year":"2021","journal-title":"Sustain. Comput. Inform. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1023\/A:1016627727877","article-title":"A survey of energy efficient network protocols for wireless networks","volume":"7","author":"Jones","year":"2001","journal-title":"Wirel. Netw."},{"key":"ref_9","first-page":"1505","article-title":"Packet scheduling using SVM models in wireless communication networks","volume":"20","author":"Bhandari","year":"2019","journal-title":"J. Internet Technol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Zhai, D., Zhang, R., and Wang, Y. (2019, January 26\u201329). Deep neural network based channel allocation for interference-limited wireless networks. Proceedings of the 2019 IEEE 20th International Conference on High Performance Switching and Routing (HPSR), Xi\u2019an, China.","DOI":"10.1109\/HPSR.2019.8808132"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Xu, S., Liu, P., Wang, R., and Panwar, S.S. (2019, January 15\u201318). Realtime scheduling and power allocation using deep neural networks. Proceedings of the 2019 IEEE Wireless Communications and Networking Conference (WCNC), Marrakesh, Morocco.","DOI":"10.1109\/WCNC.2019.8886140"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, J., Zhang, G., Nie, J., Peng, Y., and Zhang, Y. (2021). Deep reinforcement learning for scheduling in an edge computing-based industrial internet of things. Wirel. Commun. Mob. Comput., 2021.","DOI":"10.1155\/2021\/8017334"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1109\/JSEN.2017.2759327","article-title":"Energy-efficient scheduling for multiple latency-sensitive Bluetooth Low Energy nodes","volume":"18","author":"Chen","year":"2017","journal-title":"IEEE Sens. J."},{"key":"ref_14","unstructured":"Bluetooth, S. (2014). Specification of the Bluetooth System v4. 2, Bluetooth SIG."},{"key":"ref_15","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_18","unstructured":"Matloff, N. (2008). Introduction to Discrete-Event Simulation and the Simpy Language, Department of Computer Science, University of California at Davis."},{"key":"ref_19","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). OpenAI Gym. arXiv e-prints. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Jones, A. (2020). Survey: Energy efficient protocols using radio scheduling in wireless sensor network. Int. J. Electr. Comput. Eng., 10.","DOI":"10.11591\/ijece.v10i2.pp1296-1307"},{"key":"ref_21","first-page":"219","article-title":"A survey on design challenges of scheduling algorithms for wireless networks","volume":"28","author":"Malekpourshahraki","year":"2022","journal-title":"Int. J. Commun. Netw. Distrib. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2042","DOI":"10.1109\/TII.2017.2763971","article-title":"A data-emergency-aware scheduling scheme for Internet of Things in smart cities","volume":"14","author":"Qiu","year":"2017","journal-title":"IEEE Trans. Ind. Inf."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Feng, J., and Zhao, H. (2018). Energy-balanced multisensory scheduling for target tracking in wireless sensor networks. Sensors, 18.","DOI":"10.3390\/s18103585"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Dementyev, A., Hodges, S., Taylor, S., and Smith, J. (2013, January 14\u201318). Power consumption analysis of Bluetooth Low Energy, ZigBee and ANT sensor nodes in a cyclic sleep scenario. Proceedings of the 2013 IEEE International Wireless Symposium (IWS), Beijing, China.","DOI":"10.1109\/IEEE-IWS.2013.6616827"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"17817","DOI":"10.1109\/ACCESS.2018.2817343","article-title":"Advertisement interval to minimize discovery time of whole BLE advertisers","volume":"6","author":"Shan","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"46066","DOI":"10.1109\/ACCESS.2018.2866323","article-title":"Detailed examination of a packet collision model for Bluetooth Low Energy advertising mode","volume":"6","author":"Ghamari","year":"2018","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Giovanelli, D., Milosevic, B., and Farella, E. (2015, January 18\u201319). Bluetooth Low Energy for data streaming: Application-level analysis and recommendation. Proceedings of the 2015 6th International Workshop on Advances in Sensors and Interfaces (IWASI), Gallipoli, Italy.","DOI":"10.1109\/IWASI.2015.7184945"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"21286","DOI":"10.1109\/ACCESS.2021.3052969","article-title":"A Q-learning-based approach for enhancing energy efficiency of Bluetooth Low Energy","volume":"9","author":"Fu","year":"2021","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rioual, Y., Le Moullec, Y., Laurent, J., Khan, M.I., and Diguet, J.P. (2018, January 8\u201310). Reward function evaluation in a reinforcement learning approach for energy management. Proceedings of the 2018 16th Biennial Baltic Electronics Conference (BEC), Tallinn, Estonia.","DOI":"10.1109\/BEC.2018.8600992"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.compeleceng.2015.01.005","article-title":"Bluetooth for Internet of Things: A fuzzy approach to improve power management in smart homes","volume":"44","author":"Collotta","year":"2015","journal-title":"Comput. Electr. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1339\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:15:11Z","timestamp":1760120111000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/3\/1339"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,25]]},"references-count":30,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["s23031339"],"URL":"https:\/\/doi.org\/10.3390\/s23031339","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,25]]}}}