{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T16:35:01Z","timestamp":1776184501415,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2024,8,15]],"date-time":"2024-08-15T00:00:00Z","timestamp":1723680000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ITRC (Information Technology Research Center)","award":["IITP-2024-2021-0-01816"],"award-info":[{"award-number":["IITP-2024-2021-0-01816"]}]},{"name":"IITP (Institute for Information &amp; Communications Technology Planning &amp; Evaluation)","award":["IITP-2024-2021-0-01816"],"award-info":[{"award-number":["IITP-2024-2021-0-01816"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Time-sensitive networking (TSN) technologies have garnered attention for supporting time-sensitive communication services, with recent interest extending to the wireless domain. However, adapting TSN to wireless areas faces challenges due to the competitive channel utilization in IEEE 802.11, necessitating exclusive channels for low-latency services. Additionally, traditional TSN scheduling algorithms may cause significant transmission delays due to dynamic wireless characteristics, which must be addressed. This paper proposes a wireless TSN model of IEEE 802.11 networks for the exclusive channel access and a novel time-sensitive traffic scheduler, named the wireless intelligent scheduler (WISE), based on deep reinforcement learning. We designed a deep reinforcement learning (DRL) framework to learn the repetitive transmission patterns of time-sensitive traffic and address potential latency issues from changing wireless conditions. Within this framework, we identified the most suitable DRL model, presenting the WISE algorithm with the best performance. Experimental results indicate that the proposed mechanisms meet up to 99.9% under the various wireless communication scenarios. In addition, they show that the processing delay is successfully limited within the specific time requirements and the scalability of TSN streams is guaranteed by the proposed mechanisms.<\/jats:p>","DOI":"10.3390\/s24165281","type":"journal-article","created":{"date-parts":[[2024,8,15]],"date-time":"2024-08-15T03:49:36Z","timestamp":1723693776000},"page":"5281","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Deep Reinforcement Learning-Based Adaptive Scheduling for Wireless Time-Sensitive Networking"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2436-3784","authenticated-orcid":false,"given":"Hanjin","family":"Kim","sequence":"first","affiliation":[{"name":"Future Convergence Engineering Major, Department of Computer Science and Engineering, Korea University of Technology and Education, Cheonan-si 31253, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9772-021X","authenticated-orcid":false,"given":"Young-Jin","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence Big Data, Sehan University, Dangjin-si 31746, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3426-3792","authenticated-orcid":false,"given":"Won-Tae","family":"Kim","sequence":"additional","affiliation":[{"name":"Future Convergence Engineering Major, Department of Computer Science and Engineering, Korea University of Technology and Education, Cheonan-si 31253, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kang, Y., Lee, S., Gwak, S., Kim, T., and An, D. (2021). Time-sensitive networking technologies for industrial automation in wireless communication systems. Energies, 14.","DOI":"10.3390\/en14154497"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TCE.2023.3279331","article-title":"Time sensitive networking-driven deterministic low-latency communication for real-time telemedicine and e-health services","volume":"69","author":"Lu","year":"2023","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Hazarika, A., and Rahmati, M. (2023). Towards an evolved immersive experience: Exploring 5G-and beyond-enabled ultra-low-latency communications for augmented and virtual reality. Sensors, 23.","DOI":"10.3390\/s23073682"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14375","DOI":"10.1109\/JIOT.2023.3264909","article-title":"A survey on in-vehicle time sensitive networking","volume":"10","author":"Peng","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fedullo, T., Morato, A., Tramarin, F., Rovati, L., and Vitturi, S. (2022). A comprehensive review on time sensitive networks with a special focus on its applicability to industrial smart and distributed measurement systems. Sensors, 22.","DOI":"10.3390\/s22041638"},{"key":"ref_6","first-page":"1061","article-title":"Online QoS management for multimedia real-time transmission in industrial networks","volume":"58","author":"Almeida","year":"2010","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/MCOMSTD.2018.8412457","article-title":"Time-sensitive networking standards","volume":"2","author":"Farkas","year":"2018","journal-title":"IEEE Commun. Stand. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"61192","DOI":"10.1109\/ACCESS.2023.3286370","article-title":"A survey of scheduling algorithms for the time-aware shaper in time-sensitive networking (TSN)","volume":"11","author":"Osswald","year":"2023","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1109\/MIE.2021.3078071","article-title":"Clock synchronization for wireless time-sensitive networking: A march from microsecond to nanosecond","volume":"16","author":"Seijo","year":"2021","journal-title":"IEEE Ind. Electron. Mag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7351","DOI":"10.1109\/TII.2022.3151786","article-title":"Wireless time sensitive networking impact on an industrial collaborative robotic workcell","volume":"18","author":"Sudhakaran","year":"2022","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_11","unstructured":"Bush, S.F., Mantelet, G., Thomsen, B., and Grossman, E. (2018). Industrial Wireless Time-Sensitive Networking: RFC on the Path Forward, Avnu Alliance. Avnu Alliance White Paper."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/OJIES.2021.3135524","article-title":"When IEEE 802.11 and 5G meet time-sensitive networking","volume":"3","author":"Atiq","year":"2021","journal-title":"IEEE Open J. Ind. Electron. Soc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1109\/MCOMSTD.0002.2200039","article-title":"WiFi TSN: Enabling deterministic wireless connectivity over 802.11","volume":"6","author":"Cavalcanti","year":"2022","journal-title":"IEEE Commun. Stand. Mag."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Haxhibeqiri, J., Jiao, X., Aslam, M., Moerman, I., and Hoebeke, J. (2021, January 10\u201312). Enabling TSN over IEEE 802.11: Low-overhead time synchronization for Wi-Fi clients. Proceedings of the 2021 22nd IEEE International Conference on Industrial Technology (ICIT), Virtual.","DOI":"10.1109\/ICIT46573.2021.9453686"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2287531","DOI":"10.1155\/2021\/2287531","article-title":"Quality of services based on intelligent IoT WLAN MAC protocol dynamic real-time applications in smart cities","volume":"2021","author":"Alnazir","year":"2021","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MCOMSTD.0001.2100082","article-title":"Overview and performance evaluation of Wi-Fi 7","volume":"6","author":"Chen","year":"2022","journal-title":"IEEE Commun. Stand. Mag."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3793","DOI":"10.1007\/s11276-020-02295-2","article-title":"MAC-layer rate control for 802.11 networks: A survey","volume":"26","author":"Yin","year":"2020","journal-title":"Wirel. Netw."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"6751","DOI":"10.1109\/ACCESS.2020.2964690","article-title":"Real-time scheduling of massive data in time sensitive networks with a limited number of schedule entries","volume":"8","author":"Jin","year":"2020","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9736","DOI":"10.1109\/TII.2024.3385503","article-title":"Performance Comparison of Offline Scheduling Algorithms for the Time-Aware Shaper (TAS)","volume":"20","author":"Eppler","year":"2024","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: A brief survey","volume":"34","author":"Arulkumaran","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"30687","DOI":"10.1109\/ACCESS.2024.3369481","article-title":"Unlocking Mobility for Wi-Fi-based Wireless Time-Sensitive Networks","volume":"12","author":"Haxhibeqiri","year":"2024","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Jayabal, R.J., Wong, D.T.C., Goh, L.K., Pang, C.M., Sun, S., Jin, B., Ma, Y., Goh, L.M., and Cheng, W.C. (October, January 27). TGT-HC: A time-aware shaper scheduled hyperchannel protocol for wireless time sensitive networks (TSNs). Proceedings of the 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), Virtual.","DOI":"10.1109\/VTC2021-Fall52928.2021.9625170"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Fang, J., Sudhakaran, S., Cavalcanti, D., Cordeiro, C., and Chen, C. (2021, January 15\u201317). Wireless TSN with multi-radio wi-fi. Proceedings of the 2021 IEEE Conference on Standards for Communications and Networking (CSCN), Virtual.","DOI":"10.1109\/CSCN53733.2021.9686180"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1109\/JSAC.2019.2898744","article-title":"5G industrial networks with CoMP for URLLC and time sensitive network architecture","volume":"37","author":"Khoshnevisan","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"8944","DOI":"10.1109\/ACCESS.2018.2805378","article-title":"Precise clock synchronization in high performance wireless communication for time sensitive networking","volume":"6","author":"Shrestha","year":"2018","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/IOTM.001.2200070","article-title":"Controlled channel access for IEEE 802.11-based wireless tsn networks","volume":"6","author":"Avallone","year":"2023","journal-title":"IEEE Internet Things Mag."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Pe\u00f3n, P.G., Karachatzis, P., Steiner, W., and Uhlemann, E. (September, January 30). Time-Sensitive Networking\u2019s Scheduled Traffic Implementation on IEEE 802.11 COTS Devices. Proceedings of the 2023 IEEE 29th International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA), Niigata, Japan.","DOI":"10.1109\/RTCSA58653.2023.00028"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Schneider, B., Sofia, R.C., and Kovatsch, M. (2022, January 25\u201329). A proposal for time-aware scheduling in wireless industrial iot environments. Proceedings of the NOMS 2022\u20142022 IEEE\/IFIP Network Operations and Management Symposium, Budapest, Hungary.","DOI":"10.1109\/NOMS54207.2022.9789864"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, Z., Yang, J., Guo, C., Xiao, J., Tao, T., and Li, C. (2024). A Joint Scheduling Scheme for WiFi Access TSN. Sensors, 24.","DOI":"10.3390\/s24082554"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tan, Q., He, J., and Gao, Y. (2024, January 10\u201312). Deep Reinforcement Learning based OFDMA Scheduling for WiFi Networks with Coexisting Latency-Sensitive and High-Throughput Services. Proceedings of the 2024 5th Information Communication Technologies Conference (ICTC), Nanjing, China.","DOI":"10.1109\/ICTC61510.2024.10601889"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Adame, T., Carrascosa-Zamacois, M., and Bellalta, B. (2021). Time-sensitive networking in IEEE 802.11 be: On the way to low-latency WiFi 7. Sensors, 21.","DOI":"10.3390\/s21154954"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1529","DOI":"10.1109\/TWC.2020.3034173","article-title":"Scheduling channel access based on target wake time mechanism in 802.11 ax WLANs","volume":"20","author":"Chen","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1109\/JSAC.2021.3087246","article-title":"DeepMux: Deep-learning-based channel sounding and resource allocation for IEEE 802.11 ax","volume":"39","author":"Sangdeh","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"8868","DOI":"10.1109\/TWC.2024.3355276","article-title":"Multi-Agent Reinforcement Learning based Uplink OFDMA for IEEE 802.11ax Networks","volume":"23","author":"Han","year":"2024","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"110389","DOI":"10.1016\/j.comnet.2024.110389","article-title":"AoI minimization of ambient backscatter-assisted EH-CRN with cooperative spectrum sensing","volume":"245","author":"Liu","year":"2024","journal-title":"Comput. Netw."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"91710","DOI":"10.1109\/ACCESS.2021.3092572","article-title":"Run-time recovery and failure analysis of time-triggered traffic in time sensitive networks","volume":"9","author":"Kong","year":"2021","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"142764","DOI":"10.1109\/ACCESS.2023.3343409","article-title":"Joint Scheduling and Routing Optimization for Deterministic Hybrid Traffic in Time-Sensitive Networks using Constraint Programming","volume":"11","author":"Akram","year":"2023","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1109\/JPROC.2019.2905334","article-title":"A perspective on IEEE time-sensitive networking for industrial communication and automation systems","volume":"107","author":"Bello","year":"2019","journal-title":"Proc. IEEE"},{"key":"ref_39","unstructured":"Craciunas, S.S., Oliver, R.S., and Ag, T. (September, January 28). An overview of scheduling mechanisms for time-sensitive networks. Proceedings of the Real-Time Summer School L\u00c9cole d\u00c9t\u00e9 Temps R\u00e9el (ETR), Paris, France."},{"key":"ref_40","unstructured":"Cavalcanti, D., Bush, S., Illouz, M., Kronauer, G., Regev, A., and Venkatesan, G. (2020). Wireless TSN\u2013Definitions, Use Cases & Standards Roadmap, Avnu Alliance."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1109\/TII.2016.2629669","article-title":"Clock synchronization over IEEE 802.11\u2014A survey of methodologies and protocols","volume":"13","author":"Mahmood","year":"2016","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"18305","DOI":"10.1109\/JIOT.2022.3158701","article-title":"Fine time measurement for the Internet of Things: A practical approach using ESP32","volume":"9","author":"Vales","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Gringoli, F., Schulz, M., Link, J., and Hollick, M. (2019, January 25). Free your CSI: A channel state information extraction platform for modern Wi-Fi chipsets. Proceedings of the 13th International Workshop on Wireless Network Testbeds, Experimental Evaluation & Characterization, Los Cabos, Mexico.","DOI":"10.1145\/3349623.3355477"},{"key":"ref_44","unstructured":"Dulac-Arnold, G., Evans, R., van Hasselt, H., Sunehag, P., Lillicrap, T., Hunt, J., Mann, T., Weber, T., Degris, T., and Coppin, B. (2016). Deep Reinforcement Learning in Large Discrete Action Spaces. arXiv."},{"key":"ref_45","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1834","DOI":"10.1109\/OJCOMS.2022.3212939","article-title":"Delay-Guaranteeing Admission Control for Time-Sensitive Networking Using the Credit-Based Shaper","volume":"3","author":"Maile","year":"2022","journal-title":"IEEE Open J. Commun. Soc."},{"key":"ref_47","unstructured":"(2013). IEEE Standard for Information Technology\u2013Telecommunications and Information Exchange between Systems Local and Metropolitan Area Networks\u2013Specific Requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications\u2013Amendment 4: Enhancements for Very High Throughput for Operation in Bands below 6 GHz (Standard No. IEEE 802.11ac-2013)."},{"key":"ref_48","unstructured":"Towers, M., Terry, J.K., Kwiatkowski, A., Balis, J.U., Cola, G.d., Deleu, T., Goul\u00e3o, M., Kallinteris, A., KG, A., and Krimmel, M. (2023). Gymnasium. OpenAI Gym."},{"key":"ref_49","first-page":"1","article-title":"Stable-baselines3: Reliable reinforcement learning implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/16\/5281\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:36:57Z","timestamp":1760110617000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/16\/5281"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,15]]},"references-count":49,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["s24165281"],"URL":"https:\/\/doi.org\/10.3390\/s24165281","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,15]]}}}