{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T16:39:23Z","timestamp":1781714363642,"version":"3.54.5"},"reference-count":27,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T00:00:00Z","timestamp":1754870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>To meet the escalating demands of massive data transmission, the next generation of wireless networks will leverage the multi-access edge computing (MEC) architecture coupled with multi-access transmission technologies to enhance communication resource utilization. This paper presents queue stability-constrained reinforcement learning algorithms designed to optimize the transmission control mechanism in MEC systems to improve both throughput and reliability. We propose an analytical framework to model the queue stability. To increase transmission performance while maintaining queue stability, queueing delay model is designed to analyze the packet scheduling process by using the M\/M\/c queueing model and estimate the expected packet queueing delay. To handle the time-varying network environment, we introduce a queue stability constraint into the reinforcement learning reward function to jointly optimize latency and queue stability. The reinforcement learning algorithm is deployed at the MEC server to reduce the workload of central cloud servers. Simulation results validate that the proposed algorithm effectively controls queueing delay and average queue length while improving packet transmission success rates in dynamic MEC environments.<\/jats:p>","DOI":"10.3390\/a18080498","type":"journal-article","created":{"date-parts":[[2025,8,11]],"date-time":"2025-08-11T09:59:13Z","timestamp":1754906353000},"page":"498","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Queue Stability-Constrained Deep Reinforcement Learning Algorithms for Adaptive Transmission Control in Multi-Access Edge Computing Systems"],"prefix":"10.3390","volume":"18","author":[{"given":"Longzhe","family":"Han","sequence":"first","affiliation":[{"name":"Collaborative Sensing and Advanced Computing Research Center, Nanchang Institute of Technology, Nanchang 330099, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tian","family":"Zeng","sequence":"additional","affiliation":[{"name":"Collaborative Sensing and Advanced Computing Research Center, Nanchang Institute of Technology, Nanchang 330099, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jia","family":"Zhao","sequence":"additional","affiliation":[{"name":"Collaborative Sensing and Advanced Computing Research Center, Nanchang Institute of Technology, Nanchang 330099, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuecai","family":"Bao","sequence":"additional","affiliation":[{"name":"Collaborative Sensing and Advanced Computing Research Center, Nanchang Institute of Technology, Nanchang 330099, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangming","family":"Liu","sequence":"additional","affiliation":[{"name":"Collaborative Sensing and Advanced Computing Research Center, Nanchang Institute of Technology, Nanchang 330099, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8667-879X","authenticated-orcid":false,"given":"Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Computer and Software Engineering, East China Normal University, Shanghai 200062, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sylla, T., Mendiboure, L., Maaloul, S., Aniss, H., Chalouf, M.A., and Delbruel, S. (2022). Multi-Connectivity for 5G Networks and Beyond: A Survey. Sensors, 22.","DOI":"10.3390\/s22197591"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1109\/COMST.2024.3442149","article-title":"A Survey on Artificial Intelligence Techniques for Improved Rich Media Content Delivery in a 5G and Beyond Network Slicing Context","volume":"27","author":"Subhan","year":"2025","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"He, Z., Li, L., Lin, Z., Dong, Y., Qin, J., and Li, K. (2024). Joint Optimization of Service Migration and Resource Allocation in Mobile Edge\u2013Cloud Computing. Algorithms, 17.","DOI":"10.3390\/a17080370"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jun, S., Choi, Y.S., and Chung, H. (2024, January 16\u201318). A Study on Mobility Enhancement in 3GPP Multi-Radio Multi-Connectivity. Proceedings of the 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC62082.2024.10827748"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rahman, M.H., Chowdhury, M.R., Sultana, A., Tripathi, A., and Silva, A.P.D. (2024, January 2\u20135). Deep Learning Based Uplink Power Allocation in Multi-Radio Dual Connectivity Heterogeneous Wireless Networks. Proceedings of the 2024 IEEE 35th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), Valencia, Spain.","DOI":"10.1109\/PIMRC59610.2024.10817336"},{"key":"ref_6","first-page":"1","article-title":"MEC-Based Cooperative Multimedia Caching Mechanism for the Internet of Vehicles","volume":"2022","author":"Han","year":"2022","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101385","DOI":"10.1016\/j.iot.2024.101385","article-title":"Load-Balanced Offloading of Multiple Task Types for Mobile Edge Computing in IoT","volume":"28","author":"Zhang","year":"2024","journal-title":"Internet Things"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TCE.2025.3569343","article-title":"Fully-Decentralized Multi-MNO Interoperability of MEC-Enabled Cooperative Autonomous Mobility","volume":"1","author":"Santos","year":"2025","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"109181","DOI":"10.1016\/j.comnet.2022.109181","article-title":"Resource Allocation for MEC System with Multi-Users Resource Competition Based on Deep Reinforcement Learning Approach","volume":"215","author":"Qu","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2958","DOI":"10.1109\/TMC.2023.3268048","article-title":"Joint Service Caching, Resource Allocation and Task Offloading for MEC-Based Networks: A Multi-Layer Optimization Approach","volume":"23","author":"Chu","year":"2024","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1109\/TCOMM.2023.3325905","article-title":"DRL-Based Computation Rate Maximization for Wireless Powered Multi-AP Edge Computing","volume":"72","author":"Zhang","year":"2024","journal-title":"IEEE Trans. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, S., Tong, X., Chi, K., Gao, W., Chen, X., and Shi, Z. (2025). Stackelberg Game-Based Multi-Agent Algorithm for Resource Allocation and Task Offloading in MEC-Enabled C-ITS. IEEE Trans. Intell. Transport. Syst., 1\u201312.","DOI":"10.1109\/TITS.2025.3553487"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3590","DOI":"10.1109\/JSAC.2016.2611964","article-title":"Dynamic Computation Offloading for Mobile-Edge Computing with Energy Harvesting Devices","volume":"34","author":"Mao","year":"2016","journal-title":"IEEE J. Select. Areas Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103194","DOI":"10.1016\/j.sysarc.2024.103194","article-title":"Lyapunov-Guided Deep Reinforcement Learning for Delay-Aware Online Task Offloading in MEC Systems","volume":"153","author":"Dai","year":"2024","journal-title":"J. Syst. Archit."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4804","DOI":"10.1109\/JIOT.2018.2868616","article-title":"A Cooperative Partial Computation Offloading Scheme for Mobile Edge Computing Enabled Internet of Things","volume":"6","author":"Ning","year":"2019","journal-title":"IEEE Internet Things J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5031","DOI":"10.1109\/TVT.2019.2904244","article-title":"Collaborative Cloud and Edge Computing for Latency Minimization","volume":"68","author":"Ren","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1841","DOI":"10.1109\/TPDS.2021.3129618","article-title":"Online Learning for Distributed Computation Offloading in Wireless Powered Mobile Edge Computing Networks","volume":"33","author":"Wang","year":"2022","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"21246","DOI":"10.1109\/ACCESS.2018.2882000","article-title":"Resource Allocation Algorithm with Multi-Platform Intelligent Offloading in D2D-Enabled Vehicular Networks","volume":"7","author":"Cui","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","unstructured":"Bae, S., Han, S., and Sung, Y. (2020). A Reinforcement Learning Formulation of the Lyapunov Optimization: Application to Edge Computing Systems with Queue Stability. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"11789","DOI":"10.1109\/TVT.2024.3370196","article-title":"URLLC-Awared Resource Allocation for Heterogeneous Vehicular Edge Computing","volume":"73","author":"Wu","year":"2024","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"154811","DOI":"10.1016\/j.aeue.2023.154811","article-title":"Analytical Modelling of a Vehicular Ad Hoc Network Using Queueing Theory Models and the Notion of Channel Availability","volume":"170","author":"Keramidi","year":"2023","journal-title":"AEU-Int. J. Electron. Commun."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.jpdc.2023.02.008","article-title":"Multi-Agent DRL for Joint Completion Delay and Energy Consumption with Queuing Theory in MEC-Based IIoT","volume":"176","author":"Wu","year":"2023","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107650","DOI":"10.1016\/j.neunet.2025.107650","article-title":"Shapley Value-Driven Multi-Modal Deep Reinforcement Learning for Complex Decision-Making","volume":"191","author":"Zhang","year":"2025","journal-title":"Neural Netw."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Femminella, M., and Reali, G. (2024). Comparison of Reinforcement Learning Algorithms for Edge Computing Applications Deployed by Serverless Technologies. Algorithms, 17.","DOI":"10.3390\/a17080320"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"107149","DOI":"10.1016\/j.cor.2025.107149","article-title":"Deep Reinforcement Learning Based Proximal Policy Optimization Algorithm for Dynamic Job Shop Scheduling","volume":"183","author":"Yuan","year":"2025","journal-title":"Comput. Oper. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3045","DOI":"10.1016\/j.procs.2024.04.288","article-title":"An Adaptive Contention Window Using Actor-Critic Reinforcement Learning Algorithm for Vehicular Ad-Hoc NETworks","volume":"235","author":"Kumar","year":"2024","journal-title":"Procedia Comput. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"112395","DOI":"10.1016\/j.automatica.2025.112395","article-title":"An Approximate Policy Iteration Viewpoint of Actor\u2013Critic Algorithms","volume":"179","author":"Chen","year":"2025","journal-title":"Automatica"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/8\/498\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:24:30Z","timestamp":1760034270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/8\/498"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,11]]},"references-count":27,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["a18080498"],"URL":"https:\/\/doi.org\/10.3390\/a18080498","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,11]]}}}