{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T16:13:23Z","timestamp":1790352803966,"version":"4.1.0"},"reference-count":30,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T00:00:00Z","timestamp":1697241600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U21B2003"],"award-info":[{"award-number":["U21B2003"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Low-Earth orbit (LEO) satellites have limited on-board resources, user terminals are unevenly distributed in the constantly changing coverage area, and the service requirements vary significantly. It is urgent to optimize resource allocation under the constraint of limited satellite spectrum resources and ensure the fairness of service admission control. Therefore, we propose an intelligent hierarchical admission control (IHAC) strategy based on deep reinforcement learning (DRL). This strategy combines the deep deterministic policy gradient (DDPG) and the deep Q network (DQN) intelligent algorithm to construct upper and lower hierarchical resource allocation and admission control frameworks. The upper controller considers the state features of each ground zone and satellite resources from a global perspective, and determines the beam resource allocation ratio of each ground zone. The lower controller formulates the admission control policy based on the decision of the upper controller and the detailed information of the users\u2019 services. At the same time, a designed reward and punishment mechanism is used to optimize the decisions of the upper and lower controllers. The fairness of users\u2019 services admissions in each ground zone is achieved as far as possible while ensuring the reasonable allocation of beam resources among zones. Finally, online decision-making and offline learning were combined, so that the controller could make full use of a large number of historical data to learn and generate intelligent strategies with stronger adaptive ability while interacting with the network environment in real time. A large number of simulation results show that IHAC has better performance in terms of a successful service admission rate, service drop rate, and fair resource allocation. Among them, the number of accepted services increased by 20.36% on average, the packet loss rate decreased by 17.56% on average, and the resource fairness increased by 17.16% on average.<\/jats:p>","DOI":"10.3390\/s23208470","type":"journal-article","created":{"date-parts":[[2023,10,14]],"date-time":"2023-10-14T14:59:59Z","timestamp":1697295599000},"page":"8470","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Intelligent Hierarchical Admission Control for Low-Earth Orbit Satellites Based on Deep Reinforcement Learning"],"prefix":"10.3390","volume":"23","author":[{"given":"Debin","family":"Wei","sequence":"first","affiliation":[{"name":"Communication and Network Laboratory, Dalian University, Dalian 116622, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6051-5578","authenticated-orcid":false,"given":"Chuanqi","family":"Guo","sequence":"additional","affiliation":[{"name":"Communication and Network Laboratory, Dalian University, Dalian 116622, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Automation, Nanjing University of Science and Technology, Nanjing 210094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"181656","DOI":"10.1109\/ACCESS.2019.2959824","article-title":"Beam Coverage Comparison of LEO Satellite Systems Based on User Diversification","volume":"7","author":"Xia","year":"2019","journal-title":"IEEE Access"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"121301","DOI":"10.1007\/s11432-022-3566-4","article-title":"Applications and prospects of artificial intelligence in covert satellite communication: A review","volume":"66","author":"Lu","year":"2023","journal-title":"Sci. China Inf. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"179","DOI":"10.23919\/JCC.2022.06.014","article-title":"User-level scheduling and resource allocation for multi-beam satellite systems with full frequency reuse","volume":"19","author":"Leng","year":"2022","journal-title":"China Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2448","DOI":"10.1109\/LCOMM.2020.3033357","article-title":"Radio Resource Management Techniques for Multibeam Satellite Systems","volume":"25","author":"Kisseleff","year":"2021","journal-title":"IEEE Commun. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1109\/TBC.2017.2755263","article-title":"Radio Resource Management Optimization of Flexible Satellite Payloads for DVB-S2 Systems","volume":"64","author":"Cocco","year":"2018","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6345","DOI":"10.1109\/TCOMM.2021.3088898","article-title":"Secrecy-energy efficient hybrid beamforming for satellite-terrestrial integrated networks","volume":"69","author":"Lin","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3717","DOI":"10.1109\/TAES.2022.3155711","article-title":"Refracting RIS-aided hybrid satellite-terrestrial relay networks: Joint beamforming design and optimization","volume":"58","author":"Lin","year":"2022","journal-title":"IEEE Tran. Aerosp. Electron. Syst."},{"key":"ref_8","first-page":"2085","article-title":"SLNR-based secure energy efficient beamforming in multibeam satellite systems","volume":"59","author":"Lin","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Deng, Z., Yu, X., Lin, W., Wang, K., Liu, H., Gu, L., Liu, Y., and Ma, X. (2021). A Multi-Beam Satellite Cooperative Transmission Scheme Based on Resources Optimization and Packets Segmentation. Electronics, 10.","DOI":"10.3390\/electronics10222841"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4269","DOI":"10.1109\/TAES.2021.3100335","article-title":"Potential Game-Based Radio Resource Allocation in Uplink Multibeam Satellite IoT Network","volume":"57","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1248","DOI":"10.1109\/LWC.2019.2912753","article-title":"Adaptive Power Resource Allocation with Multi-Beam Directivity Control in High-Throughput Satellite Communication Systems","volume":"8","author":"Takahashi","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"6052","DOI":"10.1109\/JIOT.2018.2870878","article-title":"Interbeam Interference Constrained Resource Allocation for Shared Spectrum Multibeam Satellite Communication Systems","volume":"6","author":"Jia","year":"2019","journal-title":"IEEE Internet Things"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, Y., Bian, D., Hu, J., Tang, J., and Wang, C. (2019, January 11\u201313). A Flexible Resource Allocation Algorithm in Full Bandwidth Beam Hopping Satellite Systems. Proceedings of the 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC), Chongqing, China.","DOI":"10.1109\/IMCEC46724.2019.8984132"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cai, B., Zhang, Q., Ge, J., and Xie, W. (2023). Resource Allocation for Cognitive LEO Satellite Systems: Facilitating IoT Communications. Sensors, 23.","DOI":"10.3390\/s23083875"},{"key":"ref_15","unstructured":"Kaloxylos, A., Gavras, A., Camps, D., and Ghoraishi, M. (2021). AI and ML\u2013Enablers for beyond 5G Networks, Centres de Recerca de Catalunya (CERCA)."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Homssi, B.A., Dakic, K., Wang, K., Allen, B., Kandeepan, S., AI-Hourani, A., and Saad, W. (2022). Artificial intelligence techniques for next-generation mega satellite networks. arXiv.","DOI":"10.36227\/techrxiv.20073125.v1"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Trakadas, P., Masip-Bruin, X., Facca, F.M., Spantideas, S.T., Giannopoulos, A.E., Kapsalis, N.C., Martins, R., Bosani, E., Ramon, J., and Prats, R.G. (2022). A Reference Architecture for Cloud\u2013Edge Meta-Operating Systems Enabling Cross-Domain, Data-Intensive, ML-Assisted Applications: Architectural Overview and Key Concepts. Sensors, 22.","DOI":"10.3390\/s22229003"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"996","DOI":"10.1109\/TGCN.2022.3186792","article-title":"Optimal computation offloading in collaborative leo-iot enabled mec: A multi-agent deep reinforcement learning approach","volume":"7","author":"Lyu","year":"2022","journal-title":"IEEE Trans. Green Commun. Netw."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Karamplias, T., Spantideas, S.T., Giannopoulos, A.E., Gkoniset, P., Kapsalisal, N., and Trakadas, P. (2022, January 7\u201310). Towards closed-loop automation in 5g open ran: Coupling an open-source simulator with xapps. Proceedings of the 2022 Joint European Conference on Networks and Communications & 6G Summit (EuCNC\/6G Summit), Grenoble, France.","DOI":"10.1109\/EuCNC\/6GSummit54941.2022.9815658"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2785","DOI":"10.1109\/LCOMM.2020.3019437","article-title":"Distributed intelligence: A verification for multi-agent DRL-based multibeam satellite resource allocation","volume":"24","author":"Liao","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"955","DOI":"10.4218\/etrij.2022-0192","article-title":"Dynamic power and bandwidth allocation for DVB-based LEO satellite systems","volume":"44","author":"Chan","year":"2022","journal-title":"ETRI J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"9849","DOI":"10.1109\/TVT.2020.3002983","article-title":"Multi-Agent Deep Reinforcement Learning-Based Flexible Satellite Payload for Mobile Terminals","volume":"69","author":"Hu","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"77","DOI":"10.23919\/JCC.2022.01.007","article-title":"Multi-objective deep reinforcement learning based time-frequency resource allocation for multi-beam satellite communications","volume":"19","author":"He","year":"2022","journal-title":"China Commun."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1016\/j.cja.2022.12.012","article-title":"Sequential dynamic resource allocation in multi-beam satellite systems: A learning-based optimization method","volume":"36","author":"Huang","year":"2023","journal-title":"Chin. J. Aeronaut."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2925","DOI":"10.1109\/TAC.2020.3014117","article-title":"Dynamic admission quota control with controllable and uncontrollable demands and random service time","volume":"66","author":"Dai","year":"2020","journal-title":"IEEE Trans. Automat. Contr."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"30873","DOI":"10.1109\/ACCESS.2023.3262117","article-title":"Joint QoS Aware Admission Control and Power Allocation in NOMA Downlink Networks","volume":"11","author":"Goudos","year":"2023","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1109\/LWC.2020.3043073","article-title":"Dynamic channel reservation strategy based on DQN algorithm for multi-service LEO satellite communication system","volume":"10","author":"Li","year":"2020","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e4757","DOI":"10.1002\/dac.4757","article-title":"On using reinforcement learning for network slice admission control in 5G: Offline vs. online","volume":"34","author":"Bakri","year":"2021","journal-title":"Int. J. Commun. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wu, Z., Ishigaki, G., Gour, R., and Jue, J.P. (2021, January 7\u201311). A reinforcement learning-based admission control strategy for elastic network slices. Proceedings of the 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain.","DOI":"10.1109\/GLOBECOM46510.2021.9685178"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Choi, H., and Pack, S. (2022). Cooperative Downloading for LEO Satellite Networks: A DRL-Based Approach. Sensors, 22.","DOI":"10.3390\/s22186853"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8470\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:07:04Z","timestamp":1760130424000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8470"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,14]]},"references-count":30,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23208470"],"URL":"https:\/\/doi.org\/10.3390\/s23208470","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,14]]}}}