{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,25]],"date-time":"2026-07-25T03:50:33Z","timestamp":1784951433007,"version":"3.55.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T00:00:00Z","timestamp":1709856000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T00:00:00Z","timestamp":1709856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cloud Comp"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In the LEO satellite communication system, the resource utilization rate is very low due to the constrained resources on satellites and the non-uniform distribution of traffics. In addition, the rapid movement of LEO satellites leads to complicated and changeable networks, which makes it difficult for traditional resource allocation strategies to improve the resource utilization rate. To solve the above problem, this paper proposes a resource allocation strategy based on deep reinforcement learning. The strategy takes the weighted sum of spectral efficiency, energy efficiency and blocking rate as the optimization objective, and constructs a joint power and channel allocation model. The strategy allocates channels and power according to the number of channels, the number of users and the type of business. In the reward decision mechanism, the maximum reward is obtained by maximizing the increment of the optimization target. However, during the optimization process, the decision always focuses on the optimal allocation for current users, and ignores QoS for new users. To avoid the situation, current service beams are integrated with high- traffic beams, and states of beams are refactored to maximize long-term benefits to improve system performance.<\/jats:p><jats:p>Simulation experiments show that in scenarios with a high number of users, the proposed resource allocation strategy reduces the blocking rate by at least 5% compared to reinforcement learning methods, effectively enhancing resource utilization.<\/jats:p>","DOI":"10.1186\/s13677-024-00621-z","type":"journal-article","created":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T11:01:46Z","timestamp":1709895706000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Multi-dimensional resource allocation strategy for LEO satellite communication uplinks based on deep reinforcement learning"],"prefix":"10.1186","volume":"13","author":[{"given":"Yu","family":"Hu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feipeng","family":"Qiu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fei","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jilong","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,3,8]]},"reference":[{"issue":"2","key":"621_CR1","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.dcan.2021.07.008","volume":"8","author":"N Ye","year":"2022","unstructured":"Ye N, Jihong Yu, Wang A, Zhang R (2022) Help from space: grant-free massive access for satellite-based IoT in the 6G era [J]. Digital Communications and Networks 8(2):215\u2013224","journal-title":"Digital Communications and Networks"},{"key":"621_CR2","doi-asserted-by":"publisher","DOI":"10.1145\/3511904","author":"F Wang","year":"2022","unstructured":"Wang F, Li G, Wang Y, Rafique W, Khosravi MR, Liu G, Liu Y, Qi L (2022) Privacy-aware traffic flow prediction based on multi-party sensor data with zero trust in Smart City [J]. ACM Trans Internet Technol. https:\/\/doi.org\/10.1145\/3511904","journal-title":"ACM Trans Internet Technol"},{"key":"621_CR3","doi-asserted-by":"publisher","DOI":"10.1109\/TNSE.2022.3157730","author":"Y Yang","year":"2022","unstructured":"Yang Y, Yang X, Heidari M, Srivastava G, Khosravi MR, Qi L (2022) ASTREAM: data-stream-driven scalable anomaly detection with accuracy guarantee in IIoT environment [J]. IEEE Trans Netw Sci Eng. https:\/\/doi.org\/10.1109\/TNSE.2022.3157730","journal-title":"IEEE Trans Netw Sci Eng"},{"issue":"6","key":"621_CR4","doi-asserted-by":"publisher","first-page":"1077","DOI":"10.1016\/j.dcan.2022.02.005","volume":"8","author":"G Li","year":"2022","unstructured":"Li G, Zijie Hong Yu, Pang YX, Huang Z (2022) Resource allocation for sum-rate maximization in NOMA-based generalized spatial modulation [J]. Digital Communications and Networks 8(6):1077\u20131084","journal-title":"Digital Communications and Networks"},{"issue":"2","key":"621_CR5","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.dcan.2021.06.007","volume":"8","author":"H Xie","year":"2022","unstructured":"Xie H, Yongjun Xu (2022) Robust Resource Allocation for NOMA-assisted Heterogeneous Networks [J]. Digital Communications and Networks 8(2):208\u2013214","journal-title":"Digital Communications and Networks"},{"key":"621_CR6","unstructured":"Hang L, Zhe Z, Zhen G, et al (2014) Dynamic Channel Assignment Scheme with Cooperative Beam Forming for Multi-beam mobile satellite networks [C]. 6th International Conference on Wireless Communications and Signal Processing (WCSP), IEEE, 1\u20135"},{"key":"621_CR7","doi-asserted-by":"crossref","unstructured":"Umehira M (2012) Centralized Dynamic Channel Assignment Schemes for Multi-beam Mobile Satellite Communications Systems [C]. AIAA International Communications Satellite System Conference (ICSSC), 24\u201327","DOI":"10.2514\/6.2012-15123"},{"key":"621_CR8","doi-asserted-by":"crossref","unstructured":"Umehira M, Fujita S, Zhen G, et al (2014) Dynamic Channel Assignment Based on Interference Measurement with Threshold for Multi-beam Mobile Satellite Networks [C]. Communications","DOI":"10.1109\/APCC.2013.6766037"},{"key":"621_CR9","doi-asserted-by":"crossref","unstructured":"Chang R, He Y, Cui G, et al (2016) An allocation scheme between random access and DAMA channels for satellite networks [C]. IEEE International Conference on Communication Systems (ICCS), IEEE, 1\u20135","DOI":"10.1109\/ICCS.2016.7833609"},{"issue":"6","key":"621_CR10","doi-asserted-by":"publisher","first-page":"2983","DOI":"10.1109\/TWC.2005.858365","volume":"4","author":"JP Choi","year":"2005","unstructured":"Choi JP, Chan VWS (2005) Optimum power and beam allocation based on traffic demands and channel conditions over satellite downlinks [J]. IEEE Trans Wireless Commun 4(6):2983\u20132993","journal-title":"IEEE Trans Wireless Commun"},{"key":"621_CR11","doi-asserted-by":"crossref","unstructured":"Lutz E (2015) Co-channel interference in high-throughput multi-beam satellite systems [C]. 2015 IEEE International Conference on Communications (ICC), IEEE, 885\u2013891","DOI":"10.1109\/ICC.2015.7248434"},{"issue":"05","key":"621_CR12","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1007\/978-981-15-4902-1_5","volume":"41","author":"L Wang","year":"2021","unstructured":"Wang L, Zheng J, He C et al (2021) Resource allocation in high throughput multi-beam communication satellite systems [J]. Chin Space Sci Technol 41(05):85\u201394","journal-title":"Chin Space Sci Technol"},{"key":"621_CR13","first-page":"103","volume":"21","author":"Y Shi","year":"2018","unstructured":"Shi Y, Zhang BN, Guo DX et al (2018) Joint Power and Bandwidth Allocation Algorithm with Inter-beam Interference for Multi-beam Satellite [J]. Comput Eng 21:103\u2013106","journal-title":"Comput Eng"},{"issue":"1","key":"621_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JSEE.2015.00001","volume":"26","author":"J Zhou","year":"2015","unstructured":"Zhou J, Ye X, Pan Y et al (2015) Dynamic channel reservation scheme based on priorities in LEO satellite systems [J]. J Syst Eng Electron 26(1):1\u20139","journal-title":"J Syst Eng Electron"},{"key":"621_CR15","doi-asserted-by":"publisher","first-page":"75192","DOI":"10.1109\/ACCESS.2018.2882795","volume":"6","author":"P Zuo","year":"2018","unstructured":"Zuo P, Peng T, Linghu W et al (2018) resource allocation for cognitive satellite communications downlink [J]. IEEE Access 6:75192\u201375205","journal-title":"IEEE Access"},{"key":"621_CR16","doi-asserted-by":"publisher","first-page":"113856","DOI":"10.1016\/j.rse.2023.113856","volume":"299","author":"D Hong","year":"2023","unstructured":"Hong D, Zhang B, Li H et al (2023) Cross-city matters: a multimodal remote sensing benchmark dataset for cross-city semantic segmentation using high-resolution domain adaptation networks. Remote Sens Environ 299:113856","journal-title":"Remote Sens Environ"},{"issue":"1","key":"621_CR17","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1038\/s41598-023-28282-z","volume":"13","author":"C Zengjing","year":"2023","unstructured":"Zengjing C, Wang Lu, Chengzhi X (2023) Efficient dynamic channel assignment through laser chaos: a multiuser parallel processing learning algorithm [J]. Sci Rep 13(1):1353","journal-title":"Sci Rep"},{"issue":"2","key":"621_CR18","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1109\/MWC.2016.1500356WC","volume":"24","author":"C Jiang","year":"2017","unstructured":"Jiang C, Zhang H, Ren Y et al (2017) Machine learning paradigms for next-generation wireless networks [J]. IEEE Wirel Commun 24(2):98\u2013105","journal-title":"IEEE Wirel Commun"},{"key":"621_CR19","doi-asserted-by":"crossref","unstructured":"Chen X, Zhang H, Tao C, et al (2013) Improving Energy Efficiency in Green Femtocell Networks: A Hierarchical Reinforcement Learning Framework [C]. IEEE International Conference on Communications (ICC), IEEE, 2241\u20132245","DOI":"10.1109\/ICC.2013.6654861"},{"key":"621_CR20","doi-asserted-by":"crossref","unstructured":"Wang Z, Zhang J, X Zhang, et al (2019) Reinforcement Learning Based Congestion Control in Satellite Internet of Things [C]. 11th International Conference on Wireless Communications and Signal Processing (WCSP), IEEE1\u20136","DOI":"10.1109\/WCSP.2019.8928132"},{"issue":"5","key":"621_CR21","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1016\/j.dcan.2021.09.013","volume":"8","author":"Y Zhi","year":"2022","unstructured":"Zhi Y, Tian J, Deng X, Qiao J, Dianjie Lu (2022) Deep reinforcement learning-based resource allocation for D2D communications in heterogeneous cellular networks [J]. Digital Communications and Networks 8(5):834\u2013842","journal-title":"Digital Communications and Networks"},{"issue":"1","key":"621_CR22","doi-asserted-by":"publisher","first-page":"16918","DOI":"10.1038\/s41598-021-96284-w","volume":"11","author":"X Liu","year":"2021","unstructured":"Liu X, Zheng J, Zhang M, Li Y, Wang R, He Y (2021) A novel D2D-MEC method for Rnhanced computation capability in cellular networks [J]. Sci Rep 11(1):16918","journal-title":"Sci Rep"},{"key":"621_CR23","doi-asserted-by":"crossref","unstructured":"Qiu Y, Ji Z, Zhu Y, et al (2018) Joint Mode Selection and Power Adaptation for D2D Communication with Reinforcement Learning [C]. 15th International Symposium on Wireless Communication Systems (ISWCS), 1\u20136","DOI":"10.1109\/ISWCS.2018.8491238"},{"key":"621_CR24","volume-title":"SpectralGPT: Spectral Foundation Model","author":"D Hong","year":"2023","unstructured":"Hong D, Zhang B, Li X et al (2023) SpectralGPT: Spectral Foundation Model"},{"key":"621_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2021.3130716","volume":"60","author":"D Hong","year":"2022","unstructured":"Hong D, Han Z, Yao J et al (2022) SpectralFormer: rethinking hyperspectral image classification with transformers. IEEE Trans Geosci Remote Sensing 60:1\u201315. https:\/\/doi.org\/10.1109\/TGRS.2021.3130716","journal-title":"IEEE Trans Geosci Remote Sensing"},{"key":"621_CR26","doi-asserted-by":"publisher","first-page":"9849","DOI":"10.1109\/TVT.2020.3002983","volume":"9","author":"X Hu","year":"2020","unstructured":"Hu X, Liao X, Liu Z et al (2020) Multi-agent deep reinforcement learning-based flexible satellite payload for mobile terminals [J]. IEEE Trans Veh Technol 9:9849\u20139865","journal-title":"IEEE Trans Veh Technol"},{"issue":"1","key":"621_CR27","doi-asserted-by":"publisher","first-page":"77","DOI":"10.23919\/JCC.2022.01.007","volume":"19","author":"Y He","year":"2022","unstructured":"He Y, Sheng B, Yin H, Yan D, Zhang Y (2022) Multi-objective deep reinforcement learning based time-frequency resource allocation for multi-beam satellite communications [J]. China Communications 19(1):77\u201391","journal-title":"China Communications"},{"key":"621_CR28","doi-asserted-by":"crossref","unstructured":"J. Li, J. Zhao, X. Sun (2021) Deep Reinforcement Learning Based Wireless Resource Allocation for V2X Communications [C]. 2021 13th International Conference on Wireless Communications and Signal Processing (WCSP), Changsha, China, 1\u20135","DOI":"10.1109\/WCSP52459.2021.9613367"},{"key":"621_CR29","doi-asserted-by":"crossref","unstructured":"Y. Han, C. Zhang, G. Zhang (2021) Dynamic Beam Hopping Resource Allocation Algorithm Based on Deep Reinforcement Learning in Multi-Beam Satellite Systems [C]. 2021 3rd International Academic Exchange Conference on Science and Technology Innovation (IAECST), Guangzhou, China, 68\u201373","DOI":"10.1109\/IAECST54258.2021.9695603"},{"key":"621_CR30","doi-asserted-by":"crossref","unstructured":"S. Ma, X. Hu, X. Liao, W. Wang (2021) Deep Reinforcement Learning for Dynamic Bandwidth Allocation in Multi-Beam Satellite Systems [C]. 2021 IEEE 6th International Conference on Computer and Communication Systems (ICCCS), Chengdu, China, 955\u2013959","DOI":"10.1109\/ICCCS52626.2021.9449160"},{"issue":"11","key":"621_CR31","doi-asserted-by":"publisher","first-page":"4593","DOI":"10.1109\/TFUZZ.2022.3158000","volume":"30","author":"Xu Xiaolong","year":"2022","unstructured":"Xiaolong Xu, Jiang Q, Zhang P, Cao X, Khosravi MR, Alex LT, Qi L, Dou W (2022) Game theory for distributed IoV task offloading with fuzzy neural network in edge computing [J]. IEEE Trans Fuzzy Syst 30(11):4593\u20134604","journal-title":"IEEE Trans Fuzzy Syst"},{"issue":"9","key":"621_CR32","doi-asserted-by":"publisher","first-page":"6300","DOI":"10.1109\/TII.2022.3154473","volume":"18","author":"Y Jia","year":"2022","unstructured":"Jia Y, Liu B, Dou W, Xiaolong Xu, Zhou X, Qi L, Yan Z (2022) CroApp: A CNN-based resource optimization approach in edge computing environment [J]. IEEE Trans Industr Inf 18(9):6300\u20136307","journal-title":"IEEE Trans Industr Inf"}],"container-title":["Journal of Cloud Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-024-00621-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13677-024-00621-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13677-024-00621-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,11]],"date-time":"2024-03-11T15:05:54Z","timestamp":1710169554000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofcloudcomputing.springeropen.com\/articles\/10.1186\/s13677-024-00621-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,8]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["621"],"URL":"https:\/\/doi.org\/10.1186\/s13677-024-00621-z","relation":{},"ISSN":["2192-113X"],"issn-type":[{"value":"2192-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,8]]},"assertion":[{"value":"2 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 March 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The research has consent for Ethical Approval.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"56"}}