{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,19]],"date-time":"2026-04-19T08:16:33Z","timestamp":1776586593114,"version":"3.51.2"},"reference-count":39,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T00:00:00Z","timestamp":1699401600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Seoul National University of Science and Technology"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>The need for reliable wireless communication in remote areas has led to the adoption of unmanned aerial vehicles (UAVs) as flying base stations (FlyBSs). FlyBSs hover over a designated area to ensure continuous communication coverage for mobile users on the ground. Moreover, rate-splitting multiple access (RSMA) has emerged as a promising interference management scheme in multi-user communication systems. In this paper, we investigate an RSMA-enhanced FlyBS downlink communication system and formulate an optimization problem to maximize the sum-rate of users, taking into account the three-dimensional FlyBS trajectory and RSMA parameters. To address this continuous non-convex optimization problem, we propose a TD3-RFBS optimization framework based on the twin-delayed deep deterministic policy gradient (TD3). This framework overcomes the limitations associated with the overestimation issue encountered in the deep deterministic policy gradient (DDPG), a well-known deep reinforcement learning method. Our simulation results demonstrate that TD3-RFBS outperforms existing solutions for FlyBS downlink communication systems, indicating its potential as a solution for future wireless networks.<\/jats:p>","DOI":"10.3390\/rs15225284","type":"journal-article","created":{"date-parts":[[2023,11,8]],"date-time":"2023-11-08T07:02:05Z","timestamp":1699426925000},"page":"5284","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["TD3-Based Optimization Framework for RSMA-Enhanced UAV-Aided Downlink Communications in Remote Areas"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2132-2290","authenticated-orcid":false,"given":"Tri-Hai","family":"Nguyen","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4680-1984","authenticated-orcid":false,"given":"Luong Vuong","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, FPT University, Da Nang 550000, Vietnam"}]},{"given":"L. Minh","family":"Dang","sequence":"additional","affiliation":[{"name":"Institute of Research and Development, Duy Tan University, Da Nang 550000, Vietnam"},{"name":"Faculty of Information Technology, Duy Tan University, Da Nang 550000, Vietnam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3464-3894","authenticated-orcid":false,"given":"Vinh Truong","family":"Hoang","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, Ho Chi Minh City Open University, Ho Chi Minh City 700000, Vietnam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7848-8470","authenticated-orcid":false,"given":"Laihyuk","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Seoul National University of Science and Technology, Seoul 01811, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1109\/COMST.2021.3059644","article-title":"Survey on Aerial Radio Access Networks: Toward a Comprehensive 6G Access Infrastructure","volume":"23","author":"Dao","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.neucom.2022.04.098","article-title":"Orbital collaborative learning in 6G space-air-ground integrated networks","volume":"497","author":"Zhao","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dong, F., Song, J., Zhang, Y., Wang, Y., and Huang, T. (2023). DRL-Based Load-Balancing Routing Scheme for 6G Space\u2013Air\u2013Ground Integrated Networks. Remote Sens., 15.","DOI":"10.3390\/rs15112801"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"5704","DOI":"10.1109\/JIOT.2022.3161571","article-title":"Intelligent Offloading and Resource Allocation in Heterogeneous Aerial Access IoT Networks","volume":"10","author":"Lakew","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1109\/COMST.2022.3171135","article-title":"What Will the Future of UAV Cellular Communications Be? A Flight From 5G to 6G","volume":"24","author":"Geraci","year":"2022","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shi, J., Cong, P., Zhao, L., Wang, X., Wan, S., and Guizani, M. (2023). A Two-Stage Strategy for UAV-enabled Wireless Power Transfer in Unknown Environments. IEEE Trans. Mob. Comput., in press.","DOI":"10.1109\/TMC.2023.3240763"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1109\/MCOM.2015.7263349","article-title":"Non-orthogonal multiple access for 5G: Solutions, challenges, opportunities, and future research trends","volume":"53","author":"Dai","year":"2015","journal-title":"IEEE Commun. Mag."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2073","DOI":"10.1109\/COMST.2022.3191937","article-title":"Rate-Splitting Multiple Access: Fundamentals, Survey, and Future Research Trends","volume":"24","author":"Mao","year":"2022","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1109\/JSAC.2023.3242718","article-title":"A Primer on Rate-Splitting Multiple Access: Tutorial, Myths, and Frequently Asked Questions","volume":"41","author":"Clerckx","year":"2023","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1186\/s13638-018-1104-7","article-title":"Rate-splitting multiple access for downlink communication systems: Bridging, generalizing, and outperforming SDMA and NOMA","volume":"2018","author":"Mao","year":"2018","journal-title":"EURASIP J. Wirel. Commun. Netw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"8754","DOI":"10.1109\/TCOMM.2019.2943168","article-title":"Rate-Splitting for Multi-Antenna Non-Orthogonal Unicast and Multicast Transmission: Spectral and Energy Efficiency Analysis","volume":"67","author":"Mao","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/TMC.2012.17","article-title":"Successive Interference Cancellation: Carving Out MAC Layer Opportunities","volume":"12","author":"Sen","year":"2013","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2820","DOI":"10.1109\/LWC.2021.3118441","article-title":"Optimal Power Allocation for Rate Splitting Communications with Deep Reinforcement Learning","volume":"10","author":"Hieu","year":"2021","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"12333","DOI":"10.1109\/TVT.2022.3191085","article-title":"Rate-Splitting Multiple Access with Finite Blocklength for Short-Packet and Low-Latency Downlink Communications","volume":"71","author":"Xu","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.1109\/JSAC.2023.3240779","article-title":"Evolutionary Games for Dynamic Network Resource Selection in RSMA-Enabled 6G Networks","volume":"41","author":"Van","year":"2023","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1555","DOI":"10.1109\/LWC.2019.2927206","article-title":"An Efficient Rate-Splitting Multiple Access Scheme for the Downlink of C-RAN Systems","volume":"8","author":"Yu","year":"2019","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"6816","DOI":"10.1109\/TVT.2022.3229822","article-title":"Max-Min Fairness of Rate-Splitting Multiple Access with Finite Blocklength Communications","volume":"72","author":"Xu","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4816","DOI":"10.1109\/TWC.2021.3133433","article-title":"Rate-Splitting Multiple Access for Multi-Antenna Downlink Communication Systems: Spectral and Energy Efficiency Tradeoff","volume":"21","author":"Zhou","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"52398","DOI":"10.1109\/ACCESS.2022.3173125","article-title":"HAMEC-RSMA: Enhanced Aerial Computing Systems with Rate Splitting Multiple Access","volume":"10","author":"Truong","year":"2022","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H., and Park, L. (2023). HAP-Assisted RSMA-Enabled Vehicular Edge Computing: A DRL-Based Optimization Framework. Mathematics, 11.","DOI":"10.3390\/math11102376"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"16258","DOI":"10.1109\/TVT.2020.3037657","article-title":"On the Downlink Performance of RSMA-Based UAV Communications","volume":"69","author":"Jaafar","year":"2020","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2586","DOI":"10.1109\/JSYST.2022.3220249","article-title":"Ergodic Capacity and Placement Optimization for RSMA-Enabled UAV-Assisted Communication","volume":"17","author":"Singh","year":"2023","journal-title":"IEEE Syst. J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3714","DOI":"10.1109\/TWC.2022.3220785","article-title":"Performance Analysis and Optimization of RSMA Enabled UAV-Aided IBL and FBL Communication with Imperfect SIC and CSI","volume":"22","author":"Singh","year":"2023","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Xiao, M., Cui, H., Huang, D., Zhao, Z., Cao, X., and Wu, D.O. (2023). Traffic-Aware Energy-Efficient Resource Allocation for RSMA Based UAV Communications. IEEE Trans. Netw. Sci. Eng., in press.","DOI":"10.1109\/TNSE.2023.3274550"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ji, J., Cai, L., Zhu, K., and Niyato, D. (2023). Decoupled Association with Rate Splitting Multiple Access in UAV-assisted Cellular Networks Using Multi-agent Deep Reinforcement Learning. IEEE Trans. Mob. Comput., in press.","DOI":"10.1109\/TMC.2023.3256404"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hua, D.T., Do, Q.T., Dao, N.N., and Cho, S. (2023). On sum-rate maximization in downlink UAV-aided RSMA systems. ICT Express, in press.","DOI":"10.1016\/j.icte.2023.03.001"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"S\u00e1nchez, J.A.H., Casilimas, K., and Rendon, O.M.C. (2022). Deep Reinforcement Learning for Resource Management on Network Slicing: A Survey. Sensors, 22.","DOI":"10.3390\/s22083031"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, Z., Pan, W., Li, H., Wang, X., and Zuo, Q. (2022). Review of Deep Reinforcement Learning Approaches for Conflict Resolution in Air Traffic Control. Aerospace, 9.","DOI":"10.3390\/aerospace9060294"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H., and Park, L. (2022, January 19\u201321). A Survey on Deep Reinforcement Learning-driven Task Offloading in Aerial Access Networks. Proceedings of the 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC55196.2022.9952687"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H., Park, H., and Park, L. (2023, January 20\u201323). Recent Studies on Deep Reinforcement Learning in RIS-UAV Communication Networks. Proceedings of the 2023 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Bali, Indonesia.","DOI":"10.1109\/ICAIIC57133.2023.10067052"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H., Park, H., Seol, K., So, S., and Park, L. (2023, January 4\u20137). Applications of Deep Learning and Deep Reinforcement Learning in 6G Networks. Proceedings of the 2023 Fourteenth International Conference on Ubiquitous and Future Networks (ICUFN), Paris, France.","DOI":"10.1109\/ICUFN57995.2023.10200390"},{"key":"ref_33","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016, January 2\u20134). Continuous control with deep reinforcement learning. Proceedings of the 4th International Conference on Learning Representations (ICLR 2016), San Juan, Puerto Rico."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"4605","DOI":"10.1109\/JIOT.2022.3218740","article-title":"FlyReflect: Joint Flying IRS Trajectory and Phase Shift Design Using Deep Reinforcement Learning","volume":"10","author":"Truong","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"17603","DOI":"10.1109\/JIOT.2023.3279196","article-title":"Learning-based Reconfigurable Intelligent Surface-aided Rate-Splitting Multiple Access Networks","volume":"10","author":"Hua","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_36","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing function approximation error in actor-critic methods. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Nguyen, T.H., Truong, T.P., Dao, N.N., Na, W., Park, H., and Park, L. (2022, January 19\u201321). Deep Reinforcement Learning-based Partial Task Offloading in High Altitude Platform-aided Vehicular Networks. Proceedings of the 2022 13th International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC55196.2022.9952890"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2856","DOI":"10.1109\/TWC.2005.858335","article-title":"Three-dimensional spatial fading correlation models for compact MIMO receivers","volume":"4","author":"Yong","year":"2005","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_39","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http:\/\/www.deeplearningbook.org."}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/22\/5284\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:19:23Z","timestamp":1760131163000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/15\/22\/5284"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,8]]},"references-count":39,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2023,11]]}},"alternative-id":["rs15225284"],"URL":"https:\/\/doi.org\/10.3390\/rs15225284","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,8]]}}}