{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T11:48:45Z","timestamp":1778932125527,"version":"3.51.4"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Soft Science Research Project of Henan Province","award":["222400410137"],"award-info":[{"award-number":["222400410137"]}]},{"name":"Key R &D projects in the autonomous region","award":["2020B02018-2"],"award-info":[{"award-number":["2020B02018-2"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Integrated satellite\u2013aerial\u2013terrestrial relay networks (ISATRNs) play a vital role in next-gen networks, particularly those with high-altitude platforms (HAP). This study introduces a new model for hybrid optical\/RF-based HAP-enabled ISATRNs, incorporating reconfigurable intelligent surfaces (RIS) on unmanned aerial vehicles (UAVs) to optimize access in dense urban areas. Non-orthogonal multiple access is employed for improved spectrum efficiency. The objective is to jointly optimize UAV trajectory, RIS phase shift, and active transmit beamforming while considering energy consumption. A deep reinforcement learning approach using LSTM-DDQN framework is proposed. Numerical results show the effectiveness of our algorithm over traditional DDQN, with higher single-step exploration reward and evaluation metrics.<\/jats:p>","DOI":"10.1186\/s13634-023-01070-7","type":"journal-article","created":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T16:04:18Z","timestamp":1701101058000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Energy efficiency performance in RIS-based integrated satellite\u2013aerial\u2013terrestrial relay networks with deep reinforcement learning"],"prefix":"10.1186","volume":"2023","author":[{"given":"Jiao","family":"Li","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huajian","family":"Xue","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9379-7714","authenticated-orcid":false,"given":"Min","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fucheng","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tieliang","family":"Gao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,11,27]]},"reference":[{"issue":"2","key":"1070_CR1","doi-asserted-by":"publisher","first-page":"1505","DOI":"10.1109\/OJCOMS.2021.3093110","volume":"2","author":"N Saeed","year":"2021","unstructured":"N. Saeed, H. Almorad, H. Dahrouj, T.Y. Al-Naffouri, J.S. Shamma, M.-S. Alouini, Point-to-point communication in integrated satellite-aerial 6G networks: state-of-the-art and future challenges. IEEE Open J. Commun. Soc. 2(2), 1505\u20131525 (2021)","journal-title":"IEEE Open J. Commun. Soc."},{"issue":"1","key":"1070_CR2","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1109\/JIOT.2021.3126825","volume":"9","author":"X Zhu","year":"2022","unstructured":"X. Zhu, C. Jiang, Integrated satellite\u2013terrestrial networks toward 6G: architectures, applications, and challenges. IEEE Internet Things J. 9(1), 437\u2013461 (2022)","journal-title":"IEEE Internet Things J."},{"issue":"11","key":"1070_CR3","doi-asserted-by":"publisher","first-page":"3025","DOI":"10.1109\/JSAC.2016.2615261","volume":"34","author":"K An","year":"2016","unstructured":"K. An, M. Lin, J. Ouyang, W.P. Zhu, Secure transmission in cognitive satellite terrestrial networks. IEEE J. Sel. Areas Commun. 34(11), 3025\u20133037 (2016)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"14","key":"1070_CR4","doi-asserted-by":"publisher","first-page":"11123","DOI":"10.1109\/JIOT.2021.3051603","volume":"8","author":"Z Lin","year":"2021","unstructured":"Z. Lin, M. Lin, T. de Cola, J.-B. Wang, W.-P. Zhu, J. Cheng, Supporting IoT with rate-splitting multiple access in satellite and aerial-integrated networks. IEEE Internet Things J. 8(14), 11123\u201311134 (2021)","journal-title":"IEEE Internet Things J."},{"issue":"2","key":"1070_CR5","doi-asserted-by":"publisher","first-page":"1676","DOI":"10.1109\/TIV.2022.3225466","volume":"8","author":"F Zhou","year":"2023","unstructured":"F. Zhou, X. Li, M. Alazab, R.H. Jhaveri, K. Guo, Secrecy performance for RIS-based integrated satellite vehicle networks with a UAV relay and MRC eavesdropping. IEEE Trans. Intell. Veh. 8(2), 1676\u20131685 (2023)","journal-title":"IEEE Trans. Intell. Veh."},{"issue":"8","key":"1070_CR6","doi-asserted-by":"publisher","first-page":"7009","DOI":"10.1109\/JIOT.2020.3037408","volume":"8","author":"X Zhang","year":"2021","unstructured":"X. Zhang, D. Guo, K. An, G. Zheng, S. Chatzinotas, B. Zhang, Auction-based multichannel cooperative spectrum sharing in hybrid satellite\u2013terrestrial IoT networks. IEEE Internet Things J. 8(8), 7009\u20137023 (2021)","journal-title":"IEEE Internet Things J."},{"issue":"4","key":"1070_CR7","doi-asserted-by":"publisher","first-page":"2231","DOI":"10.1109\/COMST.2014.2329501","volume":"16","author":"MA Khalighi","year":"2014","unstructured":"M.A. Khalighi, M. Uysal, Survey on free space optical communication: a communication theory perspective. IEEE Commun. Surv. Tutor. 16(4), 2231\u20132258 (2014)","journal-title":"IEEE Commun. Surv. Tutor."},{"issue":"7","key":"1070_CR8","doi-asserted-by":"publisher","first-page":"8287","DOI":"10.1109\/TVT.2023.3252822","volume":"72","author":"G Xu","year":"2023","unstructured":"G. Xu, N. Zhang, M. Xu, Z. Xu, Q. Zhang, Z. Song, Outage probability and average BER of UAV-assisted dual-hop FSO communication with amplify-and-forward relaying. IEEE Trans. Veh. Technol. 72(7), 8287\u20138302 (2023)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"3","key":"1070_CR9","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1109\/MWC.001.2000343","volume":"28","author":"G Pan","year":"2021","unstructured":"G. Pan, J. Ye, J. An, M.-S. Alouini, Full-duplex enabled intelligent reflecting surface systems: opportunities and challenges. IEEE Wirel. Commun. 28(3), 122\u2013129 (2021)","journal-title":"IEEE Wirel. Commun."},{"issue":"3","key":"1070_CR10","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1109\/TWC.2021.3106346","volume":"21","author":"L Lv","year":"2022","unstructured":"L. Lv, Q. Wu, Z. Li, Z. Ding, N. Al-Dhahir, J. Chen, Covert communication in intelligent reflecting surface-assisted NOMA systems: design, analysis, and optimization. IEEE Trans. Wirel. Commun. 21(3), 1735\u20131750 (2022)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"6","key":"1070_CR11","doi-asserted-by":"publisher","first-page":"6843","DOI":"10.1109\/TVT.2022.3162197","volume":"71","author":"T Wang","year":"2022","unstructured":"T. Wang, F. Fang, Z. Ding, An SCA and relaxation based energy efficiency optimization for multi-user RIS-assisted NOMA networks. IEEE Trans. Veh. Technol. 71(6), 6843\u20136847 (2022)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"3","key":"1070_CR12","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1109\/JSTSP.2019.2899731","volume":"13","author":"Z Lin","year":"2019","unstructured":"Z. Lin, M. Lin, J.-B. Wang, T. de Cola, J. Wang, Joint beamforming and power allocation for satellite\u2013terrestrial integrated networks with non-orthogonal multiple access. IEEE J. Sel. Top. Signal Process. 13(3), 657\u2013670 (2019)","journal-title":"IEEE J. Sel. Top. Signal Process."},{"issue":"20","key":"1070_CR13","doi-asserted-by":"publisher","first-page":"15041","DOI":"10.1109\/JIOT.2020.3004432","volume":"8","author":"Z Na","year":"2021","unstructured":"Z. Na, Y. Liu, J. Shi, C. Liu, Z. Gao, UAV-supported clustered NOMA for 6G-enabled Internet of Things: trajectory planning and resource allocation. IEEE Internet Things J. 8(20), 15041\u201315048 (2021)","journal-title":"IEEE Internet Things J."},{"issue":"9","key":"1070_CR14","doi-asserted-by":"publisher","first-page":"6310","DOI":"10.1109\/TCOMM.2022.3188020","volume":"70","author":"S Yang","year":"2022","unstructured":"S. Yang, J. Zhang, W. Xia, Y. Ren, H. Yin, H. Zhu, A unified framework for distributed RIS-aided downlink systems between MIMO-NOMA and MIMO-SDMA. IEEE Trans. Commun. 70(9), 6310\u20136324 (2022)","journal-title":"IEEE Trans. Commun."},{"issue":"5","key":"1070_CR15","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1109\/MWC.010.2000528","volume":"28","author":"X Pang","year":"2021","unstructured":"X. Pang et al., When UAV meets IRS: expanding air-ground networks via passive reflection. IEEE Wirel. Commun. 28(5), 164\u2013170 (2021)","journal-title":"IEEE Wirel. Commun."},{"issue":"4","key":"1070_CR16","doi-asserted-by":"publisher","first-page":"1090","DOI":"10.1109\/LWC.2019.2907852","volume":"8","author":"L Lv","year":"2019","unstructured":"L. Lv, Z. Ding, J. Chen, N. Al-Dhahir, Design of secure NOMA against full-duplex proactive eavesdropping. IEEE Wirel. Commun. Lett. 8(4), 1090\u20131094 (2019)","journal-title":"IEEE Wirel. Commun. Lett."},{"issue":"9","key":"1070_CR17","doi-asserted-by":"publisher","first-page":"10197","DOI":"10.1109\/TITS.2023.3267607","volume":"24","author":"K Guo","year":"2023","unstructured":"K. Guo, M. Wu, X. Li, H. Song, N. Kumar, Deep reinforcement learning and NOMA-based multi-objective RIS-assisted IS-UAV-TNs: trajectory optimization and beamforming design. IEEE Trans. Intell. Transp. Syst. 24(9), 10197\u201310210 (2023)","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"issue":"4","key":"1070_CR18","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1109\/LWC.2023.3238073","volume":"12","author":"K Li","year":"2023","unstructured":"K. Li, C. Huang, Y. Gong, G. Chen, Double deep learning for joint phase-shift and beamforming based on cascaded channels in RIS-assisted MIMO networks. IEEE Wirel. Commun. Lett. 12(4), 659\u2013663 (2023)","journal-title":"IEEE Wirel. Commun. Lett."},{"issue":"4","key":"1070_CR19","doi-asserted-by":"publisher","first-page":"259","DOI":"10.23919\/JCC.fa.2022-0421.202304","volume":"20","author":"P Chen","year":"2023","unstructured":"P. Chen, W. Huang, X. Li, S. Jin, Deep reinforcement learning based power minimization for RIS-assisted MISO-OFDM systems. China Commun. 20(4), 259\u2013269 (2023)","journal-title":"China Commun."},{"issue":"21","key":"1070_CR20","doi-asserted-by":"publisher","first-page":"21121","DOI":"10.1109\/JIOT.2022.3177705","volume":"9","author":"Z Peng","year":"2023","unstructured":"Z. Peng, Z. Zhang, L. Kong, C. Pan, L. Li, J. Wang, Deep reinforcement learning for RIS-aided multiuser full-duplex secure communications with hardware impairments. IEEE Internet Things J. 9(21), 21121\u201321135 (2023)","journal-title":"IEEE Internet Things J."},{"issue":"4","key":"1070_CR21","doi-asserted-by":"publisher","first-page":"2288","DOI":"10.1109\/TCOMM.2023.3245659","volume":"71","author":"X Li","year":"2023","unstructured":"X. Li et al., Physical-layer authentication for ambient backscatter-aided NOMA symbiotic systems. IEEE Trans. Commun. 71(4), 2288\u20132303 (2023)","journal-title":"IEEE Trans. Commun."},{"issue":"2","key":"1070_CR22","doi-asserted-by":"publisher","first-page":"2122","DOI":"10.1109\/TVT.2021.3131568","volume":"71","author":"H Niu","year":"2022","unstructured":"H. Niu, Z. Chu, F. Zhou, P. Xiao, N. Al-Dhahir, Weighted sum rate optimization for STAR-RIS-assisted MIMO system. IEEE Trans. Veh. Technol. 71(2), 2122\u20132127 (2022)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"5","key":"1070_CR23","doi-asserted-by":"publisher","first-page":"5129","DOI":"10.1109\/TVT.2020.2979496","volume":"69","author":"K Guo","year":"2020","unstructured":"K. Guo et al., Physical layer security for multiuser satellite communication systems with threshold-based scheduling scheme. IEEE Trans. Veh. Technol. 69(5), 5129\u20135141 (2020)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"8","key":"1070_CR24","doi-asserted-by":"publisher","first-page":"4157","DOI":"10.1109\/TWC.2019.2922609","volume":"18","author":"C Huang","year":"2019","unstructured":"C. Huang, A. Zappone, G.C. Alexandropoulos, M. Debbah, C. Yuen, Reconfigurable intelligent surfaces for energy efficiency in wireless communication. IEEE Trans. Wirel. Commun. 18(8), 4157\u20134170 (2019)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"3","key":"1070_CR25","doi-asserted-by":"publisher","first-page":"2761","DOI":"10.1109\/TVT.2021.3129075","volume":"71","author":"C Gong","year":"2022","unstructured":"C. Gong, X. Yue, X. Wang, X. Dai, R. Zou, M. Essaaidi, Intelligent reflecting surface aided secure communications for NOMA networks. IEEE Trans. Veh. Technol. 71(3), 2761\u20132773 (2022)","journal-title":"IEEE Trans. Veh. Technol."},{"issue":"8","key":"1070_CR26","doi-asserted-by":"publisher","first-page":"1554","DOI":"10.1117\/1.1386641","volume":"40","author":"MA AI-Habash","year":"2001","unstructured":"M.A. AI-Habash, Mathematical model for the irradiance probability density function of a laser beam propagating through turbulent media. Opt. Eng. 40(8), 1554\u20131562 (2001)","journal-title":"Opt. Eng."},{"issue":"3","key":"1070_CR27","doi-asserted-by":"publisher","first-page":"1618","DOI":"10.1109\/TIT.2022.3220261","volume":"69","author":"J Gao","year":"2023","unstructured":"J. Gao, Y. Wu, S. Shao, W. Yang, H.V. Poor, Energy efficiency of massive random access in MIMO quasi-static Rayleigh fading channels with finite blocklength. IEEE Trans. Inf. Theory 69(3), 1618\u20131657 (2023)","journal-title":"IEEE Trans. Inf. Theory"},{"issue":"12","key":"1070_CR28","doi-asserted-by":"publisher","first-page":"9814","DOI":"10.1109\/JIOT.2020.3021255","volume":"8","author":"Z Jia","year":"2021","unstructured":"Z. Jia, M. Sheng, J. Li, D. Niyato, Z. Han, LEO-satellite-assisted UAV: joint trajectory and data collection for internet of remote things in 6G aerial access networks. IEEE Internet Things J. 8(12), 9814\u20139826 (2021)","journal-title":"IEEE Internet Things J."},{"issue":"7","key":"1070_CR29","doi-asserted-by":"publisher","first-page":"5057","DOI":"10.1109\/TWC.2020.2989302","volume":"19","author":"H-T Ye","year":"2020","unstructured":"H.-T. Ye, X. Kang, J. Joung, Y.-C. Liang, Optimization for full-duplex rotary-wing UAV enabled wireless-powered IoT networks. IEEE Trans. Wirel. Commun. 19(7), 5057\u20135072 (2020)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"1070_CR30","doi-asserted-by":"publisher","first-page":"2329","DOI":"10.1109\/TWC.2019.2902559","volume":"18","author":"Y Zeng","year":"2013","unstructured":"Y. Zeng, J. Xu, R. Zhang, Energy minimization for wireless communication with rotary-wing UAV. IEEE Trans. Wirel. Commun. 18(4), 2329\u20132345 (2013)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"1070_CR31","doi-asserted-by":"publisher","first-page":"2583","DOI":"10.1109\/TWC.2022.3212830","volume":"22","author":"H Zhang","year":"2023","unstructured":"H. Zhang, M. Huang, H. Zhou, X. Wang, N. Wang, K. Long, Capacity maximization in RIS-UAV networks: a DDQN-based trajectory and phase shift optimization approach. IEEE Trans. Wirel. Commun. 22(4), 2583\u20132591 (2023)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"2","key":"1070_CR32","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1109\/TCOMM.2021.3136563","volume":"70","author":"X Pang","year":"2022","unstructured":"X. Pang, N. Zhao, J. Tang, C. Wu, D. Niyato, K.-K. Wong, IRS-assisted secure UAV transmission via joint trajectory and beamforming design. IEEE Trans. Commun. 70(2), 1140\u20131152 (2022)","journal-title":"IEEE Trans. Commun."},{"issue":"9","key":"1070_CR33","doi-asserted-by":"publisher","first-page":"6949","DOI":"10.1109\/TWC.2022.3153316","volume":"21","author":"N Zhao","year":"2022","unstructured":"N. Zhao, Z. Ye, Y. Pei, Y.-C. Liang, D. Niyato, Multi-agent deep reinforcement learning for task offloading in UAV-assisted mobile edge computing. IEEE Trans. Wirel. Commun. 21(9), 6949\u20136960 (2022)","journal-title":"IEEE Trans. Wirel. Commun."},{"issue":"4","key":"1070_CR34","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1109\/JSAC.2020.3018823","volume":"39","author":"X Liu","year":"2021","unstructured":"X. Liu, Y. Liu, Y. Chen, H.V. Poor, RIS enhanced massive non-orthogonal multiple access networks: deployment and passive beamforming design. IEEE J. Sel. Areas Commun. 39(4), 1057\u20131071 (2021)","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"6","key":"1070_CR35","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1109\/MWC.001.2100116","volume":"28","author":"J Xu","year":"2021","unstructured":"J. Xu, B. Ai, Deep reinforcement learning for handover-aware MPTCP congestion control in space-ground integrated network of railways. IEEE Wirel. Commun. 28(6), 200\u2013207 (2021)","journal-title":"IEEE Wirel. Commun."}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-023-01070-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-023-01070-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-023-01070-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T16:18:34Z","timestamp":1701101914000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-023-01070-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,27]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["1070"],"URL":"https:\/\/doi.org\/10.1186\/s13634-023-01070-7","relation":{},"ISSN":["1687-6180"],"issn-type":[{"value":"1687-6180","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,27]]},"assertion":[{"value":"11 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 October 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 November 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"121"}}