{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T09:21:03Z","timestamp":1771924863027,"version":"3.50.1"},"reference-count":42,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,1,26]],"date-time":"2026-01-26T00:00:00Z","timestamp":1769385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Commun. Netw."],"abstract":"<jats:sec>\n                    <jats:title>Introduction<\/jats:title>\n                    <jats:p>Energy efficiency is a critical challenge in Beyond-5G (B5G) cellular networks, where ground base stations (GBSs) are responsible for a substantial portion of network energy consumption. Reducing this consumption while maintaining minimum user data rate requirements remains a key research problem.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This paper proposes an Aerial Base Station (ABS)-assisted energy optimization framework that integrates ABS deployment with low-power sleep states of GBSs. Traffic is selectively offloaded from lightly loaded GBSs to ABSs, enabling energy savings without violating user quality-of-service constraints. A Deep Deterministic Policy Gradient (DDPG) algorithm is employed to jointly optimize ABS positioning, GBS sleep mode scheduling, and resource allocation under dynamic traffic conditions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Simulation results demonstrate that the proposed DDPG-based framework significantly reduces network energy consumption while improving achievable user data rates compared to baseline schemes without ABS assistance or learning-based optimization.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The results highlight the effectiveness of integrating ABSs with GBS low-power sleep states using reinforcement learning. By enforcing minimum data rate constraints and dynamically adapting to traffic variations, the proposed approach provides a scalable and energy-efficient solution for sustainable operation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frcmn.2025.1764320","type":"journal-article","created":{"date-parts":[[2026,1,26]],"date-time":"2026-01-26T07:19:31Z","timestamp":1769411971000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["DDPG-based energy efficiency optimization for ABS-assisted beyond-5G cellular networks with sleep mode management"],"prefix":"10.3389","volume":"6","author":[{"given":"Vala","family":"Saleh","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Shiraz University of Technology","place":["Shiraz, Iran"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohsen","family":"Eslami","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Shiraz University of Technology","place":["Shiraz, Iran"]},{"name":"Electrical and Computer Engineering Department, University of Alberta","place":["Edmonton, AB, Canada"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kamran","family":"Kazemi","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Shiraz University of Technology","place":["Shiraz, Iran"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,1,26]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"214","DOI":"10.3390\/drones7030214","article-title":"A survey on energy optimization techniques in UAV-based cellular networks: from conventional to machine learning approaches","volume":"7","author":"Abubakar","year":"2023","journal-title":"Drones"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e3254","DOI":"10.1002\/ett.3254","article-title":"How to make key 5G wireless technologies environmental friendly: a review","volume":"29","author":"Alsharif","year":"2018","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"4310","DOI":"10.1109\/tnsm.2022.3157650","article-title":"Energy optimization with multi-sleeping control in 5G heterogeneous networks using reinforcement learning","volume":"19","author":"Amine","year":"2022","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"B4","doi-asserted-by":"publisher","first-page":"39","DOI":"10.3390\/drones6020039","article-title":"Drones in B5G\/6G networks as flying base stations","volume":"6","author":"Amponis","year":"2022","journal-title":"Drones"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e4916","DOI":"10.1002\/ett.4916","article-title":"Multi-UAV path planning utilizing the PGA algorithm for terrestrial IoT sensor network under ISAC framework","volume":"35","author":"Azadur","year":"2024","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"B6","doi-asserted-by":"publisher","first-page":"e70002","DOI":"10.1002\/ett.70002","article-title":"Placement and power assignment for hierarchical UAV networks under hovering fluctuations in mmWave communications","volume":"35","author":"Azarhava","year":"2024","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"B7","doi-asserted-by":"publisher","first-page":"7786","DOI":"10.1109\/access.2023.3349208","article-title":"A comprehensive survey on 5G-and-Beyond networks with UAVs: applications, emerging technologies, regulatory aspects, research trends and challenges","volume":"12","author":"Banafaa","year":"2024","journal-title":"IEEE Access"},{"key":"B8","doi-asserted-by":"publisher","first-page":"e4464","DOI":"10.1002\/ett.4464","article-title":"Resource optimization in UAV-Assisted wireless networks\u2014A comprehensive survey","volume":"33","author":"Basharat","year":"2022","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"B9","doi-asserted-by":"publisher","first-page":"1529453","DOI":"10.3389\/frcmn.2025.1529453","article-title":"UAV-assisted federated learning with hybrid LoRa P2P\/LoRaWAN for sustainable biosphere","volume":"6","author":"Behjati","year":"2025","journal-title":"Front. Commun. Netw."},{"key":"B10","doi-asserted-by":"publisher","first-page":"2809","DOI":"10.1109\/lcomm.2020.3015462","article-title":"User scheduling based on multi-agent deep Q-learning for robust beamforming in multicell MISO systems","volume":"24","author":"Braga","year":"2020","journal-title":"IEEE Commun. Lett."},{"key":"B11","doi-asserted-by":"publisher","first-page":"124505","DOI":"10.1109\/access.2021.3111087","article-title":"Joint resource allocation and UAV scheduling with ground radio station sleeping","volume":"9","author":"Chowdary","year":"2021","journal-title":"IEEE Access"},{"key":"B12","doi-asserted-by":"publisher","first-page":"67512","DOI":"10.1109\/access.2020.3031234","article-title":"A survey on beyond 5G network with the advent of 6G: architecture and emerging technologies","volume":"9","author":"Dogra","year":"2020","journal-title":"IEEE Access"},{"key":"B13","doi-asserted-by":"publisher","first-page":"101564","DOI":"10.1016\/j.phycom.2021.101564","article-title":"A survey on UAV placement optimization for UAV-Assisted communication in 5G and beyond networks","volume":"51","author":"Elnabty","year":"2022","journal-title":"Phys. Commun."},{"key":"B14","doi-asserted-by":"publisher","first-page":"60836","DOI":"10.1109\/access.2022.3179720","article-title":"Double deep Q-Learning with prioritized experience replay for anomaly detection in smart environments","volume":"10","author":"F\u00e4hrmann","year":"2022","journal-title":"IEEE Access"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1304","DOI":"10.1109\/comst.2022.3171135","article-title":"What will the future of UAV cellular communications be? A flight from 5G to 6G","volume":"24","author":"Geraci","year":"2022","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"B16","doi-asserted-by":"publisher","first-page":"2165","DOI":"10.1109\/tvt.2019.2893374","article-title":"Joint position and travel path optimization for energy efficient wireless data gathering using unmanned aerial vehicles","volume":"68","author":"Ghorbel","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"B17","doi-asserted-by":"publisher","first-page":"1286073","DOI":"10.3389\/frcmn.2024.1286073","article-title":"A systematic literature review on the role of UAV-enabled communications in advancing the UN\u2019s sustainable development goals","volume":"5","author":"Gryech","year":"2024","journal-title":"Front. Commun. Netw."},{"key":"B18","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1016\/j.comcom.2023.05.013","article-title":"A survey on UAV-assisted wireless communications: recent advances and future trends","volume":"208","author":"Gu","year":"2023","journal-title":"Comput. Commun."},{"key":"B19","doi-asserted-by":"publisher","first-page":"15","DOI":"10.23919\/jcin.2019.8917869","article-title":"An overview of intelligent wireless communications using deep reinforcement learning","volume":"4","author":"Huang","year":"2019","journal-title":"J. Commun. Inf. Netw."},{"key":"B20","doi-asserted-by":"publisher","first-page":"2898","DOI":"10.1109\/taes.2022.3220493","article-title":"Energy constrained sum-rate maximization in IRS-assisted UAV networks with imperfect channel information","volume":"59","author":"Jangsher","year":"2022","journal-title":"IEEE Trans. Aerosp. Electron. Syst."},{"key":"B21","doi-asserted-by":"publisher","first-page":"6539","DOI":"10.1109\/twc.2022.3150425","article-title":"Energy-efficient ultra-dense network with deep reinforcement learning","volume":"21","author":"Ju","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"B22","doi-asserted-by":"publisher","first-page":"2361","DOI":"10.1109\/comst.2019.2915069","article-title":"A survey of air-to-ground propagation channel modeling for unmanned aerial vehicles","volume":"21","author":"Khawaja","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"B23","doi-asserted-by":"publisher","first-page":"3786","DOI":"10.1109\/tvt.2015.2445922","article-title":"Base-station sleep management in open-access femtocell networks","volume":"65","author":"Kim","year":"2015","journal-title":"IEEE Trans. Veh. Technol."},{"key":"B24","doi-asserted-by":"publisher","first-page":"9070","DOI":"10.3390\/s23229070","article-title":"An energy-efficient multi-level sleep strategy for periodic uplink transmission in industrial private 5G networks","volume":"23","author":"Kim","year":"2023","journal-title":"Sensors"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1432","DOI":"10.1109\/access.2022.3233430","article-title":"Energy-efficient sleep mode schemes for cell-less RAN in 5G and beyond 5G networks","volume":"11","author":"Kooshki","year":"2023","journal-title":"IEEE Access"},{"key":"B26","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1109\/comst.2022.3142532","article-title":"A survey on 5G radio access network energy efficiency: massive MIMO, lean carrier design, sleep modes, and machine learning","volume":"24","author":"L\u00f3pez-P\u00e9rez","year":"2022","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"B27","doi-asserted-by":"publisher","first-page":"102596","DOI":"10.1016\/j.adhoc.2021.102596","article-title":"Resource management in UAV-assisted wireless networks: an optimization perspective","volume":"121","author":"Masroor","year":"2021","journal-title":"Ad Hoc Netw."},{"key":"B28","doi-asserted-by":"publisher","first-page":"137184","DOI":"10.1109\/access.2019.2942390","article-title":"Machine learning for 5G\/B5G Mobile and wireless communications: potential, limitations, and future directions","volume":"7","author":"Morocho-Cayamcela","year":"2019","journal-title":"IEEE Access"},{"key":"B29","doi-asserted-by":"publisher","first-page":"7709","DOI":"10.3390\/s23187709","article-title":"Emerging technologies for 6G communication networks: machine learning approaches","volume":"23","author":"Puspitasari","year":"2023","journal-title":"Sensors"},{"key":"B30","doi-asserted-by":"publisher","first-page":"103440","DOI":"10.1016\/j.adhoc.2024.103440","article-title":"Non-terrestrial UAV clients for beyond 5G networks: a comprehensive survey","volume":"157","author":"Qazzaz","year":"2024","journal-title":"Ad Hoc Netw."},{"key":"B31","doi-asserted-by":"publisher","first-page":"322","DOI":"10.3390\/drones7050322","article-title":"Artificial intelligence-based autonomous UAV networks: a survey","volume":"7","author":"Sarkar","year":"2023","journal-title":"Drones"},{"key":"B32","doi-asserted-by":"publisher","first-page":"103114","DOI":"10.1016\/j.jnca.2021.103114","article-title":"UAV assisted 5G and beyond wireless networks: a survey","volume":"189","author":"Shahzadi","year":"2021","journal-title":"J. Netw. Comput. Appl."},{"key":"B33","doi-asserted-by":"publisher","first-page":"5097","DOI":"10.1109\/tmc.2023.3301506","article-title":"Double deep Q-Learning-Based path selection and service placement for latency-sensitive beyond 5G applications","volume":"23","author":"Shokrnezhad","year":"2024","journal-title":"IEEE Trans. Mob. Comput."},{"key":"B34","doi-asserted-by":"publisher","first-page":"37","DOI":"10.5120\/9594-4216","article-title":"Comparison of okumura, hata and COST-231 models on the basis of path loss and signal strength","volume":"59","author":"Singh","year":"2012","journal-title":"Int. J. Comput. Appl."},{"key":"B35","doi-asserted-by":"publisher","first-page":"2200","DOI":"10.3390\/electronics12102200","article-title":"From 5G to beyond 5G: a comprehensive survey of wireless network evolution, challenges, and promising technologies","volume":"12","author":"Sufyan","year":"2023","journal-title":"Electronics"},{"key":"B36","doi-asserted-by":"publisher","first-page":"3840","DOI":"10.3390\/math10203840","article-title":"Joint resource and trajectory optimization for energy efficiency maximization in UAV-based networks","volume":"10","author":"Tung","year":"2022","journal-title":"Mathematics"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v30i1.10295","article-title":"Deep reinforcement learning with double Q-learning","volume-title":"Proc. AAAI Conf. Artif. Intell","author":"Van Hasselt","year":"2016"},{"key":"B38","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1016\/j.icte.2022.01.015","article-title":"A survey on UAV placement and trajectory optimization in communication networks: from the perspective of air-to-ground channel models","volume":"9","author":"Won","year":"2023","journal-title":"ICT Express"},{"key":"B39","doi-asserted-by":"publisher","first-page":"10991","DOI":"10.1109\/tvt.2022.3184869","article-title":"Energy-constrained UAV flight scheduling for IoT data collection with 60 GHz communication","volume":"71","author":"Wu","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"B40","doi-asserted-by":"publisher","first-page":"6361","DOI":"10.1109\/tcomm.2021.3089476","article-title":"Multi-objective optimization for UAV-assisted wireless powered IoT networks based on extended DDPG algorithm","volume":"69","author":"Yu","year":"2021","journal-title":"IEEE Trans. Commun."},{"key":"B41","doi-asserted-by":"publisher","first-page":"2645","DOI":"10.1109\/lwc.2022.3213176","article-title":"Radio resource management for C-V2X using graph matching and actor\u2013critic learning","volume":"11","author":"Zhou","year":"2022","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"B42","doi-asserted-by":"publisher","first-page":"9540","DOI":"10.1109\/tvt.2021.3102161","article-title":"UAV trajectory planning in wireless sensor networks for energy consumption minimization by deep reinforcement learning","volume":"70","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."}],"container-title":["Frontiers in Communications and Networks"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frcmn.2025.1764320\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T08:31:17Z","timestamp":1771921877000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frcmn.2025.1764320\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,26]]},"references-count":42,"alternative-id":["10.3389\/frcmn.2025.1764320"],"URL":"https:\/\/doi.org\/10.3389\/frcmn.2025.1764320","relation":{},"ISSN":["2673-530X"],"issn-type":[{"value":"2673-530X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,26]]},"article-number":"1764320"}}