{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T07:52:22Z","timestamp":1774338742420,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T00:00:00Z","timestamp":1720656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS KAKENHI","award":["24K14913"],"award-info":[{"award-number":["24K14913"]}]},{"name":"Support Center for Advanced Telecommunications Technology Research, Japan","award":["24K14913"],"award-info":[{"award-number":["24K14913"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>For a communication control system in a disaster area where drones (also called unmanned aerial vehicles (UAVs)) are used as aerial base stations (ABSs), the reliability of communication is a key challenge for drones to provide emergency communication services. However, the effective configuration of UAVs remains a major challenge due to limitations in their communication range and energy capacity. In addition, the relatively high cost of drones and the issue of mutual communication interference make it impractical to deploy an unlimited number of drones in a given area. To maximize the communication services provided by a limited number of drones to the ground user equipment (UE) within a certain time frame while minimizing the drone energy consumption, we propose a multi-agent proximal policy optimization (MAPPO) algorithm. Considering the dynamic nature of the environment, we analyze diverse observation data structures and design novel objective functions to enhance the drone performance. We find that, when drone energy consumption is used as a penalty term in the objective function, the drones\u2014acting as agents\u2014can identify the optimal trajectory that maximizes the UE coverage while minimizing the energy consumption. At the same time, the experimental results reveal that, without considering the machine computing power required for training and convergence time, the proposed key algorithm demonstrates better performance in communication coverage and energy saving as compared with other methods. The average coverage performance is 10\u201345% higher than that of the other three methods, and it can save up to 3% more energy.<\/jats:p>","DOI":"10.3390\/fi16070245","type":"journal-article","created":{"date-parts":[[2024,7,11]],"date-time":"2024-07-11T11:30:55Z","timestamp":1720697455000},"page":"245","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Optimizing Drone Energy Use for Emergency Communications in Disasters via Deep Reinforcement Learning"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4503-8331","authenticated-orcid":false,"given":"Wen","family":"Qiu","sequence":"first","affiliation":[{"name":"Information Processing Center, Kitami Institute of Technology, Kitami 090-8507, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2728-5619","authenticated-orcid":false,"given":"Xun","family":"Shao","sequence":"additional","affiliation":[{"name":"Department of Electrical and Electronic Information Engineering, Toyohashi University of Technology, Toyohashi 441-8580, Japan"}]},{"given":"Hiroshi","family":"Masui","sequence":"additional","affiliation":[{"name":"Information Processing Center, Kitami Institute of Technology, Kitami 090-8507, Japan"}]},{"given":"William","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Information Technology and Software Engineering, School of Engineering, Computer and Mathematical Sciences, Unitec Institute of Technology, Auckland 1025, New Zealand"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1016\/j.comcom.2023.05.013","article-title":"A survey on UAV-assisted wireless communications: Recent advances and future trends","volume":"208","author":"Gu","year":"2023","journal-title":"Comput. Commun."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Frattolillo, F., Brunori, D., and Locchi, L. (2023). Scalable and cooperative deep reinforcement learning approaches for multi-UAV systems: A systematic review. Drones, 7.","DOI":"10.3390\/drones7040236"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3038","DOI":"10.1109\/COMST.2023.3323344","article-title":"Towards autonomous multi-UAV wireless network: A survey of reinforcement learning-based approaches","volume":"25","author":"Bai","year":"2023","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Chittoor, P.K., and Bharatiraja, C. (2021, January 17\u201319). Solar Integrated Wireless Drone Charging System for Smart City Applications. Proceedings of the 2021 IEEE 6th International Conference on Computing, Communication and Automation (ICCCA), Arad, Romania.","DOI":"10.1109\/ICCCA52192.2021.9666263"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"872","DOI":"10.1109\/TCCN.2020.2968311","article-title":"Cognition in UAV-aided 5G and beyond communications: A survey","volume":"6","author":"Ullah","year":"2020","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"10983","DOI":"10.1109\/JSEN.2023.3260168","article-title":"Managing sets of flying base stations using energy efficient 3D trajectory planning in cellular networks","volume":"23","author":"Sobouti","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3596444","article-title":"Deep reinforcement learning verification: A survey","volume":"55","author":"Landers","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5719","DOI":"10.1109\/TITS.2023.3248841","article-title":"Communication and control in collaborative UAVs: Recent advances and future trends","volume":"24","author":"Javaid","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"109468","DOI":"10.1016\/j.comnet.2022.109468","article-title":"Connectivity and collision constrained opportunistic routing for emergency communication using UAV","volume":"220","author":"Sharvari","year":"2023","journal-title":"Comput. Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"192303","DOI":"10.1007\/s11432-022-3667-3","article-title":"Joint task scheduling and multi-UAV deployment for aerial computing in emergency communication networks","volume":"66","author":"Zhang","year":"2023","journal-title":"Sci. China Inf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Na, Y., Li, Y., Chen, D., Yao, Y., Li, T., Liu, H., and Wang, K. (2023). Optimal energy consumption path planning for unmanned aerial vehicles based on improved particle swarm optimization. Sustainability, 15.","DOI":"10.3390\/su151612101"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2364","DOI":"10.1109\/TCOMM.2023.3240697","article-title":"Joint power and 3D trajectory optimization for UAV-enabled wireless powered communication networks with obstacles","volume":"71","author":"Pan","year":"2023","journal-title":"IEEE Trans. Commun."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"119243","DOI":"10.1016\/j.eswa.2022.119243","article-title":"A novel UAV path planning approach: Heuristic crossing search and rescue optimization algorithm","volume":"215","author":"Zhang","year":"2023","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4775","DOI":"10.1109\/JIOT.2023.3300718","article-title":"Robust computation offloading and trajectory optimization for multi-UAV-assisted mec: A multi-agent DRL approach","volume":"11","author":"Li","year":"2023","journal-title":"IEEE Internet Things J."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102200","DOI":"10.1016\/j.phycom.2023.102200","article-title":"Energy efficiency maximization for WPT-enabled UAV-assisted emergency communication with user mobility","volume":"61","author":"Sun","year":"2023","journal-title":"Phys. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ao, T., Zhang, K., Shi, H., Jin, Z., Zhou, Y., and Liu, F. (2023). Energy-efficient multi-UAVs cooperative trajectory optimization for communication coverage: An MADRL approach. Remote Sens., 15.","DOI":"10.3390\/rs15020429"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3296","DOI":"10.1109\/TSG.2022.3224517","article-title":"Sustainable wireless services with UAV swarms tailored to renewable energy sources","volume":"14","author":"Donevski","year":"2022","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"58383","DOI":"10.1109\/ACCESS.2018.2875040","article-title":"Comprehensive energy consumption model for unmanned aerial vehicles, based on empirical studies of battery performance","volume":"6","author":"Abeywickrama","year":"2018","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1109\/LWC.2017.2700840","article-title":"3D placement of an unmanned aerial vehicle base station (UAV-BS) for energy-efficient maximal coverage","volume":"6","author":"Alzenad","year":"2017","journal-title":"IEEE Wirel. Commun. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2361","DOI":"10.1109\/COMST.2019.2915069","article-title":"A survey of air-to-ground propagation channel modeling for unmanned aerial vehicles","volume":"21","author":"Khawaja","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement learning: An introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_22","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_23","unstructured":"Yu, C., Velu, A., Vinitsky, E., Wang, Y., Bayen, A., and Wu, Y. (2021). The surprising effectiveness of PPO in cooperative, multi-agent games. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2835","DOI":"10.1109\/TMC.2020.2991326","article-title":"Leveraging UAVs for coverage in cell-free vehicular networks: A deep reinforcement learning approach","volume":"20","author":"Samir","year":"2021","journal-title":"IEEE Trans. Mob. Comput."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/7\/245\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:13:09Z","timestamp":1760109189000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/16\/7\/245"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,11]]},"references-count":24,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["fi16070245"],"URL":"https:\/\/doi.org\/10.3390\/fi16070245","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,11]]}}}