{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:05:54Z","timestamp":1773929154125,"version":"3.50.1"},"reference-count":22,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T00:00:00Z","timestamp":1738454400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"key technology research project","award":["J2024028"],"award-info":[{"award-number":["J2024028"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Energy scheduling for hybrid unmanned aerial vehicles (UAVs) is of critical importance to their safe and stable operation. However, traditional approaches, predominantly rule-based, often lack the dynamic adaptability and stability necessary to address the complexities of changing operational environments. To overcome these limitations, this paper proposes a novel energy scheduling framework that integrates the Model Predictive Control (MPC) with a Deep Reinforcement Learning algorithm, specifically the Deep Deterministic Policy Gradient (DDPG). The proposed method is designed to optimize energy management in hydrogen-powered UAVs across diverse flight missions. The energy system comprises a proton exchange membrane fuel cell (PEMFC), a lithium-ion battery, and a hydrogen storage tank, enabling robust optimization through the synergistic application of MPC and DDPG. The simulation results demonstrate that the MPC effectively minimizes electric power consumption under various flight conditions, while the DDPG achieves convergence and facilitates efficient scheduling. By leveraging advanced mechanisms, including continuous action space representation, efficient policy learning, experience replay, and target networks, the proposed approach significantly enhances optimization performance and system stability in complex, continuous decision-making scenarios.<\/jats:p>","DOI":"10.3390\/a18020080","type":"journal-article","created":{"date-parts":[[2025,2,4]],"date-time":"2025-02-04T05:13:43Z","timestamp":1738646023000},"page":"80","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Energy Scheduling of Hydrogen Hybrid UAV Based on Model Predictive Control and Deep Deterministic Policy Gradient Algorithm"],"prefix":"10.3390","volume":"18","author":[{"given":"Haitao","family":"Li","sequence":"first","affiliation":[{"name":"State Grid Changzhou Power Supply Company, Changzhou 213200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chenyu","family":"Wang","sequence":"additional","affiliation":[{"name":"National Engineering Research Center of Power Generation Control and Safety, Liyang Research Institute, Southeast University, Liyang 213300, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shufu","family":"Yuan","sequence":"additional","affiliation":[{"name":"National Engineering Research Center of Power Generation Control and Safety, Liyang Research Institute, Southeast University, Liyang 213300, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Zhu","sequence":"additional","affiliation":[{"name":"State Grid Changzhou Power Supply Company, Changzhou 213200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Li","sequence":"additional","affiliation":[{"name":"State Grid Changzhou Power Supply Company, Changzhou 213200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuexin","family":"Liu","sequence":"additional","affiliation":[{"name":"State Grid Changzhou Power Supply Company, Changzhou 213200, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8960-8773","authenticated-orcid":false,"given":"Li","family":"Sun","sequence":"additional","affiliation":[{"name":"National Engineering Research Center of Power Generation Control and Safety, Liyang Research Institute, Southeast University, Liyang 213300, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1049\/cje.2019.12.006","article-title":"Review on the technological development and application of UAV systems","volume":"29","author":"Fan","year":"2020","journal-title":"Chin. J. Electron."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, Y., Zhu, Q., and Elahi, A. (2024). Sequential Convex Programming for Nonlinear Optimal Control in UAV Trajectory Planning. Algorithms, 17.","DOI":"10.3390\/a17070304"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1016\/j.apenergy.2019.02.079","article-title":"Recent advances in fuel cells based propulsion systems for unmanned aerial vehicles","volume":"240","author":"Pan","year":"2019","journal-title":"Appl. Energy"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4371","DOI":"10.1016\/j.ijhydene.2022.10.156","article-title":"An overview: Current progress on hydrogen fuel cell vehicles","volume":"48","author":"Aminudin","year":"2023","journal-title":"Int. J. Hydrogen Energy"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Salah, O., Shamayleh, A., and Mukhopadhyay, S. (2021). Energy management of a multi-source power system. Algorithms, 14.","DOI":"10.3390\/a14070206"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"112132","DOI":"10.1016\/j.est.2024.112132","article-title":"Review of intelligent energy management techniques for hybrid electric vehicles","volume":"92","author":"Urooj","year":"2024","journal-title":"J. Energy Storage"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1016\/j.egyr.2023.05.058","article-title":"Energy management strategy of fuzzy logic control for fuel cell truck","volume":"9","author":"Jin","year":"2023","journal-title":"Energy Rep."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1016\/j.energy.2019.03.155","article-title":"Development of energy management system based on a rule-based power distribution strategy for hybrid power sources","volume":"175","author":"Wang","year":"2019","journal-title":"Energy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"113192","DOI":"10.1016\/j.rser.2023.113192","article-title":"A comprehensive review on optimization of hybrid renewable energy systems using various optimization techniques","volume":"176","author":"Thirunavukkarasu","year":"2023","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.apenergy.2017.10.065","article-title":"Dynamic programming for optimal operation of a biofuel micro CHP-HES system","volume":"208","author":"Chen","year":"2017","journal-title":"Appl. Energy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e12925","DOI":"10.1002\/2050-7038.12925","article-title":"Simultaneous components sizing and flight scheduling for an hybrid aerial vehicle as a multi-energy mobile microgrid","volume":"31","author":"Salehpour","year":"2021","journal-title":"Int. Trans. Electr. Energy Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1016\/j.renene.2022.05.123","article-title":"A review on the integrated optimization techniques and machine learning approaches for modeling, prediction, and decision making on integrated energy systems","volume":"194","author":"Alabi","year":"2022","journal-title":"Renew. Energy"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"109991","DOI":"10.1016\/j.ijepes.2024.109991","article-title":"A frequency cooperative control strategy for multi microgrids with EVs based on improved evolutionary-deep reinforcement learning","volume":"159","author":"Fan","year":"2024","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Oleh, U., Obermaisser, R., and Ahammed, A.S. (2024). A Review of Recent Techniques for Human Activity Recognition: Multimodality, Reinforcement Learning, and Language Models. Algorithms, 17.","DOI":"10.3390\/a17100434"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/MIE.2022.3148568","article-title":"Reinforcement learning energy management for fuel cell hybrid systems: A review","volume":"17","author":"Li","year":"2022","journal-title":"IEEE Ind. Electron. Mag."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"7171","DOI":"10.1109\/TVT.2022.3168870","article-title":"Energy management strategy for fuel cell\/battery\/ultracapacitor hybrid electric vehicles using deep reinforcement learning with action trimming","volume":"71","author":"Fu","year":"2022","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1016\/j.apenergy.2015.02.089","article-title":"Zero carbon infinite cop heat from fuel cell CHP","volume":"147","author":"Staffell","year":"2015","journal-title":"Appl. Energy"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Saco, A., Sundari, P.S., and Paul, A. (2022). An optimized data analysis on a real-time application of PEM fuel cell design by using machine learning algorithms. Algorithms, 15.","DOI":"10.3390\/a15100346"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1872","DOI":"10.1109\/TIA.2014.2345951","article-title":"Study of the characteristics of battery packs in electric vehicles with parallel-connected lithium-ion battery cells","volume":"51","author":"Gong","year":"2015","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6610","DOI":"10.1016\/j.ijhydene.2016.03.004","article-title":"Standardized equation for hydrogen gas compressibility factor for fuel consumption applications","volume":"41","author":"Zheng","year":"2016","journal-title":"Int. J. Hydrogen Energy"},{"key":"ref_21","unstructured":"Lemmon, E.W., Huber, M.L., and McLinden, M.O. (2010). NIST Standard Reference Database 23. Reference Fluid Thermodynamic and Transport Properties (REFPROP), Version 10.0, National Institute of Standards and Technology."},{"key":"ref_22","first-page":"200","article-title":"Investigation of the Standard Deviation of Ornstein-Uhlenbeck Noise in the DDPG Algorithm","volume":"9","author":"Bingol","year":"2021","journal-title":"Gazi Univ. J. Sci. Part C Des. Technol."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/2\/80\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:25:56Z","timestamp":1760027156000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/2\/80"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,2]]},"references-count":22,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["a18020080"],"URL":"https:\/\/doi.org\/10.3390\/a18020080","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,2]]}}}