{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:02:05Z","timestamp":1760144525851,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,4,14]],"date-time":"2024-04-14T00:00:00Z","timestamp":1713052800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U2006222"],"award-info":[{"award-number":["U2006222"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Surrounded by the Shandong Peninsula, the Bohai Sea and Yellow Sea possess vast marine energy resources. An analysis of actual meteorological data from these regions indicates significant seasonality and intra-day uncertainty in wind and photovoltaic power generation. The challenge of scheduling to leverage the complementary characteristics of various renewable energy sources for maintaining grid stability is substantial. In response, we have integrated wave energy with offshore photovoltaic and wind power generation and propose a day-ahead and intra-day multi-time-scale rolling optimization scheduling strategy for the complementary dispatch of these three energy sources. Using real meteorological data from this maritime area, we employed a CNN-LSTM neural network to predict the power generation and load demand of the area on both day-ahead 24 h and intra-day 1 h time scales, with the DDPG algorithm applied for refined electricity management through rolling optimization scheduling of the forecast data. Simulation results demonstrate that the proposed strategy effectively meets load demands through complementary scheduling of wave power, wind power, and photovoltaic power generation based on the climatic characteristics of the Bohai and Yellow Sea regions, reducing the negative impacts of the seasonality and intra-day uncertainty of these three energy sources on the grid. Additionally, compared to the day-ahead scheduling strategy alone, the day-ahead and intra-day rolling optimization scheduling strategy achieved a reduction in system costs by 16.1% and 22% for a typical winter day and a typical summer day, respectively.<\/jats:p>","DOI":"10.3390\/e26040331","type":"journal-article","created":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T10:24:26Z","timestamp":1713176666000},"page":"331","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Multi-Time-Scale Optimal Scheduling Strategy for Marine Renewable Energy Based on Deep Reinforcement Learning Algorithm"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-4389-6547","authenticated-orcid":false,"given":"Ren","family":"Xu","sequence":"first","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2771-7688","authenticated-orcid":false,"given":"Fei","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]},{"given":"Wenyi","family":"Shao","sequence":"additional","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]},{"given":"Haoran","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]},{"given":"Fanping","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]},{"given":"Jun","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information and Automation, Qilu University of Technology, Jinan 250353, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/TSTE.2021.3131560","article-title":"Investigating the Complementarity Characteristics of Wind and Solar Power for Load Matching Based on the Typical Load Demand in China","volume":"13","author":"Ren","year":"2022","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"218","DOI":"10.12720\/sgce.7.3.218-224","article-title":"Review on China\u2019s renewable energy and future projections","volume":"7","author":"Zhang","year":"2018","journal-title":"Int. J. Smart Grid Clean Energy"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1109\/TIA.2017.2782207","article-title":"Ultra-Short-Term wind generation forecast based on multivariate empirical dynamic modeling","volume":"54","author":"Ma","year":"2018","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1109\/TSTE.2018.2831238","article-title":"A multi-model combination approach for probabilistic wind power forecasting","volume":"10","author":"Lin","year":"2019","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1109\/TSTE.2020.2978634","article-title":"Confidence interval based distributionally robust real-time economic dispatch approach considering wind power accommodation risk","volume":"12","author":"Li","year":"2021","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/TSTE.2020.2964949","article-title":"Risk based distributionally robust real-time dispatch considering voltage security","volume":"12","author":"Li","year":"2021","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1016\/j.apenergy.2018.05.043","article-title":"Time-series prediction of wind speed using machine learning algorithms: A case study Osorio wind farm Brazil","volume":"224","author":"Khosravi","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1109\/JPHOTOV.2019.2898521","article-title":"Prediction model for PV performance with correlation analysis of environmental variables","volume":"9","author":"Kim","year":"2019","journal-title":"IEEE J. Photovolt."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1186\/s41601-020-00167-7","article-title":"Bayesian bootstrap quantile regression for probabilistic photovoltaic power forecasting","volume":"5","author":"Bozorg","year":"2020","journal-title":"Prot. Control Mod. Power Syst."},{"key":"ref_10","first-page":"3385","article-title":"Deep learning based surface irradiance mapping model for solar PV power forecasting using sky image","volume":"56","author":"Zhen","year":"2020","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.renene.2013.11.067","article-title":"An ARMAX model for forecasting the power output of a grid connected photovoltaic system","volume":"66","author":"Li","year":"2014","journal-title":"Renew. Energy"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhang, L., Shi, J., Wang, L., and Xu, C. (2020). Electricity, Heat, and Gas Load Forecasting Based on Deep Multitask Learning in Industrial-Park Integrated Energy System. Entropy, 22.","DOI":"10.3390\/e22121355"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1109\/TSG.2010.2078842","article-title":"Short-term load forecast of microgrids by a new bilevel prediction strategy","volume":"1","author":"Amjady","year":"2008","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/TSTE.2016.2609283","article-title":"Optimization and Reliability Evaluation of an Offshore Wind Farm Architecture","volume":"8","author":"Dahmani","year":"2017","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e12005","DOI":"10.1002\/2050-7038.12005","article-title":"A multi-energy complementary coordinated dispatch method for integrated system of wind-photovoltaic-hydro-thermal-energy storage","volume":"29","author":"Xiao","year":"2019","journal-title":"Int. Trans. Electr. Energy Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6547","DOI":"10.1109\/TIA.2021.3105497","article-title":"A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System","volume":"57","author":"Huang","year":"2021","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"283","DOI":"10.17775\/CSEEJPES.2017.01250","article-title":"Day-ahead scheduling of multi-carrier energy systems with multi-type energy storages and wind power","volume":"4","author":"Zhang","year":"2018","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_18","first-page":"2130","article-title":"Convexification Technique for Optimal Thermal-Wind-Photovoltaic Coordination Dispatch Incorporating an Energy Storage System","volume":"9","author":"Wang","year":"2023","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1357","DOI":"10.1016\/j.renene.2016.10.022","article-title":"Optimal scheduling of thermal-wind-solar power system with storage","volume":"101","author":"Reddy","year":"2017","journal-title":"Renew. Energy"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, Y., Chen, A., Liu, T., and Wang, W. (2022, January 27\u201330). Assessment and Configuration of the Wind-PV-wave Complementary System for Improving the Stability and Power Generation Ability. Proceedings of the 2022 4th International Conference on Smart Power & Internet Energy Systems (SPIES), Beijing, China.","DOI":"10.1109\/SPIES55999.2022.10082339"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2075","DOI":"10.1016\/j.renene.2020.09.131","article-title":"Feasibility study of on\/off grid large-scale PV\/WT\/WEC hybrid energy system in coastal cities: A case-based research","volume":"162","author":"Jahangir","year":"2020","journal-title":"Renew. Energy"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1016\/j.renene.2021.04.048","article-title":"Evaluating the complementarity of solar, wind and hydropower to mitigate the impact of El Ni\u00f1o Southern Oscillation in Latin America","volume":"174","author":"Poganietz","year":"2021","journal-title":"Renew. Energy"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1174","DOI":"10.1016\/j.enconman.2019.05.088","article-title":"Hydrodynamic response and power efficiency analysis of heaving wave energy converter integrated with breakwater","volume":"195","author":"Reabroy","year":"2019","journal-title":"Energy Convers. Manag."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Pecher, A., and Kofoed, J.P. (2017). Handbook of Ocean Wave Energy, Springer Nature.","DOI":"10.1007\/978-3-319-39889-1"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gao, W., and Lin, Y. (2023). Energy Dispatch for CCHP System in Summer Based on Deep Reinforcement Learning. Entropy, 25.","DOI":"10.3390\/e25030544"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chen, S., Li, J., Jiang, C., and Xiao, W. (2022). Optimal Energy-Storage Configuration for Microgrids Based on SOH Estimation and Deep Q-Network. Entropy, 24.","DOI":"10.3390\/e24050630"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2289","DOI":"10.1109\/JSAC.2021.3087233","article-title":"Joint deep reinforcement learning and unfolding: Beam selection and precoding for mmWave multiuser MIMO with lens arrays","volume":"39","author":"Hu","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/TCCN.2018.2809722","article-title":"Deep reinforcement learning for dynamic multichannel access in wireless networks","volume":"4","author":"Wang","year":"2018","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4674","DOI":"10.1109\/TWC.2018.2829773","article-title":"Proactive resource management for LTE in unlicensed spectrum: A deep learning perspective","volume":"17","author":"Challita","year":"2018","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1814","DOI":"10.1109\/LCOMM.2023.3266931","article-title":"DDPG-Based Joint Resource Management for Latency Minimization in NOMA-MEC Networks","volume":"27","author":"Wang","year":"2023","journal-title":"IEEE Commun. Lett."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/331\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:27:49Z","timestamp":1760106469000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/4\/331"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,14]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["e26040331"],"URL":"https:\/\/doi.org\/10.3390\/e26040331","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,4,14]]}}}