{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T13:11:56Z","timestamp":1784121116845,"version":"3.55.0"},"reference-count":33,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T00:00:00Z","timestamp":1679443200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science and Technology Project of China Southern Power Grid Yunnan Power Grid Co., Ltd.","award":["YNKJXM20220048"],"award-info":[{"award-number":["YNKJXM20220048"]}]},{"name":"Science and Technology Project of China Southern Power Grid Yunnan Power Grid Co., Ltd.","award":["2021MD703895"],"award-info":[{"award-number":["2021MD703895"]}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["YNKJXM20220048"],"award-info":[{"award-number":["YNKJXM20220048"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2021MD703895"],"award-info":[{"award-number":["2021MD703895"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>As a promising information theory, reinforcement learning has gained much attention. This paper researches a wind-storage cooperative decision-making strategy based on dueling double deep Q-network (D3QN). Firstly, a new wind-storage cooperative model is proposed. Besides wind farms, energy storage systems, and external power grids, demand response loads are also considered, including residential price response loads and thermostatically controlled loads (TCLs). Then, a novel wind-storage cooperative decision-making mechanism is proposed, which combines the direct control of TCLs with the indirect control of residential price response loads. In addition, a kind of deep reinforcement learning algorithm called D3QN is utilized to solve the wind-storage cooperative decision-making problem. Finally, the numerical results verify the effectiveness of D3QN for optimizing the decision-making strategy of a wind-storage cooperation system.<\/jats:p>","DOI":"10.3390\/e25030546","type":"journal-article","created":{"date-parts":[[2023,3,22]],"date-time":"2023-03-22T07:09:31Z","timestamp":1679468971000},"page":"546","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["An Improved Deep Reinforcement Learning Method for Dispatch Optimization Strategy of Modern Power Systems"],"prefix":"10.3390","volume":"25","author":[{"given":"Suwei","family":"Zhai","sequence":"first","affiliation":[{"name":"Electric Power Research Institute of China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenyun","family":"Li","sequence":"additional","affiliation":[{"name":"Yunnan Power Dispatching Control Center of China Southern Power Grid, Kunming 650011, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhenyu","family":"Qiu","sequence":"additional","affiliation":[{"name":"College of IOT Engineering, Hohai University, Nanjing 210098, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyi","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of IOT Engineering, Hohai University, Nanjing 210098, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shixi","family":"Hou","sequence":"additional","affiliation":[{"name":"College of IOT Engineering, Hohai University, Nanjing 210098, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Li, J., and Zhao, H. (2021). Multi-Objective Optimization and Performance Assessments of an Integrated Energy System Based on Fuel, Wind 541 and Solar Energies. Entropy, 23.","DOI":"10.3390\/e23040431"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bin, L., Shahzad, M., Javed, H., Muqeet, H.A., Akhter, M.N., Liaqat, R., and Hussain, M.M. (2022). Scheduling and Sizing of Campus Microgrid Considering Demand Response and Economic Analysis. Sensors, 22.","DOI":"10.3390\/s22166150"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chu, Y., Fu, S., Hou, S., and Fei, J. (2022). Intelligent Terminal Sliding Mode Control of Active Power Filters by Self-evolving Emotional Neural Network. IEEE Trans. Ind. Inform.","DOI":"10.1109\/TII.2022.3168654"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Almughram, O., Ben Slama, S., and Zafar, B. (2022). Model for Managing the Integration of a Vehicle-to-Home Unit into an Intelligent Home Energy Management System. Sensors, 22.","DOI":"10.3390\/s22218142"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/TIA.2017.2754978","article-title":"Generation Scheduling Optimization of Wind-Energy Storage System Based on Wind Power Output Fluctuation Features","volume":"54","author":"Shi","year":"2018","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"4745","DOI":"10.1109\/TPWRS.2021.3067199","article-title":"VSC-HVDC Interties for Urban Power Grid Enhancement","volume":"36","author":"Sun","year":"2021","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2029","DOI":"10.1109\/TCST.2019.2923779","article-title":"Model-Optimized Dispatch for Closed-Loop Power Control of Waked Wind Farms","volume":"28","author":"Kazda","year":"2020","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1086","DOI":"10.1109\/TSTE.2022.3150965","article-title":"A Frequency Security Constrained Scheduling Approach Considering Wind Farm Providing Frequency Support and Reserve","volume":"13","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3251","DOI":"10.1109\/TIE.2020.2979560","article-title":"Deep Neural Learning Based Distributed Predictive Control for Offshore Wind Farm Using High-Fidelity LES Data","volume":"68","author":"Yin","year":"2021","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1100","DOI":"10.1109\/TSTE.2019.2929673","article-title":"Online Tracking of Reactive Power Reserve for Wind Farms","volume":"11","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1861","DOI":"10.1109\/TSTE.2022.3156426","article-title":"Self-Dispatch of Wind-Storage Integrated System: A Deep Reinforcement Learning Approach","volume":"13","author":"Wei","year":"2022","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3520","DOI":"10.1109\/TPWRS.2021.3051523","article-title":"Two-Stage Chance-Constrained Stochastic Thermal Unit Commitment for Optimal Provision of Virtual Inertia in Wind-Storage Systems","volume":"36","author":"Ding","year":"2021","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4639","DOI":"10.1109\/TNNLS.2020.3025711","article-title":"Learning Automata-Based Multiagent Reinforcement Learning for Optimization of Cooperative Tasks","volume":"32","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3612","DOI":"10.1109\/TNNLS.2021.3053633","article-title":"Optimizing Attention for Sequence Modeling via Reinforcement Learning","volume":"33","author":"Fei","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3964","DOI":"10.1109\/TPWRS.2019.2917632","article-title":"Cooperation-Based Distributed Economic MPC for Economic Load Dispatch and Load Frequency Control of Interconnected Power Systems","volume":"34","author":"Jia","year":"2019","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"860","DOI":"10.1109\/TCST.2021.3070861","article-title":"Control Performance Standards-Oriented Event-Triggered Load Frequency Control for Power Systems Under Limited Communication Bandwidth","volume":"30","author":"Shangguan","year":"2022","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chu, Y., Hou, S., Wang, C., and Fei, J. (2023). Recurrent-Neural-Network-Based Fractional Order Sliding Mode Control for Harmonic Suppression of Power Grid. IEEE Trans. Ind. Inform., 305.","DOI":"10.1109\/TII.2023.3234305"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"985","DOI":"10.1109\/JSYST.2020.3040217","article-title":"A Clustering-Based Approach for Wind Farm Placement in Radial Distribution Systems Considering Wake Effect and a Time-Acceleration Constraint","volume":"15","author":"Sadeghian","year":"2021","journal-title":"IEEE Syst. J."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"6547","DOI":"10.1109\/TIA.2021.3105497","article-title":"A Control Strategy Based on Deep Reinforcement Learning Under the Combined Wind-Solar Storage System","volume":"57","author":"Huang","year":"2021","journal-title":"IEEE Trans. Ind. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"108608","DOI":"10.1016\/j.ijepes.2022.108608","article-title":"Deep reinforcement learning based energy storage management strategy considering prediction intervals of wind power","volume":"145","author":"Liu","year":"2023","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"105928","DOI":"10.1016\/j.ijepes.2020.105928","article-title":"A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing","volume":"119","author":"Yang","year":"2020","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sang, J., Sun, H., and Kou, L. (2022). Deep Reinforcement Learning Microgrid Optimization Strategy Considering Priority Flexible Demand Side. Sensors, 22.","DOI":"10.3390\/s22062256"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2647","DOI":"10.1016\/j.egyr.2021.04.055","article-title":"A novel energy management method based on Deep Q Network algorithm for low operating cost of an integrated hybrid system","volume":"7","author":"Sanaye","year":"2021","journal-title":"Energy Rep."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"792","DOI":"10.1016\/j.renene.2022.10.094","article-title":"Optimal scheduling of a wind energy dominated distribution network via a deep reinforcement learning approach","volume":"201","author":"Zhu","year":"2022","journal-title":"Renew. Energy"},{"key":"ref_25","unstructured":"Fingrid (2019, December 12). Fingrid Open Datasets. Available online: https:\/\/data.fingrid.fi\/open-data-forms\/search\/en\/index.html."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.apenergy.2017.12.056","article-title":"Community energy storage: A smart choice for the smart grid?","volume":"212","author":"Barbour","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3259","DOI":"10.1109\/TSG.2016.2629450","article-title":"Convolutional neural networks for automatic state-time feature extraction in reinforcement learning applied to residential load control","volume":"9","author":"Claessens","year":"2018","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1619","DOI":"10.12688\/f1000research.20421.1","article-title":"Optimal price-based control of heterogeneous thermostatically controlled loads under uncertainty using LSTM networks and genetic algorithms","volume":"8","author":"Nakabi","year":"2019","journal-title":"F1000Research"},{"key":"ref_29","first-page":"4236","article-title":"Robust coordination of distributed generation and price-based demand response in microgrids. IEEE Trans","volume":"9","author":"Zhang","year":"2018","journal-title":"Smart Grid"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1109\/TPWRS.2013.2283516","article-title":"Value of price responsive load for wind integration in unit commitment","volume":"29","author":"Hobbs","year":"2014","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3112","DOI":"10.1109\/TSG.2018.2817925","article-title":"Impact of Uncertain Parameters on TCL Power Capacity Calculation via HDMR for Generating Power Pulses","volume":"10","author":"Song","year":"2019","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_32","unstructured":"(2019, December 16). Residential Electric Rates & Line Items. Available online: https:\/\/austinenergy.com\/ae\/residential\/rates\/residential-electric-rates-and-line-items."},{"key":"ref_33","unstructured":"Littman, M.L. (2001). International Encyclopedia of the Social & Behavioral Sciences, Elsevier."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/3\/546\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:00:34Z","timestamp":1760122834000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/3\/546"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,22]]},"references-count":33,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["e25030546"],"URL":"https:\/\/doi.org\/10.3390\/e25030546","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,22]]}}}