{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,6]],"date-time":"2025-12-06T16:50:09Z","timestamp":1765039809059,"version":"build-2065373602"},"reference-count":25,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,5,30]],"date-time":"2025-05-30T00:00:00Z","timestamp":1748563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52377111","52407122"],"award-info":[{"award-number":["52377111","52407122"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Large-scale integration of renewable energy introduces significant random perturbations to the power system, disrupting the symmetry and balance of active power, which complicates the stabilization of the system\u2019s frequency. Inter-regional energy cooperation plays a crucial role in maintaining the symmetry and balance of the overall power system\u2019s active power. However, when the power system area is expanded, automatic generation control (AGC) based on reinforcement learning faces the challenge of not being able to leverage the prior experience of the original system topology to train the new area, making it difficult to quickly develop an effective control strategy. To address these challenges, this paper proposes a novel data-driven AGC method that employs a multi-agent reinforcement learning algorithm with a learning to coordinate and teach reinforcement (LECTR) mechanism. Specifically, under the LECTR mechanism, when the power system region expands, agents in the original region will instruct the agents in the newly merged region by providing demonstration actions. This accelerates the convergence of their strategy networks and improves control accuracy. Additionally, the proposed algorithm introduces a double critic network to mitigate the issue of target critic network value overestimation in reinforcement learning, thereby obtaining higher-quality empirical data and improving algorithm stability. Finally, simulations are conducted to evaluate the method\u2019s effectiveness in scenarios with an increasing number of IEEE interconnected grid areas.<\/jats:p>","DOI":"10.3390\/sym17060854","type":"journal-article","created":{"date-parts":[[2025,5,30]],"date-time":"2025-05-30T09:22:41Z","timestamp":1748596961000},"page":"854","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Data-Driven Automatic Generation Control Based on Learning to Coordinate and Teach Reinforcement Mechanism"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0359-1187","authenticated-orcid":false,"given":"Fan","family":"Yang","sequence":"first","affiliation":[{"name":"College of Electric Engineering, Shanghai University of Electric Power, Shanghai 200090, China"}]},{"given":"Xinyi","family":"Shao","sequence":"additional","affiliation":[{"name":"Pudong Power Supply Branch State Grid, Shanghai Electric Power Company, Shanghai 200120, China"}]},{"given":"Bo","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Electric Engineering, Shanghai University of Electric Power, Shanghai 200090, China"}]},{"given":"Yuexing","family":"Shi","sequence":"additional","affiliation":[{"name":"College of Electric Engineering, Shanghai University of Electric Power, Shanghai 200090, China"}]},{"given":"Yunwei","family":"Shen","sequence":"additional","affiliation":[{"name":"College of Electric Engineering, Shanghai University of Electric Power, Shanghai 200090, China"}]},{"given":"Dongdong","family":"Li","sequence":"additional","affiliation":[{"name":"College of Electric Engineering, Shanghai University of Electric Power, Shanghai 200090, China"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1377465","DOI":"10.3389\/fenrg.2024.1377465","article-title":"Generating adversarial deep reinforcement learning -based frequency control of island city microgrid considering generalization of scenarios","volume":"12","author":"Wang","year":"2024","journal-title":"Front Energy Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"4132","DOI":"10.1002\/er.5033","article-title":"Limitations, challenges, and solution approaches in grid-connected renewable energy systems","volume":"44","author":"Basit","year":"2020","journal-title":"Int. J. Energy Res."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4769","DOI":"10.1109\/TPWRS.2020.2989533","article-title":"Integrated modelling and enhanced utilization of power-to-ammonia for high renewable penetrated multi-energy systems","volume":"35","author":"Xu","year":"2020","journal-title":"IEEE Trans Power Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Liu, Z., Li, J., Zhang, P., Ding, Z., and Zhao, Y. (2022). An AGC dynamic optimization method based on proximal policy optimization. Front. Energy Res., 10.","DOI":"10.3389\/fenrg.2022.947532"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1109\/TSG.2021.3109627","article-title":"Adaptive frequency control of microgrid based on fractional order control and a data-driven control with stability analysis","volume":"13","author":"Kazemi","year":"2022","journal-title":"IEEE Trans. Smart Grid."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s41601-022-00238-x","article-title":"Novel chaos game optimization tuned-fractional-order PID fractional-order Pi Controller for load\u2013frequency control of interconnected power systems","volume":"7","author":"Barakat","year":"2022","journal-title":"Prot. Control. Mod. Power Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1186\/s41601-023-00289-8","article-title":"Effective participation of wind turbines in frequency control of a two-area power system using Coot Optimization","volume":"8","author":"Lotfy","year":"2023","journal-title":"Prot. Control. Mod. Power Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1186\/s41601-022-00241-2","article-title":"and Shaw. B. A new two-degree of Freedom Combined PID controller for automatic generation control of a wind integrated interconnected power system","volume":"7","author":"Karanam","year":"2022","journal-title":"Prot. Control. Mod. Power Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"127417","DOI":"10.1109\/ACCESS.2024.3454794","article-title":"A novel proportional-derivative sliding mode for load frequency control","volume":"12","author":"Guo","year":"2024","journal-title":"IEEE Access."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1186\/s41601-022-00266-7","article-title":"Cascade Controller based modeling of a four area thermal: Gas AGC system with dependency of wind turbine generator and pevs under restructured environment","volume":"7","author":"Saha","year":"2022","journal-title":"Prot. Control. Mod. Power Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5151","DOI":"10.1109\/TIE.2017.2668983","article-title":"Distributed multi-agent system-based load frequency control for multi-area power system in smart grid","volume":"64","author":"Singh","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1669","DOI":"10.1109\/TPWRS.2014.2357079","article-title":"Multi-agent correlated equilibrium Q(\u03bb) learning for coordinated smart generation control of interconnected power grids","volume":"30","author":"Yu","year":"2015","journal-title":"IEEE Trans Power Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1048","DOI":"10.1109\/TSG.2020.3034827","article-title":"Multi-agent safe policy learning for power management of networked microgrids","volume":"12","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Smart Grid."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2483","DOI":"10.1109\/TNNLS.2020.3006080","article-title":"Automatic Generation Control Based on Multiple Neural Networks with Actor-Critic Strategy","volume":"32","author":"Xi","year":"2021","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1272","DOI":"10.1109\/TPWRS.2010.2102372","article-title":"Stochastic Optimal Relaxed Automatic Generation Control in Non-Markov Environment Based on Multi-Step Q(\u03bb)","volume":"26","author":"Yu","year":"2011","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"977","DOI":"10.1016\/j.energy.2018.04.042","article-title":"Smart generation control based on multi-agent reinforcement learning with the idea of the Time Tunnel","volume":"153","author":"Xi","year":"2018","journal-title":"Energy"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1049\/iet-gtd.2009.0168","article-title":"Load\u2013frequency control: A GA-based multi-agent reinforcement learning","volume":"4","author":"Daneshfar","year":"2010","journal-title":"IET Gener. Transm. Distrib."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5525","DOI":"10.1109\/TPWRS.2021.3078446","article-title":"A Deep Reinforcement Learning-Based Multi-Agent Framework to Enhance Power System Resilience Using Shunt Resources","volume":"36","author":"Kamruzzaman","year":"2021","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"156839","DOI":"10.1109\/ACCESS.2020.3019535","article-title":"Deep Reinforcement Learning Based Multi-Objective Integrated Automatic Generation Control for Multiple Continuous Power Disturbances","volume":"8","author":"Li","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"5560","DOI":"10.1109\/TPWRS.2022.3223255","article-title":"Data-driven load frequency control based on multi-agent reinforcement learning with attention mechanism","volume":"38","author":"Yang","year":"2023","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2417","DOI":"10.1109\/TSTE.2019.2958361","article-title":"A novel multi-agent DDQN-AD method-based distributed strategy for automatic generation control of Integrated Energy Systems","volume":"11","author":"Xi","year":"2020","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_22","first-page":"6128","article-title":"Learning to teach in cooperative multiagent reinforcement learning","volume":"33","author":"Omidshafiei","year":"2019","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2518","DOI":"10.1109\/TASE.2023.3263005","article-title":"Brain-inspired deep meta-reinforcement learning for active coordinated fault-tolerant load frequency control of multi-area grids","volume":"21","author":"Li","year":"2024","journal-title":"IEEE Trans. Autom Sci Eng."},{"key":"ref_24","first-page":"04018033","article-title":"Design of a novel smart generation controller based on deep Q learning for large-scale interconnected power system","volume":"144","author":"Yin","year":"2018","journal-title":"J. Energy Chem."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1653","DOI":"10.1109\/TPWRS.2018.2881359","article-title":"Data-driven load frequency control for stochastic power systems: A deep reinforcement learning method with Continuous Action Search","volume":"34","author":"Yan","year":"2019","journal-title":"IEEE Trans. Power Syst."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/6\/854\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:44:07Z","timestamp":1760031847000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/6\/854"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,30]]},"references-count":25,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["sym17060854"],"URL":"https:\/\/doi.org\/10.3390\/sym17060854","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,5,30]]}}}