{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T12:43:33Z","timestamp":1780317813310,"version":"3.54.1"},"reference-count":23,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,1,25]],"date-time":"2025-01-25T00:00:00Z","timestamp":1737763200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With increasing power system complexity and distributed energy penetration, traditional voltage control methods struggle with dynamic changes and complex conditions. While existing deep reinforcement learning (DRL) methods have advanced grid control, challenges persist in leveraging topological features and ensuring computational efficiency. To address these issues, this paper proposes a DRL method combining Graph Convolutional Networks (GCNs) and soft actor-critic (SAC) for voltage control through load shedding. The method uses GCNs to extract higher-order topological features of the power grid, enhancing the state representation capability, while the SAC optimizes the load shedding strategy in continuous action space, dynamically adjusting the control scheme to balance load shedding costs and voltage stability. Results from the simulation of the IEEE 39-bus system indicate that the proposed method significantly reduces the amount of load shedding, improves voltage recovery levels, and demonstrates strong control performance and robustness when dealing with complex disturbances and topological changes. This study provides an innovative solution to voltage control problems in smart grids.<\/jats:p>","DOI":"10.3390\/s25030733","type":"journal-article","created":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T07:54:08Z","timestamp":1738050848000},"page":"733","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Graph-Based Topological Embedding and Deep Reinforcement Learning for Autonomous Voltage Control in Power System"],"prefix":"10.3390","volume":"25","author":[{"given":"Hongtao","family":"Wei","sequence":"first","affiliation":[{"name":"College of Information Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Siyu","family":"Chang","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiaming","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Information Engineering, Wuhan University of Technology, Wuhan 430070, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,1,25]]},"reference":[{"key":"ref_1","first-page":"213","article-title":"Deep Reinforcement Learning for Power System Applications: An Overview","volume":"6","author":"Zhang","year":"2020","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2704","DOI":"10.1109\/TPWRS.2016.2629620","article-title":"An Efficient Optimal Control Method for Open-Loop Transient Stability Emergency Control","volume":"32","author":"Li","year":"2017","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1449","DOI":"10.1109\/TPWRS.2021.3105090","article-title":"Real-Time Adaptive UVLS by Optimized Fuzzy Controllers for Short-Term Voltage Stability Control","volume":"37","author":"Yang","year":"2022","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jianjun, Z., Dongyu, S., Dong, Z., and Yang, G. (2018, January 17\u201319). Load Shedding Control Strategy for Power System Based on the System Frequency and Voltage Stability. Proceedings of the 2018 China International Conference on Electricity Distribution (CICED), Tianjin, China.","DOI":"10.1109\/CICED.2018.8592262"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Usman, M., Amin, A., Azam, M.M., and Mokhlis, H. (2018, January 9\u201310). Optimal under Voltage Load. Shedding Scheme for a Distribution Network Using. EPSO Algorithm. Proceedings of the 2018 1st International Conference on Power, Energy and Smart Grid (ICPESG), Mirpur Azad Kashmir, Pakistan.","DOI":"10.1109\/ICPESG.2018.8384525"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lozano, P.D., Marquez, R.C., and Pacis, M.C. (2022, January 1\u20134). A Phasor Measurement Unit Load Shedding Algorithm Integrated with Voltage Stability Analysis Using a Decision Tree Algorithm for Subtransmission Networks. Proceedings of the 2022 IEEE 14th International Conference on Humanoid, Nanotechnology, Information Technology, Communication and Control, Environment, and Management (HNICEM), Boracay Island, Philippines.","DOI":"10.1109\/HNICEM57413.2022.10109403"},{"key":"ref_7","first-page":"6918","article-title":"Reinforcement Learning for Electric Power System Decision and Control: Past Considerations and Perspectives","volume":"50","author":"Glavic","year":"2017","journal-title":"IFAC-Pap."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Chen, S., Bai, Y., and Jun, Z. (November, January 30). Dynamic Load. Shedding Strategy Using. Distributional Deep. Reinforcement Learning in Power System Emergency Control. Proceedings of the 2020 IEEE 4th Conference on Energy Internet and Energy System Integration (EI2), Wuhan, China.","DOI":"10.1109\/EI250167.2020.9346749"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, Y., and Yue, M. (2024, January 19\u201322). Cooperative Multi-Agent Deep Reinforcement Learning for Adaptive Decentralized Emergency Voltage Control. Proceedings of the 2024 IEEE Power Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA.","DOI":"10.1109\/ISGT59692.2024.10454171"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1109\/TPWRS.2021.3095179","article-title":"Accelerated Derivative-Free Deep Reinforcement Learning for Large-Scale Grid Emergency Voltage Control","volume":"37","author":"Huang","year":"2022","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_11","first-page":"13","article-title":"Parallel System Based Quantitative Assessment and Self-Evolution for Artificial Intelligence of Active Power Corrective Control","volume":"10","author":"Zhang","year":"2024","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1653","DOI":"10.1109\/TPWRS.2018.2881359","article-title":"Data-Driven Load Frequency Control for Stochastic Power Systems: A Deep Reinforcement Learning Method with Continuous Action Search","volume":"34","author":"Yan","year":"2019","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_13","unstructured":"Kipf, T.N., and Welling, M. (2017). Semi-Supervised Classification with Graph Convolutional Networks. arXiv."},{"key":"ref_14","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2037","DOI":"10.1109\/TSG.2020.3041620","article-title":"Two-Stage Deep Reinforcement Learning for Inverter-Based Volt-VAR Control in Active Distribution Networks","volume":"12","author":"Liu","year":"2021","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Gao, Y., Wang, W., Shi, J., and Yu, N. (2020). Batch-Constrained Reinforcement Learning for Dynamic Distribution Network Reconfiguration. arXiv.","DOI":"10.1109\/TSG.2020.3005270"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4120","DOI":"10.1109\/TPWRS.2020.3000652","article-title":"A Multi-Agent Deep Reinforcement Learning Based Voltage Regulation Using Coordinated PV Inverters","volume":"35","author":"Cao","year":"2020","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"114772","DOI":"10.1016\/j.apenergy.2020.114772","article-title":"Safe Deep Reinforcement Learning-Based Constrained Optimal Control Scheme for Active Distribution Networks","volume":"264","author":"Kou","year":"2020","journal-title":"Appl. Energy"},{"key":"ref_19","unstructured":"Fujimoto, S., van Hoof, H., and Meger, D. (2018). Addressing Function Approximation Error in Actor-Critic Methods. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"108175","DOI":"10.1016\/j.ijepes.2022.108175","article-title":"Flexible Selection Framework for Secondary Frequency Regulation Units Based on Learning Optimisation Method","volume":"142","author":"Zhang","year":"2022","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"4067","DOI":"10.1109\/TPWRS.2022.3142969","article-title":"Indirect Multi-Energy Transactions of Energy Internet with Deep Reinforcement Learning Approach","volume":"37","author":"Yang","year":"2022","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level Control Through Deep Reinforcement Learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_23","unstructured":"PJM Transmission Planning Department (2009). Exelon Transmission Planning Criteria, PJM Transmission Planning Department. Available online: https:\/\/www.pjm.com\/-\/media\/DotCom\/planning\/planning-criteria\/exelon-planning-criteria.ashx."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/3\/733\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T10:36:08Z","timestamp":1759919768000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/3\/733"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,25]]},"references-count":23,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,2]]}},"alternative-id":["s25030733"],"URL":"https:\/\/doi.org\/10.3390\/s25030733","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,25]]}}}