{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T18:43:37Z","timestamp":1775673817953,"version":"3.50.1"},"reference-count":34,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T00:00:00Z","timestamp":1775606400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Major municipal science and technology projects","award":["2024-ZD-009"],"award-info":[{"award-number":["2024-ZD-009"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Under the general trend of global energy transformation, the proportion of renewable energy in the power sector continues to increase. Power routers are of great significance for improving energy utilization efficiency and ensuring the stable operation of power systems. However, the intermittent and uncertain nature of distributed energy makes energy management of power routers difficult, and traditional optimization methods are also difficult to adapt. Therefore, this study proposes the integration of Proximal Policy Optimization with a multi-agent framework, combined with a Generative Adversarial Imitation Learning based on a double-buffer mechanism. The double-buffer mechanism is used to improve data utilization efficiency and training stability, and to optimize communication and collaboration among multiple agents, thereby realizing energy collaborative optimization of power routers. Experimental results show that after 420 trainings, the average round reward of the improved algorithm is stable at about \u2212410, and the strategy loss function is the first to stabilize after 500 times. In practical scenarios, the proposed model maintains a DC bus voltage fluctuation range between 728V and 732V. Additionally, its electricity cost amounts to 3846.36 yuan, and its total runtime is 53.32 seconds\u2014both of which are lower than those of the other two models. Overall, the enhanced algorithm and model notably improve the energy collaboration optimization of power routers, offering a practical solution to energy management issues and significantly advancing the progress in this area.<\/jats:p>","DOI":"10.1371\/journal.pone.0346372","type":"journal-article","created":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T17:30:51Z","timestamp":1775669451000},"page":"e0346372","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Energy collaborative optimization of power routing based on PPO and generative adversarial imitation learning"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-0637-3838","authenticated-orcid":true,"given":"Junyan","family":"Lyu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Huang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2026,4,8]]},"reference":[{"issue":"3","key":"pone.0346372.ref001","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1109\/TSTE.2023.3252944","article-title":"Fairness-aware distributed energy coordination for voltage regulation in power distribution systems","volume":"14","author":"S Poudel","year":"2023","journal-title":"IEEE Trans Sustain Energy"},{"issue":"6","key":"pone.0346372.ref002","doi-asserted-by":"crossref","first-page":"7047","DOI":"10.1109\/TIA.2022.3201060","article-title":"Double-Consensus-Based Distributed Energy Management in a Virtual Power Plant","volume":"58","author":"PM Naina","year":"2022","journal-title":"IEEE Trans Ind Appl"},{"issue":"4","key":"pone.0346372.ref003","doi-asserted-by":"crossref","first-page":"3957","DOI":"10.1109\/JESTPE.2023.3277828","article-title":"Overview of wide\/ultrawide bandgap power semiconductor devices for distributed energy resources","volume":"11","author":"SK Mazumder","year":"2023","journal-title":"IEEE J Emerg Sel Top Power Electron"},{"issue":"14","key":"pone.0346372.ref004","doi-asserted-by":"crossref","first-page":"12401","DOI":"10.1109\/JIOT.2023.3246925","article-title":"Optimizing AoI in UAV-RIS-assisted IoT networks: Off policy versus on policy","volume":"10","author":"M Sherman 4","year":"2023","journal-title":"IEEE Internet Things J"},{"issue":"3","key":"pone.0346372.ref005","doi-asserted-by":"crossref","first-page":"2874","DOI":"10.1109\/TITS.2022.3227738","article-title":"Modeling Human Driving Behavior Through Generative Adversarial Imitation Learning","volume":"24","author":"R Bhattacharyya","year":"2023","journal-title":"IEEE Trans Intell Transport Syst"},{"issue":"3","key":"pone.0346372.ref006","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/LRA.2023.3236882","article-title":"Learning From Guided Play: Improving Exploration for Adversarial Imitation Learning With Simple Auxiliary Tasks","volume":"8","author":"T Ablett","year":"2023","journal-title":"IEEE Robot Autom Lett"},{"issue":"11","key":"pone.0346372.ref007","doi-asserted-by":"crossref","first-page":"7434","DOI":"10.1109\/TCOMM.2022.3211071","article-title":"Fair Virtual Network Function Mapping and Scheduling Using Proximal Policy Optimization","volume":"70","author":"Z Kuai","year":"2022","journal-title":"IEEE Trans Commun"},{"issue":"6","key":"pone.0346372.ref008","doi-asserted-by":"crossref","first-page":"4232","DOI":"10.1109\/TWC.2022.3224143","article-title":"Multi-UAV Collaborative Sensing and Communication: Joint Task Allocation and Power Optimization","volume":"22","author":"K Meng","year":"2023","journal-title":"IEEE Trans Wireless Commun"},{"issue":"4","key":"pone.0346372.ref009","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1109\/TCNS.2022.3165022","article-title":"A distributed-optimization-based architecture for management of interconnected energy hubs","volume":"9","author":"G Ferro","year":"2022","journal-title":"IEEE Trans Control Netw Syst"},{"issue":"10","key":"pone.0346372.ref010","doi-asserted-by":"crossref","first-page":"10293","DOI":"10.1109\/TVT.2022.3184514","article-title":"Uncertainty modeling for participation of electric vehicles in collaborative energy consumption","volume":"71","author":"N Hashemipour","year":"2022","journal-title":"IEEE Trans Veh Technol"},{"issue":"7","key":"pone.0346372.ref011","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.70003\/160792642024122507011","article-title":"Optimizing Resource Scheduling for Multi-Scenario Mixed Service Groups under Edge Cloud-Native Environments Using Simulation Learning","volume":"25","author":"W Xiong","year":"2024","journal-title":"J Internet Technol"},{"issue":"1","key":"pone.0346372.ref012","doi-asserted-by":"crossref","first-page":"4024","DOI":"10.1109\/TTE.2024.3450731","article-title":"Safe Reinforcement Learning for Power Allocation of Hybrid Energy Storage Systems in Electric Vehicles Using PPO and Predictive Safety Filter","volume":"11","author":"X Xu","year":"2025","journal-title":"IEEE Trans Transp Electrific"},{"issue":"3","key":"pone.0346372.ref013","doi-asserted-by":"crossref","first-page":"3749","DOI":"10.1109\/JIOT.2023.3316269","article-title":"AGRIC: Artificial-Intelligence-Based Green Routing for Industrial Cyber\u2013Physical System Pertaining to Extreme Environment","volume":"11","author":"S Verma","year":"2024","journal-title":"IEEE Internet Things J"},{"issue":"5","key":"pone.0346372.ref014","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1109\/MCOM.003.2200573","article-title":"DQC2O: Distributed Quantum Computing for Collaborative Optimization in Future Networks","volume":"61","author":"N Ngoenriang","year":"2023","journal-title":"IEEE Commun Mag"},{"issue":"21","key":"pone.0346372.ref015","doi-asserted-by":"crossref","first-page":"18641","DOI":"10.1109\/JIOT.2023.3244854","article-title":"Smart Navigation and Energy Management Framework for Autonomous Electric Vehicles in Complex Environments","volume":"10","author":"G Raja","year":"2023","journal-title":"IEEE Internet Things J"},{"issue":"4","key":"pone.0346372.ref016","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1109\/TCOMM.2023.3333345","article-title":"RIS-empowered MEC for URLLC systems with digital-twin-driven architecture","volume":"72","author":"S Kurma","year":"2024","journal-title":"IEEE Trans Commun"},{"issue":"6","key":"pone.0346372.ref017","doi-asserted-by":"crossref","first-page":"50","DOI":"10.23919\/JCC.2022.06.005","article-title":"Proximal policy optimization-based committee selection algorithm in blockchain-enabled mobile edge computing systems","volume":"19","author":"W Wu","year":"2022","journal-title":"China Commun"},{"issue":"3","key":"pone.0346372.ref018","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1109\/TEMC.2023.3265260","article-title":"The Application of the Cholesky-Based Monte Carlo Method to Evaluate Lightning Channel Base Current","volume":"65","author":"AF Nematollahi","year":"2023","journal-title":"IEEE Trans Electromagn Compat"},{"issue":"1","key":"pone.0346372.ref019","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1109\/MCS.2021.3122269","article-title":"Nonlinear System Identification: Learning While Respecting Physical Models Using a Sequential Monte Carlo Method","volume":"42","author":"A Wigren","year":"2022","journal-title":"IEEE Control Syst"},{"issue":"10","key":"pone.0346372.ref020","doi-asserted-by":"crossref","first-page":"18568","DOI":"10.1109\/JIOT.2024.3364649","article-title":"Joint NTP-MAPPO and SDN for energy trading among multi-base-station microgrids","volume":"11","author":"M Yan","year":"2024","journal-title":"IEEE Internet Things J"},{"issue":"12","key":"pone.0346372.ref021","doi-asserted-by":"crossref","first-page":"10497","DOI":"10.1109\/JIOT.2023.3240173","article-title":"Cooperative UAV Resource Allocation and Task Offloading in Hierarchical Aerial Computing Systems: A MAPPO-Based Approach","volume":"10","author":"H Kang","year":"2023","journal-title":"IEEE Internet Things J"},{"issue":"4","key":"pone.0346372.ref022","doi-asserted-by":"crossref","first-page":"6037","DOI":"10.1109\/TII.2022.3192675","article-title":"Federated Imitation Learning for UAV Swarm Coordination in Urban Traffic Monitoring","volume":"19","author":"B Yang","year":"2023","journal-title":"IEEE Trans Ind Inf"},{"issue":"2","key":"pone.0346372.ref023","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/TBC.2024.3349773","article-title":"DMML: Deep Multi-Prior and Multi-Discriminator Learning for Underwater Image Enhancement","volume":"70","author":"A Esmaeilzehi","year":"2024","journal-title":"IEEE Trans Broadcast"},{"issue":"4","key":"pone.0346372.ref024","doi-asserted-by":"crossref","first-page":"11166","DOI":"10.1109\/LRA.2022.3196122","article-title":"Learning Category-Level Generalizable Object Manipulation Policy Via Generative Adversarial Self-Imitation Learning From Demonstrations","volume":"7","author":"H Shen","year":"2022","journal-title":"IEEE Robot Autom Lett"},{"issue":"8","key":"pone.0346372.ref025","doi-asserted-by":"crossref","first-page":"10119","DOI":"10.1109\/TVT.2023.3254138","article-title":"Intelligent resource allocation in backscatter-NOMA networks: A soft actor critic framework","volume":"72","author":"A Alajmi","year":"2023","journal-title":"IEEE Trans Veh Technol"},{"issue":"4","key":"pone.0346372.ref026","doi-asserted-by":"crossref","first-page":"5380","DOI":"10.1109\/JSYST.2021.3136586","article-title":"Hierarchical Microenergy Hub Sizing and Placement in Integrated Electricity and Natural Gas Distribution Systems","volume":"16","author":"SA Seifossadat","year":"2022","journal-title":"IEEE Syst J"},{"issue":"3","key":"pone.0346372.ref027","doi-asserted-by":"crossref","first-page":"3798","DOI":"10.1109\/JSYST.2023.3244498","article-title":"Reliability-Oriented Planning Framework for Smart Cities: From Interconnected Micro Energy Hubs to Macro Energy Hub Scale","volume":"17","author":"M Nozarian","year":"2023","journal-title":"IEEE Syst J"},{"issue":"1","key":"pone.0346372.ref028","doi-asserted-by":"crossref","first-page":"848","DOI":"10.1109\/TSG.2023.3270821","article-title":"Reliability Analysis of Cyber\u2013Physical Energy Hubs: A Monte Carlo Approach","volume":"15","author":"J Faraji","year":"2024","journal-title":"IEEE Trans Smart Grid"},{"issue":"4","key":"pone.0346372.ref029","doi-asserted-by":"crossref","first-page":"1186","DOI":"10.1109\/TPDS.2023.3240404","article-title":"Leveraging Deep Reinforcement Learning With Attention Mechanism for Virtual Network Function Placement and Routing","volume":"34","author":"N He","year":"2023","journal-title":"IEEE Trans Parallel Distrib Syst"},{"issue":"4","key":"pone.0346372.ref030","doi-asserted-by":"crossref","first-page":"4097","DOI":"10.1109\/TNSM.2023.3263373","article-title":"FairNet: A Measurement Framework for Traffic Discrimination Detection on the Internet","volume":"20","author":"VS Khandkar","year":"2023","journal-title":"IEEE Trans Netw Serv Manage"},{"issue":"2","key":"pone.0346372.ref031","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/MRA.2023.3262461","article-title":"Mastering the Complex Assembly Task With a Dual-Arm Robot: A Novel Reinforcement Learning Method","volume":"30","author":"D Jiang","year":"2023","journal-title":"IEEE Robot Automat Mag"},{"issue":"9","key":"pone.0346372.ref032","doi-asserted-by":"crossref","first-page":"2153","DOI":"10.1109\/TLA.2022.9878171","article-title":"Comparison of Reinforcement and Imitation Learning algorithms in autonomous sailboat Digital Twins","volume":"20","author":"R Picinini M\u00e9xas","year":"2022","journal-title":"IEEE Latin Am Trans"},{"issue":"2","key":"pone.0346372.ref033","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1109\/TEC.2023.3334035","article-title":"A Proximal Policy Optimization Based Control Framework for Flexible Battery Energy Storage System","volume":"39","author":"J Meng","year":"2024","journal-title":"IEEE Trans Energy Convers"},{"issue":"4","key":"pone.0346372.ref034","first-page":"1788","article-title":"A Two-Level Energy Management Strategy for Multi-Microgrid Systems With Interval Prediction and Reinforcement Learning","volume":"69","author":"L Xiong","year":"2022","journal-title":"IEEE Trans Circuits Syst I"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0346372","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T17:30:59Z","timestamp":1775669459000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0346372"}},"subtitle":[],"editor":[{"given":"Lei","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2026,4,8]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,4,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0346372","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,8]]}}}