{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T11:11:14Z","timestamp":1767611474060,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,4,29]],"date-time":"2022-04-29T00:00:00Z","timestamp":1651190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62173032","BK20AF005"],"award-info":[{"award-number":["62173032","BK20AF005"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Foshan Science and Technology Innovation Program","award":["62173032","BK20AF005"],"award-info":[{"award-number":["62173032","BK20AF005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Energy storage is an important adjustment method to improve the economy and reliability of a power system. Due to the complexity of the coupling relationship of elements such as the power source, load, and energy storage in the microgrid, there are problems of insufficient performance in terms of economic operation and efficient dispatching. In view of this, this paper proposes an energy storage configuration optimization model based on reinforcement learning and battery state of health assessment. Firstly, a quantitative assessment of battery health life loss based on deep learning was performed. Secondly, on the basis of considering comprehensive energy complementarity, a two-layer optimal configuration model was designed to optimize the capacity configuration and dispatch operation. Finally, the feasibility of the proposed method in microgrid energy storage planning and operation was verified by experimentation. By integrating reinforcement learning and traditional optimization methods, the proposed method did not rely on the accurate prediction of the power supply and load and can make decisions based only on the real-time information of the microgrid. In this paper, the advantages and disadvantages of the proposed method and existing methods were analyzed, and the results show that the proposed method can effectively improve the performance of dynamic planning for energy storage in microgrids.<\/jats:p>","DOI":"10.3390\/e24050630","type":"journal-article","created":{"date-parts":[[2022,4,29]],"date-time":"2022-04-29T16:49:57Z","timestamp":1651250997000},"page":"630","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Optimal Energy-Storage Configuration for Microgrids Based on SOH Estimation and Deep Q-Network"],"prefix":"10.3390","volume":"24","author":[{"given":"Shuai","family":"Chen","sequence":"first","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinglin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengpeng","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2270-7889","authenticated-orcid":false,"given":"Wendong","family":"Xiao","sequence":"additional","affiliation":[{"name":"School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"},{"name":"Key Laboratory of Knowledge Automation for Industrial Processes of Ministry of Education, School of Automation and Electrical Engineering, University of Science and Technology Beijing, Beijing 100083, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ullah, K., Jiang, Q., Geng, G., Rahim, S., and Khan, R.A. (2022). Optimal Power Sharing in Microgrids using the Artificial Bee Colony Algorithm. Energies, 15.","DOI":"10.3390\/en15031067"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Abou El-Ela, A.A., El-Sehiemy, R.A., Allam, S.M., Shaheen, A.M., Nagem, N.A., and Sharaf, A.M. (2022). Renewable Energy Micro-grid Interfacing: Economic and Environmental Issues. Electronics, 11.","DOI":"10.3390\/electronics11050815"},{"key":"ref_3","first-page":"171","article-title":"Energy Management of Microgrid based on Generalized Stackelberg Game","volume":"42","author":"Chen","year":"2022","journal-title":"Electr. Power Autom. Equip."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Gallego, D., Gonzalez-Ayala, J., Calvo Hern\u00e1ndez, A., and Medina, A. (2021). Thermodynamic Performance of a Brayton Pumped Heat Energy Storage System: Influence of Internal and External Irreversibilities. Entropy, 23.","DOI":"10.3390\/e23121564"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1360\/N972017-01213","article-title":"Experimental Research on GEIS of C\/LiFePO4 Battery","volume":"63","author":"Wang","year":"2018","journal-title":"Chin. Sci. Bull."},{"key":"ref_6","first-page":"6815","article-title":"Hierarchical Optimization Method for Electrical Energy Storage and Heat Storage Capacity Planning in Multi-energy Complementary Generation System","volume":"40","author":"Shi","year":"2020","journal-title":"Power Syst. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1016\/j.est.2018.12.002","article-title":"A Two-stage Stochastic Model for Energy Storage Planning in A Microgrid Incorporating Bilateral Contracts and Demand Response Program","volume":"21","author":"Nazari","year":"2019","journal-title":"J. Energy Storage"},{"key":"ref_8","first-page":"123","article-title":"Optimal Capacity Configuration of Wind\/PV\/Storage Hybrid Energy Storage System in Microgrid","volume":"32","author":"Li","year":"2020","journal-title":"Proc. CSU-EP-SA"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"100970","DOI":"10.1016\/j.est.2019.100970","article-title":"Two-stage Hybrid Stochastic\/Robust Optimal Coordination of Distributed Battery Storage Planning and Flexible Energy Management in Smart Distribution Network","volume":"26","author":"Bozorgavari","year":"2019","journal-title":"J. Energy Storage"},{"key":"ref_10","first-page":"45","article-title":"Configuration Method of Energy Storage for Wind Farms Considering Wind Power Uncertainty and Wind Curtailment Constraint","volume":"44","author":"Yang","year":"2020","journal-title":"Autom. Electr. Power Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"116646","DOI":"10.1016\/j.apenergy.2021.116646","article-title":"A Techno-economic Assessment on the Adoption of Latent Heat Thermal Energy Storage Systems for District Cooling Optimal Dispatch & Operations","volume":"289","author":"Mazzoni","year":"2021","journal-title":"Appl. Energy"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1016\/j.renene.2020.10.055","article-title":"Solar Power-to-gas Application to An Island Energy System","volume":"164","author":"Nastasi","year":"2021","journal-title":"Renew. Energy"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"718310","DOI":"10.3389\/fenrg.2021.718310","article-title":"The Role of Biowaste: A Multi-Objective Optimization Platform for Combined Heat, Power and Fuel","volume":"9","author":"Morisod","year":"2021","journal-title":"Front. Energy Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"117954","DOI":"10.1016\/j.apenergy.2021.117954","article-title":"Overcoming Decision Paralysis-A Digital Twin for Decision Making in Energy System Design","volume":"306","author":"Granacher","year":"2022","journal-title":"Appl. Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1016\/j.apenergy.2018.09.185","article-title":"Battery Aging in Multi-energy Microgrid Design using Mixed Integer Linear Programming","volume":"231","author":"Cardoso","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_16","first-page":"3627","article-title":"A Real-time Rain Flow Algorithm and Its Application to State of Health Modeling for LiCoO2 Lithium-ion Batteries","volume":"37","author":"Bin","year":"2017","journal-title":"Proc. CSEE"},{"key":"ref_17","first-page":"206","article-title":"On-line Diagnosis Model of SOH based on Thermal Characteristics of Lithium-ion Battery","volume":"41","author":"Shi","year":"2020","journal-title":"Chin. J. Sci. Instrum."},{"key":"ref_18","first-page":"63","article-title":"State of Health Estimation of Lead-acid Battery based on Deep Learning","volume":"51","author":"Hu","year":"2021","journal-title":"Battery Bimonthly"},{"key":"ref_19","first-page":"187","article-title":"A Coupled State Estimation Method of Lithium Batteries Based on Partial Charging Voltage Segment","volume":"48","author":"Wang","year":"2021","journal-title":"J. Hunan Univ. Nat. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1016\/j.eng.2019.11.015","article-title":"A Grid as Smart as the Internet","volume":"6","author":"Liu","year":"2020","journal-title":"Engineering"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"166502","DOI":"10.1109\/ACCESS.2021.3135083","article-title":"A Review on Microgrids\u2019 Challenges & Perspectives","volume":"9","author":"Muhammad","year":"2021","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bole, B., Kulkarni, C., and Daigle, M. (October, January 29). Adaptation of An Electrochemistry-based Li-ion Battery Model to Account for Deterioration Observed under Randomized Use. Proceedings of the 2014 Conference of the Prognostics Health Management Society, Fort Worth, TX, USA.","DOI":"10.36001\/phmconf.2014.v6i1.2490"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1109\/TEC.2006.874229","article-title":"Accurate Electrical Battery Model Capable of Predicting Runtime and I-V Performance","volume":"21","author":"Chen","year":"2006","journal-title":"IEEE Trans. Energy Convers."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"678","DOI":"10.1016\/j.energy.2015.05.148","article-title":"Performance Analysis and SOH (State of Health) Evaluation of Lithium Polymer Batteries Through Electrochemical Impedance Spectroscopy","volume":"89","author":"Galeotti","year":"2015","journal-title":"Energy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1016\/j.jpowsour.2018.06.036","article-title":"A Fast Estimation Algorithm for Lithium-ion Battery State of Health","volume":"396","author":"Tang","year":"2018","journal-title":"J. Power Sources"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1109\/TTE.2017.2776558","article-title":"State of Health Estimation of Lithium-ion Batteries using Capacity Fade and Internal Resistance Growth Models","volume":"4","author":"Guha","year":"2018","journal-title":"IEEE Trans. Transp. Electrif."},{"key":"ref_27","first-page":"4016","article-title":"An Approach to Lithium-Ion Battery SOH Estimation Based on Convolutional Neural Network","volume":"35","author":"Li","year":"2020","journal-title":"Trans. China Electrotech. Soc."},{"key":"ref_28","first-page":"5661","article-title":"State of Health Estimation of Lithium-ion Batteries Based on the Partial Charging Voltage Segment and Kernel Ridge Regression","volume":"41","author":"Fan","year":"2021","journal-title":"Proc. CSEE"},{"key":"ref_29","unstructured":"Howard, A., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). Mobilenets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L. (2018, January 18\u201323). Mobilenetv2: Inverted Residuals and Linear Bottlenecks. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (November, January 27). Searching for Mobilenetv3. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00140"},{"key":"ref_32","unstructured":"Kingma, D., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"6047","DOI":"10.1109\/TSG.2017.2703126","article-title":"A Two-layer Energy Management System for Microgrids with Hybrid Energy Storage Considering Degradation Costs","volume":"9","author":"Ju","year":"2018","journal-title":"IEEE Trans. Smart Grid."},{"key":"ref_34","first-page":"66","article-title":"Optimal Configuration of Electric\/Thermal Hybrid Energy Storage for Park-level Integrated Energy System Considering Battery Life Loss","volume":"45","author":"Guo","year":"2021","journal-title":"Autom. Electr. Power Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Li, J., and Zhao, H. (2021). Multi-Objective Optimization and Performance Assessments of an Integrated Energy System Based on Fuel, Wind and Solar Energies. Entropy, 23.","DOI":"10.3390\/e23040431"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1049\/iet-gtd.2016.0772","article-title":"Economic Scheduling Model of Microgrid Considering the Lifetime of Batteries","volume":"11","author":"Liu","year":"2017","journal-title":"IET Gener. Transm. Distrib."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hao, L., Wu, J., Li, B., Xu, J., Liu, C., Gan, P., Wang, Z., and Zhang, Y. (2019, January 7\u20139). A Multi-Objective Optimization Planning of Park-level Integrated Energy System Based on Two-tier Planning Mode. Proceedings of the 2019 IEEE 3rd International Electrical and Energy Conference, Beijing, China.","DOI":"10.1109\/CIEEC47146.2019.CIEEC-2019330"},{"key":"ref_38","first-page":"128","article-title":"Optimal Energy-storage Configuration of Multi-energy Microgrid Considering Battery Life Degradation","volume":"33","author":"Sun","year":"2021","journal-title":"Proc. CSU-EPSA"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chen, S., Jiang, C., Li, J., Xiang, J., and Xiao, W. (2021). Improved Deep Q-Network for User-Side Battery Energy Storage Charging and Discharging Strategy in Industrial Parks. Entropy, 23.","DOI":"10.3390\/e23101311"},{"key":"ref_40","first-page":"1005","article-title":"Optimized Allocation of Various Energy Storage Capacities in A Multi-energy Micro-grid Considering Electrical\/Thermal\/Gas Coupling Demand Response","volume":"48","author":"Wu","year":"2020","journal-title":"Power Syst. Prot. Control."},{"key":"ref_41","unstructured":"(2022, March 13). Beijing Sales Electricity Price Table, Available online: http:\/\/www.beijing.gov.cn\/fuwu\/bmfw\/jmsh\/jmshshjf\/shjfd\/dj\/201709\/t20170914_1844488.html."},{"key":"ref_42","unstructured":"Sun, J. (2016). Energy Storage Capacity Optimization for Micro Grid Including Electricity and Heat. [Ph.D. Thesis, Tianjin University]."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/630\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:04:30Z","timestamp":1760137470000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/630"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,29]]},"references-count":42,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["e24050630"],"URL":"https:\/\/doi.org\/10.3390\/e24050630","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2022,4,29]]}}}