{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T03:26:09Z","timestamp":1785381969498,"version":"3.55.0"},"reference-count":54,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2020,5,27]],"date-time":"2020-05-27T00:00:00Z","timestamp":1590537600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004663","name":"Ministry of Science and Technology, Taiwan","doi-asserted-by":"publisher","award":["MOST 108-2622-E-309-001-CC1"],"award-info":[{"award-number":["MOST 108-2622-E-309-001-CC1"]}],"id":[{"id":"10.13039\/501100004663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>On the issues of global environment protection, the renewable energy systems have been widely considered. The photovoltaic (PV) system converts solar power into electricity and significantly reduces the consumption of fossil fuels from environment pollution. Besides introducing new materials for the solar cells to improve the energy conversion efficiency, the maximum power point tracking (MPPT) algorithms have been developed to ensure the efficient operation of PV systems at the maximum power point (MPP) under various weather conditions. The integration of reinforcement learning and deep learning, named deep reinforcement learning (DRL), is proposed in this paper as a future tool to deal with the optimization control problems. Following the success of deep reinforcement learning (DRL) in several fields, the deep Q network (DQN) and deep deterministic policy gradient (DDPG) are proposed to harvest the MPP in PV systems, especially under a partial shading condition (PSC). Different from the reinforcement learning (RL)-based method, which is only operated with discrete state and action spaces, the methods adopted in this paper are used to deal with continuous state spaces. In this study, DQN solves the problem with discrete action spaces, while DDPG handles the continuous action spaces. The proposed methods are simulated in MATLAB\/Simulink for feasibility analysis. Further tests under various input conditions with comparisons to the classical Perturb and observe (P&amp;O) MPPT method are carried out for validation. Based on the simulation results in this study, the performance of the proposed methods is outstanding and efficient, showing its potential for further applications.<\/jats:p>","DOI":"10.3390\/s20113039","type":"journal-article","created":{"date-parts":[[2020,5,28]],"date-time":"2020-05-28T12:36:58Z","timestamp":1590669418000},"page":"3039","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":76,"title":["A Deep Reinforcement Learning-Based MPPT Control for PV Systems under Partial Shading Condition"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9625-9257","authenticated-orcid":false,"given":"Bao Chau","family":"Phan","sequence":"first","affiliation":[{"name":"Department of Aeronautics and Aeronautics, National Cheng Kung University, Tainan 701, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3471-3290","authenticated-orcid":false,"given":"Ying-Chih","family":"Lai","sequence":"additional","affiliation":[{"name":"Department of Aeronautics and Aeronautics, National Cheng Kung University, Tainan 701, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chin E.","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Aeronautics and Aeronautics, National Cheng Kung University, Tainan 701, Taiwan"},{"name":"UAV Center, Chang Jung Christian University, Tainan 701, Taiwan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Lin, C.E., and Phan, B.C. (2016, January 8\u201310). Optimal Hybrid Energy Solution for Island Micro-Grid. Proceedings of the 2016 IEEE International Conferences on Big Data and Cloud Computing (BDCloud), Social Computing and Networking (SocialCom), Sustainable Computing and Communications (SustainCom) (BDCloud-SocialCom-SustainCom), Atlanta, GA, USA.","DOI":"10.1109\/BDCloud-SocialCom-SustainCom.2016.74"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.rser.2018.04.094","article-title":"A review of global maximum power point tracking techniques of photovoltaic system under partial shading conditions","volume":"92","author":"Belhachat","year":"2018","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.rser.2016.09.013","article-title":"A review on maximum power point tracking for photovoltaic systems with and without shading conditions","volume":"67","author":"Ramli","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.rser.2017.02.051","article-title":"A comparison of different global MPPT techniques based on meta-heuristic algorithms for photovoltaic system subjected to partial shading conditions","volume":"74","author":"Rezk","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2743","DOI":"10.1016\/j.rser.2017.10.009","article-title":"Comparative and comprehensive review of maximum power point tracking methods for PV cells","volume":"82","author":"Danandeh","year":"2018","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.rser.2016.09.132","article-title":"General review and classification of different MPPT Techniques","volume":"68","author":"Karami","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"854","DOI":"10.1016\/j.rser.2017.05.083","article-title":"A review on MPPT techniques of PV system under partial shading condition","volume":"80","author":"Mohapatra","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/j.apenergy.2015.04.006","article-title":"An improved perturb and observe (P&O) maximum power point tracking (MPPT) algorithm for higher efficiency","volume":"150","author":"Ahmed","year":"2015","journal-title":"Appl. Energy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"14158","DOI":"10.1016\/j.ijhydene.2018.06.002","article-title":"A novel maximum power point tracking technique based on fuzzy logic for photovoltaic systems","volume":"43","author":"Abbod","year":"2018","journal-title":"Int. J. Hydrogen Energy"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1016\/j.ijepes.2012.04.047","article-title":"MPPT control design and performance improvements of a PV generator powered DC motor-pump system based on artificial neural networks","volume":"43","author":"Kassem","year":"2012","journal-title":"Int. J. Electr. Power Energy Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"875","DOI":"10.1016\/j.rser.2017.02.056","article-title":"Global maximum power point tracking based on ANFIS approach for PV array configurations under partial shading conditions","volume":"77","author":"Belhachat","year":"2017","journal-title":"Renew. Sustain. Energy Rev."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mumtaz, S., Ahmad, S., Khan, L., Ali, S., Kamal, T., and Hassan, S. (2018). Adaptive Feedback Linearization Based NeuroFuzzy Maximum Power Point Tracking for a Photovoltaic System. Energies, 11.","DOI":"10.3390\/en11030606"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.solener.2013.01.005","article-title":"Comparison between conventional methods and GA approach for maximum power point tracking of shaded solar PV generators","volume":"90","author":"Shaiek","year":"2013","journal-title":"Sol. Energy"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.apenergy.2013.12.062","article-title":"A Maximum Power Point Tracking (MPPT) for PV system using Cuckoo Search with partial shading capability","volume":"119","author":"Ahmed","year":"2014","journal-title":"Appl. Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1016\/j.asoc.2017.05.017","article-title":"A new MPPT controller based on the Ant colony optimization algorithm for Photovoltaic systems under partial shading conditions","volume":"58","author":"Titri","year":"2017","journal-title":"Appl. Soft Comput."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.asoc.2015.03.047","article-title":"Artificial bee colony based algorithm for maximum power point tracking (MPPT) for PV systems operating under partial shaded conditions","volume":"32","author":"Benyoucef","year":"2015","journal-title":"Appl. Soft Comput."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"490","DOI":"10.1016\/j.solener.2017.09.063","article-title":"Bat algorithm based maximum power point tracking for photovoltaic system under partial shading conditions","volume":"158","author":"Kaced","year":"2017","journal-title":"Sol. Energy"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.1016\/j.jclepro.2019.01.150","article-title":"Novel bio-inspired memetic salp swarm algorithm and application to MPPT for PV systems considering partial shading condition","volume":"215","author":"Yang","year":"2019","journal-title":"J. Cleaner Prod."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.rser.2018.01.006","article-title":"Computational intelligence techniques for maximum power point tracking in PV systems: A review","volume":"85","author":"Jiang","year":"2018","journal-title":"Renewable Sustainable Energy Rev."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1109\/TSTE.2016.2606421","article-title":"A Novel MPPT Algorithm Based on Particle Swarm Optimization for Photovoltaic Systems","volume":"8","author":"Koad","year":"2017","journal-title":"IEEE Trans. Sustain. Energy"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Suryavanshi, R., Joshi, D.R., and Jangamshetti, S.H. (2012, January 3\u20136). PSO and P&O based MPPT technique for SPV panel under varying atmospheric conditions. Proceedings of the 2012 International Conference on Power, Signals, Controls and Computation, Thrissur, Kerala, India.","DOI":"10.1109\/EPSCICON.2012.6175270"},{"key":"ref_22","first-page":"292","article-title":"A hybrid PSO-GA algorithm for constrained optimization problems","volume":"274","author":"Garg","year":"2016","journal-title":"Appl. Math. Comput."},{"key":"ref_23","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press. [2nd ed.]."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.arcontrol.2019.09.008","article-title":"(Deep) Reinforcement learning for electric power system control and related problems: A short review and perspectives","volume":"48","author":"Glavic","year":"2019","journal-title":"Annu. Rev. Control"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1016\/j.renene.2017.03.008","article-title":"A reinforcement learning approach for MPPT control method of photovoltaic sources","volume":"108","author":"Kofinas","year":"2017","journal-title":"Renew. Energy"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6360","DOI":"10.1109\/TIE.2015.2420792","article-title":"Reinforcement-Learning-Based Intelligent Maximum Power Point Tracking Control for Wind Energy Conversion Systems","volume":"62","author":"Wei","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_27","unstructured":"Nambiar, A., Anderlini, E., Payne, G., Forehand, D., Kiprakis, A., and Wallace, A. (2017, January 27). Reinforcement Learning Based Maximum Power Point Tracking Control of Tidal Turbines. Proceedings of the 12th European Wave and Tidal Energy Conference, Cork, Ireland."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Hsu, R., Liu, C.T., Chen, W.Y., Hsieh, H.-I., and Wang, H.L. (2015). A Reinforcement Learning-Based Maximum Power Point Tracking Method for Photovoltaic Array. Int. J. Photoenergy, 2015.","DOI":"10.1155\/2015\/496401"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"245","DOI":"10.7763\/JOCET.2016.V4.290","article-title":"Reinforcement Learning for Online Maximum Power Point Tracking Control","volume":"4","author":"Youssef","year":"2016","journal-title":"J. Clean Energy Technol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Phan, B.C., and Lai, Y.-C. (2019). Control Strategy of a Hybrid Renewable Energy System Based on Reinforcement Learning Approach for an Isolated Microgrid. Appl. Sci., 9.","DOI":"10.3390\/app9194001"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chou, K.-Y., Yang, S.-T., and Chen, Y.-P. (2019). Maximum Power Point Tracking of Photovoltaic System Based on Reinforcement Learning. Sensors, 19.","DOI":"10.3390\/s19225054"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1016\/j.energy.2019.03.053","article-title":"Memetic reinforcement learning based maximum power point tracking design for PV systems under partial shading condition","volume":"174","author":"Zhang","year":"2019","journal-title":"Energy"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dong, M., Li, D., Yang, C., Li, S., Fang, Q., Yang, B., and Zhang, X. (2019). Global Maximum Power Point Tracking of PV Systems under Partial Shading Condition: A Transfer Reinforcement Learning Approach. Appl. Sci., 9.","DOI":"10.3390\/app9132769"},{"key":"ref_34","unstructured":"Lapan, M. (2018). Deep Reinforcement Learning Hands-On: Apply Modern RL Methods, with Deep Q-Networks, Value Iteration, Policy Gradients, TRPO, AlphaGo Zero and More, Packt Publishing Ltd."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gu, S., Holly, E., Lillicrap, T., and Levine, S. (June, January 29). Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989385"},{"key":"ref_36","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv Preprint."},{"key":"ref_37","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, L., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv Preprint."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kahn, G., A Villaflor, B.D., Abbeel, P., and Levine, S. (2018, January 20\u201325). Self-Supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8460655"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, J., Chen, J., He, X., Gao, J., Li, L., Deng, L., and Ostendorf, M. (2015). Deep reinforcement learning with a natural language action space. arXiv Preprint.","DOI":"10.18653\/v1\/P16-1153"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1007\/s10916-018-1045-z","article-title":"Maintaining Security and Privacy in Health Care System Using Learning Based Deep-Q-Networks","volume":"42","author":"Baskar","year":"2018","journal-title":"J. Med. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"362","DOI":"10.17775\/CSEEJPES.2018.00520","article-title":"Review on the research and practice of deep learning and reinforcement learning in smart grids","volume":"4","author":"Zhang","year":"2018","journal-title":"CSEE J. Power Energy Syst."},{"key":"ref_42","unstructured":"Zhang, Z., Zhang, D., and Qiu, R.C. (2019). Deep reinforcement learning for power system: An. overview. CSEE J. Power Energy Syst., 1\u201312."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"7837","DOI":"10.1109\/TPEL.2016.2514370","article-title":"An Adaptive Network-Based Reinforcement Learning Method for MPPT Control of PMSG Wind Energy Conversion Systems","volume":"31","author":"Wei","year":"2016","journal-title":"IEEE Trans. Power Electron."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Saenz-Aguirre, A., Zulueta, E., Fernandez-Gamiz, U., Lozano, J., and Lopez-Guede, J. (2019). Artificial Neural Network Based Reinforcement Learning for Wind Turbine Yaw Control. Energies, 12.","DOI":"10.3390\/en12030436"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1016\/j.rser.2016.09.076","article-title":"A comprehensive review on solar PV maximum power point tracking techniques","volume":"67","author":"Ram","year":"2017","journal-title":"Renewable Sustainable Energy Rev."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1016\/j.rser.2015.02.009","article-title":"A survey of the most used MPPT methods: Conventional and advanced algorithms applied for photovoltaic systems","volume":"45","author":"Bendib","year":"2015","journal-title":"Renewable Sustainable Energy Rev."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"628","DOI":"10.1016\/j.solener.2019.04.034","article-title":"Novel MPPT techniques for photovoltaic systems under uniform irradiance and Partial shading","volume":"184","author":"Mirza","year":"2019","journal-title":"Sol. Energy"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/j.energy.2016.10.084","article-title":"A new global maximum power point tracking technique for solar photovoltaic (PV) system under partial shading conditions (PSC)","volume":"118","author":"Rajasekar","year":"2017","journal-title":"Energy"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_50","unstructured":"Casas, N. (2017). Deep deterministic policy gradient for urban traffic light control. arXiv Preprint."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Li, Y. (2018). Deep Reinforcement Learning: An Overview. arXiv Preprint.","DOI":"10.1201\/9781351006620-1"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1016\/j.apenergy.2018.03.104","article-title":"Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus","volume":"222","author":"Wu","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_53","unstructured":"Fan, J., Wang, Z., Xie, Y., and Yang, Z. (2019). A Theoretical Analysis of Deep Q-Learning. arXiv Preprint."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"454","DOI":"10.1016\/j.apenergy.2019.04.021","article-title":"Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus","volume":"247","author":"Wu","year":"2019","journal-title":"Appl. Energy"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/11\/3039\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:33:04Z","timestamp":1760175184000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/11\/3039"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,27]]},"references-count":54,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["s20113039"],"URL":"https:\/\/doi.org\/10.3390\/s20113039","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,27]]}}}