{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T05:20:18Z","timestamp":1784092818063,"version":"3.55.0"},"reference-count":59,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2024,3,19]],"date-time":"2024-03-19T00:00:00Z","timestamp":1710806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"Ministry of Education","doi-asserted-by":"publisher","award":["2021R1I1A3047456"],"award-info":[{"award-number":["2021R1I1A3047456"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>Multi-objective power scheduling (MOPS) aims to address the simultaneous minimization of economic costs and different types of environmental emissions during electricity generation. Recognizing it as an NP-hard problem, this article proposes a novel multi-agent deep reinforcement learning (MADRL)-based optimization algorithm. Within a custom multi-agent simulation environment, representing power-generating units as collaborative types of reinforcement learning (RL) agents, the MOPS problem is decomposed into sequential Markov decision processes (MDPs). The MDPs are then utilized for training an MADRL model, which subsequently offers the optimal solution to the optimization problem. The practical viability of the proposed method is evaluated across several experimental test systems consisting of up to 100 units featuring bi-objective and tri-objective problems. The results demonstrate that the proposed MADRL algorithm has better performance compared to established methods, such as teaching learning-based optimization (TLBO), real coded grey wolf optimization (RCGWO), evolutionary algorithm based on decomposition (EAD), non-dominated sorting algorithm II (NSGA-II), and non-dominated sorting algorithm III (NSGA-III).<\/jats:p>","DOI":"10.3390\/systems12030106","type":"journal-article","created":{"date-parts":[[2024,3,20]],"date-time":"2024-03-20T05:04:12Z","timestamp":1710911052000},"page":"106","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Reinforcement Learning-Based Multi-Objective Optimization for Generation Scheduling in Power Systems"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9374-7538","authenticated-orcid":false,"given":"Awol Seid","family":"Ebrie","sequence":"first","affiliation":[{"name":"Major in Industrial Data Science & Engineering, Department of Industrial and Data Engineering, Pukyong National University, Busan 48513, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3502-5112","authenticated-orcid":false,"given":"Young Jin","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Systems Management and Engineering, Pukyong National University, Busan 48513, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,3,19]]},"reference":[{"key":"ref_1","unstructured":"Huang, Y., Pardalos, P., and Zheng, Q. (2017). Springer Briefs in Energy, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Goyal, S., Singh, J., Saraswat, A., Kanwar, N., Shrivastava, M., and Mahela, O. (2020, January 17\u201319). Economic Load Dispatch with Emission and Line Constraints using Biogeography Based Optimization Technique. Proceedings of the 2020 International Conference on Intelligent Engineering and Management (ICIEM), London, UK.","DOI":"10.1109\/ICIEM48762.2020.9160266"},{"key":"ref_3","first-page":"56","article-title":"State of art in combined economic and emission dispatch","volume":"25","author":"Rex","year":"2017","journal-title":"Middle-East J. Sci. Res."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Montero, L., Bello, A., and Reneses, J. (2022). Review on the Unit Commitment Problem: Approaches, Techniques, and Resolution Methods. Energies, 15.","DOI":"10.3390\/en15041296"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Qin, J., Yu, N., and Gao, Y. (2021, January 25\u201328). Solving Unit Commitment Problems with Multi-step Deep Reinforcement Learning. Proceedings of the 2021 IEEE International Conference on Communications, Control, and Computing Technologies for Smart Grids (SmartGridComm), Virtually.","DOI":"10.1109\/SmartGridComm51999.2021.9632339"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"de Oliveira, L., da Silva Junior, I., and Abritta, R. (2022). Search Space Reduction for the Thermal Unit Commitment Problem through a Relevance Matrix. Energies, 15.","DOI":"10.3390\/en15197153"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10479-018-2827-x","article-title":"On the complexity of the unit commitment problem","volume":"274","author":"Bendotti","year":"2019","journal-title":"Ann. Oper. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4244","DOI":"10.1016\/j.asoc.2013.07.006","article-title":"Modified shuffled frog leaping algorithm with genetic algorithm crossover for solving economic load dispatch problem with valve-point effect","volume":"13","author":"Roy","year":"2013","journal-title":"Appl. Soft Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.neunet.2017.05.004","article-title":"Collective neurodynamic optimization for economic emission dispatch problem considering valve point effect in microgrid","volume":"93","author":"Wang","year":"2017","journal-title":"Neural Netw."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1109\/59.260859","article-title":"Effects of ramp rate Limits on unit commitment and Economic Dispatch","volume":"8","author":"Wang","year":"1993","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1976092","DOI":"10.1080\/08839514.2021.1976092","article-title":"Solution of combined economic and emission dispatch problems of power systems without penalty","volume":"36","author":"Zaoui","year":"2022","journal-title":"Appl. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jasmin, E., Ahamed, T., and Remani, T. (2016, January 14\u201317). A function approximation approach to reinforcement learning for solving unit commitment problem with photo voltaic sources. Proceedings of the 2016 IEEE International Conference on Power Electronics, Drives and Energy Systems, Kerala, India.","DOI":"10.1109\/PEDES.2016.7914428"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Park, H. (2022). A Unit Commitment Model Considering Feasibility of Operating Reserves under Stochastic Optimization Framework. Energies, 15.","DOI":"10.3390\/en15176221"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"618","DOI":"10.1016\/j.energy.2019.03.117","article-title":"A mixed integer linear programming model for unit commitment of thermal plants with peak shaving operation aspect in regional power grid lack of flexible hydropower energy","volume":"175","author":"Feng","year":"2019","journal-title":"Energy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"857520","DOI":"10.3389\/fenrg.2022.857520","article-title":"A Solution Method for Many-Objective Security-Constrained Unit Commitment Considering Flexibility","volume":"10","author":"Lin","year":"2022","journal-title":"Front. Energy Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1016\/j.compeleceng.2017.07.023","article-title":"Meta-heuristic framework: Quantum inspired binary grey wolf optimizer for unit commitment problem","volume":"70","author":"Srikanth","year":"2018","journal-title":"Comput. Electr. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.knosys.2017.07.013","article-title":"A novel parallel-series hybrid meta-heuristic method for solving a hybrid unit commitment problem","volume":"134","author":"Yang","year":"2017","journal-title":"Knowl. Based Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"12267","DOI":"10.1109\/ACCESS.2021.3051175","article-title":"Short-Term Operational Scheduling of Unit Commitment Using Binary Alternative Moth-Flame Optimization","volume":"9","author":"Kigsirisin","year":"2021","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1016\/j.swevo.2015.04.001","article-title":"Hybridizing genetic algorithm with differential evolution for solving the unit commitment scheduling problem","volume":"23","author":"Trivedi","year":"2015","journal-title":"Swarm Evol. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"121829","DOI":"10.1016\/j.energy.2021.121829","article-title":"A parallel meta-heuristic method for solving large scale unit commitment considering the integration of new energy sectors","volume":"238","author":"Zhu","year":"2022","journal-title":"Energy"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2845","DOI":"10.1016\/j.asoc.2010.11.014","article-title":"Economic environmental dispatch using multi-objective differential evolution","volume":"11","author":"Basu","year":"2011","journal-title":"Appl. Soft Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4005","DOI":"10.1109\/TPWRS.2020.2986710","article-title":"An improved genetic algorithm approach to the unit commitment\/economic dispatch problem","volume":"35","author":"Ponciroli","year":"2020","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2","DOI":"10.17485\/ijst\/2016\/v9i2\/80998","article-title":"Best Compromised Schedule for Multi-Objective Unit Commitment Problems","volume":"9","author":"Balasubramanian","year":"2016","journal-title":"Indian J. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1016\/j.ijepes.2014.02.008","article-title":"Solution of unit commitment problem using quasi-oppositional teaching learning based algorithm","volume":"60","author":"Roy","year":"2014","journal-title":"Electr. Power Energy Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3873","DOI":"10.1016\/j.asoc.2013.05.002","article-title":"Unit commitment problem with ramp rate constraint using a binary-real-coded genetic algorithm","volume":"13","author":"Datta","year":"2013","journal-title":"Appl. Soft Comput."},{"key":"ref_26","unstructured":"Reddy, S., Kumar, R., and Panigrahi, B. (2017, January 14\u201316). Binary Bat Search Algorithm for Unit Commitment Problem in Power system. Proceedings of the IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE), Dehradun, India."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Khunkitti, S., Watson, N., Chatthaworn, R., Premrudeepreechacharn, S., and Siritaratiwat, A. (2019). An Improved DA-PSO Optimization Approach for Unit Commitment Problem. Energies, 12.","DOI":"10.3390\/en12122335"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.swevo.2017.08.002","article-title":"Binary grey wolf optimizer for large scale unit commitment problem","volume":"38","author":"Panwar","year":"2018","journal-title":"Swarm Evol. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"4146","DOI":"10.1109\/TCYB.2019.2921475","article-title":"Distributed Q-learning-based online optimization algorithm for unit commitment and dispatch in smart grid","volume":"50","author":"Li","year":"2020","journal-title":"IEEE Trans. Cybern."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1016\/j.jocs.2017.04.011","article-title":"Solution to unit commitment in power system operation planning using binary coded modified moth flame optimization algorithm (BMMFOA): A flame selection based computational technique","volume":"25","author":"Reddy","year":"2018","journal-title":"J. Comput. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nassef, A.A.M., Maghrabie, H., and Baroutaji, A. (2023). Review of Metaheuristic Optimization Algorithms for Power. Sustainability, 15.","DOI":"10.3390\/su15129434"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2095","DOI":"10.1007\/s00521-018-3796-3","article-title":"Binary whale optimization algorithm and its application to unit commitment problem","volume":"32","author":"Kumar","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"9161","DOI":"10.1007\/s00500-019-04445-x","article-title":"A two-layer algorithm based on PSO for solving unit commitment problem","volume":"24","author":"Zhai","year":"2020","journal-title":"Soft Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"972","DOI":"10.1049\/iet-gtd.2015.0726","article-title":"Cost, emission and reserve pondered predispatch of thermal power generating units coordinated with real coded grey wolf optimization","volume":"10","author":"Rameshkumar","year":"2016","journal-title":"IET Gener. Transm. Distrib."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1016\/j.applthermaleng.2018.10.020","article-title":"Multiobjective optimization of the environmental-economic dispatch with reinforcement learning based on non-dominated sorting genetic algorithm","volume":"146","author":"Bora","year":"2019","journal-title":"Appl. Therm. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"207768","DOI":"10.1109\/ACCESS.2020.3022245","article-title":"Low Carbon Multi-Objective Unit Commitment Integrating Renewable Generations","volume":"8","author":"Yang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1346","DOI":"10.1109\/TII.2015.2485520","article-title":"Enhanced Multiobjective Evolutionary Algorithm based on Decomposition for Solving the Unit Commitment Problem","volume":"11","author":"Trivedi","year":"2009","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2266","DOI":"10.1109\/TPWRS.2015.2463725","article-title":"Two-Stage Multi-Objective Unit Commitment Optimization Under Hybrid Uncertainties","volume":"31","author":"Wang","year":"2015","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.energy.2016.05.029","article-title":"Multi-objective unit commitment with wind penetration and emission concerns under stochastic and fuzzy uncertainties","volume":"111","author":"Wang","year":"2016","journal-title":"Energy"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"117519","DOI":"10.1016\/j.apenergy.2021.117519","article-title":"Applying reinforcement learning and tree search to the unit commitment problem","volume":"302","year":"2021","journal-title":"Appl. Energy"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.swevo.2012.06.001","article-title":"Biogeography based optimization technique for best compromise solution of economic emission dispatch","volume":"7","author":"Rajasomashekar","year":"2012","journal-title":"Swarm Evol. Comput."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Ebrie, A., Paik, C., Chung, Y., and Kim, Y. (2023). Environment-Friendly Power Scheduling Based on Deep Contextual Reinforcement Learning. Energies, 16.","DOI":"10.3390\/en16165920"},{"key":"ref_43","unstructured":"Jasmin, E., Ahamed, T., and Jagathy, R. (2009, January 28\u201329). Reinforcement learning solution for unit commitment problem through pursuit method. Proceedings of the 2009 International Conference on Advances in Computing, Control, and Telecommunication Technologies, Kerala, India."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.procs.2015.02.012","article-title":"Reinforcement Learning in Adaptive Control of Power System Generation","volume":"46","author":"Rajua","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1007\/s00521-017-3106-5","article-title":"A fuzzy reinforcement learning approach to thermal unit commitment problem","volume":"31","author":"Navin","year":"2019","journal-title":"Neural Comput. Appl."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"100179","DOI":"10.1016\/j.egyai.2022.100179","article-title":"Reinforcement learning and A* search for the unit commitment problem","volume":"9","year":"2022","journal-title":"Energy AI"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Dalal, G., and Mannor, S. (July, January 29). Reinforcement learning for the unit commitment problem. Proceedings of the 2015 IEEE Eindhoven PowerTech, Eindhoven, The Netherlands.","DOI":"10.1109\/PTC.2015.7232646"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1006160","DOI":"10.1016\/j.simpa.2024.100616","article-title":"pymops: A multi-agent simulation-based optimization package for power scheduling","volume":"19","author":"Ebrie","year":"2024","journal-title":"Softw. Impacts"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1109\/TPWRS.2003.820707","article-title":"Unit commitment by enhanced adaptive Lagrangian relaxation","volume":"19","author":"Ongsakul","year":"2004","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1325","DOI":"10.1109\/59.260861","article-title":"Genetic algorithm solution of economic dispatch with valve point loading","volume":"PAS 97","author":"Walters","year":"1993","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1801","DOI":"10.1049\/iet-rpg.2019.1081","article-title":"Chaotic sine\u2013cosine algorithm for chance-constrained economic emission dispatch problem including wind energy","volume":"14","author":"Guesmi","year":"2020","journal-title":"IET Renew. Power Gener."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"117411","DOI":"10.1016\/j.eswa.2022.117411","article-title":"A hybrid dynamic economic environmental dispatch model for balancing operating costs and pollutant emissions in renewable energy: A novel improved mayfly algorithm","volume":"203","author":"Li","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/j.ref.2022.08.008","article-title":"Application of NSGA-II for environmental constraint economic dispatch of thermal-wind-solar power system","volume":"43","author":"Dey","year":"2022","journal-title":"Renew. Energy Focus"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"2660","DOI":"10.1109\/TPWRS.2013.2241795","article-title":"A memetic evolutionary multiobjective optimization method for environmental power unit commitment","volume":"28","author":"Li","year":"2013","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_55","unstructured":"Sutton, R., and Barto, A. (2018). Reinforcement Learning, An Introduction, The MIT Press."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nat. Methods"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1109\/TSMCC.2011.2106494","article-title":"Experience Replay for Real-Time Reinforcement Learning Control","volume":"42","author":"Adam","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1109\/TPWRS.2002.1007911","article-title":"A Hybrid EP and SQP for Dynamic Economic Dispatch with Nonsmooth Fuel Cost Function","volume":"17","author":"Attaviriyanupap","year":"2002","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_59","unstructured":"Truby, J. (2014). Thermal Power Plant Economics and Variable Renewable Energies: A Model-Based Case Study for Germany, International Energy Agency."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/3\/106\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:16:08Z","timestamp":1760105768000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/12\/3\/106"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,19]]},"references-count":59,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["systems12030106"],"URL":"https:\/\/doi.org\/10.3390\/systems12030106","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,19]]}}}