{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T23:43:04Z","timestamp":1778542984459,"version":"3.51.4"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T00:00:00Z","timestamp":1645488000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T00:00:00Z","timestamp":1645488000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"UKRI Future Leaders Fellowship","award":["MR\/S017062\/1"],"award-info":[{"award-number":["MR\/S017062\/1"]}]},{"name":"Royal Society International Exchange Program","award":["IES R2 212077"],"award-info":[{"award-number":["IES R2 212077"]}]},{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","award":["2TF1 100422"],"award-info":[{"award-number":["2TF1 100422"]}],"id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Amazon Research Award"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Memetic Comp."],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Grid-connected microgrids comprising renewable energy, energy storage systems and local load, play a vital role in decreasing the energy consumption of fossil diesel and greenhouse gas emissions. A distribution power network connecting several microgrids can promote more potent and reliable operations to enhance the security and privacy of the power system. However, the operation control for a multi-microgrid system is a big challenge. To design a multi-microgrid power system, an intelligent multi-microgrids energy management method is proposed based on the preference-based multi-objective reinforcement learning (PMORL) techniques. The power system model can be divided into three layers: the consumer layer, the independent system operator layer, and the power grid layer. Each layer intends to maximize its benefit. The PMORL is proposed to lead to a Pareto optimal set for each object to achieve these objectives. A non-dominated solution is decided to execute a balanced plan not to favor any particular participant. The preference-based results show that the proposed method can effectively learn different preferences. The simulation outcomes confirm the performance of the PMORL and verify the viability of the proposed method.<\/jats:p>","DOI":"10.1007\/s12293-022-00357-w","type":"journal-article","created":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T06:03:51Z","timestamp":1645509831000},"page":"225-235","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Preference based multi-objective reinforcement learning for multi-microgrid system optimization problem in smart grid"],"prefix":"10.1007","volume":"14","author":[{"given":"Jiangjiao","family":"Xu","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7200-4244","authenticated-orcid":false,"given":"Ke","family":"Li","sequence":"additional","affiliation":[]},{"given":"Mohammad","family":"Abusara","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,2,22]]},"reference":[{"key":"357_CR1","unstructured":"Department\u00a0for Business E and Strategy I (2020) Average annual domestic electricity bills by home and non-home supplier (QEP 2.2.1), Available https:\/\/www.gov.uk\/government\/statistical-data-sets\/annual-domestic-energy-price-statistics"},{"key":"357_CR2","unstructured":"Agency IE (2019) Electricity information 2019. [Online]. Available: https:\/\/www.oecd-ilibrary.org\/content\/publication\/e0ebb7e9-en"},{"key":"357_CR3","doi-asserted-by":"crossref","unstructured":"Sinha AK and Kumar N (2016) Demand response managemengt of smart grids using dynamic pricing. In: 2016 International conference on inventive computation technologies (ICICT), vol.\u00a01, pp 1\u20134","DOI":"10.1109\/INVENTIVE.2016.7823253"},{"issue":"2","key":"357_CR4","first-page":"879","volume":"7","author":"M Yu","year":"2016","unstructured":"Yu M, Hong SH (2016) A real-time demand-response algorithm for smart grids: a stackelberg game approach. IEEE Trans Smart Grid 7(2):879\u2013888","journal-title":"IEEE Trans Smart Grid"},{"issue":"3","key":"357_CR5","doi-asserted-by":"publisher","first-page":"1364","DOI":"10.1109\/TSG.2014.2376522","volume":"6","author":"W Wei","year":"2015","unstructured":"Wei W, Liu F, Mei S (2015) Energy pricing and dispatch for smart grid retailers under demand response and market price uncertainty. IEEE Trans Smart Grid 6(3):1364\u20131374","journal-title":"IEEE Trans Smart Grid"},{"key":"357_CR6","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1016\/j.apenergy.2015.10.119","volume":"167","author":"L Bai","year":"2016","unstructured":"Bai L, Li F, Cui H, Jiang T, Sun H, Zhu J (2016) Interval optimization based operating strategy for gas-electricity integrated energy systems considering demand response and wind uncertainty. Appl Energy 167:270\u2013279","journal-title":"Appl Energy"},{"issue":"2","key":"357_CR7","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1109\/JAS.2017.7510529","volume":"4","author":"Q Dong","year":"2017","unstructured":"Dong Q, Yu L, Song W, Yang J, Wu Y, Qi J (2017) Fast distributed demand response algorithm in smart grid. IEEE\/CAA J Autom Sin 4(2):280\u2013296","journal-title":"IEEE\/CAA J Autom Sin"},{"key":"357_CR8","doi-asserted-by":"crossref","unstructured":"Fahrioglu M,Alvarado FL (2002) Using utility information to calibrate customer demand management behavior models. In: 2002 IEEE power engineering society winter meeting. Conference Proceedings (Cat. No.02CH37309), vol.\u00a01, pp 317\u2013322","DOI":"10.1109\/59.918305"},{"issue":"4","key":"357_CR9","doi-asserted-by":"publisher","first-page":"2139","DOI":"10.1109\/TSG.2013.2265556","volume":"4","author":"S Salinas","year":"2013","unstructured":"Salinas S, Li M, Li P, Fu Y (2013) Dynamic energy management for the smart grid with distributed energy resources. IEEE Trans Smart Grid 4(4):2139\u20132151","journal-title":"IEEE Trans Smart Grid"},{"issue":"3","key":"357_CR10","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1109\/TPWRS.2005.852060","volume":"20","author":"AL Dimeas","year":"2005","unstructured":"Dimeas AL, Hatziargyriou ND (2005) Operation of a multiagent system for microgrid control. IEEE Trans Power Syst 20(3):1447\u20131455","journal-title":"IEEE Trans Power Syst"},{"key":"357_CR11","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1016\/j.energy.2014.05.101","volume":"77","author":"A Soares","year":"2014","unstructured":"Soares A, Antunes CH, Oliveira C, Gomes A (2014) A multi-objective genetic approach to domestic load scheduling in an energy management system. Energy 77:144\u2013152","journal-title":"Energy"},{"key":"357_CR12","doi-asserted-by":"publisher","first-page":"117104","DOI":"10.1016\/j.apenergy.2021.117104","volume":"299","author":"K Ullah","year":"2021","unstructured":"Ullah K, Hafeez G, Khan I, Jan S, Javaid N (2021) A multi-objective energy optimization in smart grid with high penetration of renewable energy sources. Appl Energy 299:117104\u2013117123","journal-title":"Appl Energy"},{"key":"357_CR13","doi-asserted-by":"crossref","unstructured":"Settaluri K, Haj-Ali A, Huang Q, Hakhamaneshi K, Nikolic B (2020) Autockt: deep reinforcement learning of analog circuit designs. In: (2020) Design. Automation test in Europe conference exhibition (DATE), pp 490\u2013495","DOI":"10.23919\/DATE48585.2020.9116200"},{"key":"357_CR14","unstructured":"Sutton RS,Barto AG (2018) Reinforcement learning: an introduction. The MIT Press"},{"key":"357_CR15","first-page":"325","volume":"5","author":"S Mannor","year":"2004","unstructured":"Mannor S, Shimkin N (2004) A geometric approach to multi-criterion reinforcement learning. J Mach Learn Res 5:325\u2013360","journal-title":"J Mach Learn Res"},{"key":"357_CR16","doi-asserted-by":"crossref","unstructured":"Tsitsiklis JN (1993) Asynchronous stochastic approximation and q-learning. In: Proceedings of 32nd IEEE conference on decision and control, vol.\u00a01,pp 395\u2013400","DOI":"10.1109\/CDC.1993.325119"},{"key":"357_CR17","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/s00291-001-0092-9","volume":"24","author":"K Miettinen","year":"2002","unstructured":"Miettinen K, Makela MM (2002) On scalarizing functions in multiobjective optimization. OR Spect 24:193\u2013213","journal-title":"OR Spect"},{"key":"357_CR18","unstructured":"G\u00e1bor Z, Kalm\u00e1r Z, Szepesv\u00e1ri C (1998) Multi-criteria reinforcement learning. In: Proceedings of the fifteenth international conference on machine learning, pp 197-205"},{"key":"357_CR19","doi-asserted-by":"crossref","unstructured":"Vamplew P, Yearwood J, Dazeley R, and Berry A (2008) On the limitations of scalarisation for multi-objective reinforcement learning of pareto fronts. In: Proceedings of the 21st Australasian joint conference on artificial intelligence: advances in artificial intelligence, pp 372-378","DOI":"10.1007\/978-3-540-89378-3_37"},{"key":"357_CR20","doi-asserted-by":"publisher","first-page":"10","DOI":"10.3390\/en9100812","volume":"9","author":"M Abbas","year":"2016","unstructured":"Abbas M, Kim E-S, Kim S-K, Kim Y-S (2016) Comparative analysis of battery behavior with different modes of discharge for optimal capacity sizing and bms operation. Energies 9:10","journal-title":"Energies"},{"key":"357_CR21","volume-title":"Microeconomic theory","author":"JR Green","year":"1995","unstructured":"Green JR, Mas-Colell A, Whinston M (1995) Microeconomic theory. Oxford University Press, New York"},{"key":"357_CR22","doi-asserted-by":"crossref","unstructured":"Fahrioglu M, Alvarado F (1999) Designing cost effective demand management contracts using game theory. In: IEEE power engineering society 1999 winter meeting (Cat. No.99CH36233), vol.\u00a01, pp 427\u2013432","DOI":"10.1109\/PESW.1999.747493"},{"key":"357_CR23","doi-asserted-by":"crossref","unstructured":"Roozbehani M, Dahleh M, Mitter S (2010) Dynamic pricing and stabilization of supply and demand in modern electric power grids. In: First IEEE international conference on smart grid communications 2010, pp 543\u2013548","DOI":"10.1109\/SMARTGRID.2010.5621994"},{"key":"357_CR24","doi-asserted-by":"crossref","unstructured":"Samadi P, Mohsenian-Rad A-H, Schober R, Wong VWS, Jatskevich J (2010) Optimal real-time pricing algorithm based on utility maximization for smart grid. In: First IEEE international conference on smart grid communications 2010, pp 415\u2013420","DOI":"10.1109\/SMARTGRID.2010.5622077"},{"issue":"4","key":"357_CR25","doi-asserted-by":"publisher","first-page":"2086","DOI":"10.1109\/TPWRS.2007.907390","volume":"22","author":"R Faranda","year":"2007","unstructured":"Faranda R, Pievatolo A, Tironi E (2007) Load shedding: a new proposal. IEEE Trans Power Syst 22(4):2086\u20132093","journal-title":"IEEE Trans Power Syst"},{"issue":"2","key":"357_CR26","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1109\/59.918305","volume":"16","author":"M Fahrioglu","year":"2001","unstructured":"Fahrioglu M, Alvarado F (2001) Using utility information to calibrate customer demand management behavior models. IEEE Trans Power Syst 16(2):317\u2013322","journal-title":"IEEE Trans Power Syst"},{"key":"357_CR27","unstructured":"Deane L (2020) one million faulty smart meters were installed in british homes. The Daily Mail"},{"key":"357_CR28","unstructured":"Department\u00a0for Business E and Strategy I (2020) Smart meter statistics in great Britain: quarterly Report to end June 2020, Online Available: Department for Business, Energy and Industrial Strategy"},{"issue":"1","key":"357_CR29","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1007\/BF01197559","volume":"14","author":"I Das","year":"1997","unstructured":"Das I, Dennis JE (1997) A closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Struct Optim 14(1):63\u201369","journal-title":"Struct Optim"},{"issue":"6","key":"357_CR30","doi-asserted-by":"publisher","first-page":"1084","DOI":"10.2514\/2.1071","volume":"38","author":"A Messac","year":"2000","unstructured":"Messac A, Sundararaj GJ, Tappeta RV, Renaud JE (2000) Ability of objective functions to generate points on nonconvex pareto frontiers. AIAA J 38(6):1084\u20131091","journal-title":"AIAA J"},{"issue":"6","key":"357_CR31","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1007\/s00158-003-0368-6","volume":"26","author":"RT Marler","year":"2004","unstructured":"Marler RT, Arora JS (2004) Survey of multi-objective optimization methods for engineering. Struct Multidiscip Optim 26(6):369\u2013395","journal-title":"Struct Multidiscip Optim"},{"issue":"2","key":"357_CR32","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1023\/A:1010035730904","volume":"1","author":"A Messac","year":"2000","unstructured":"Messac A, Puemi-Sukam C, Melachrinoudis E (2000) Aggregate objective functions and pareto frontiers: required relationships and practical implications. Optim Eng 1(2):171-188","journal-title":"Optim Eng"},{"key":"357_CR33","unstructured":"Dunford N, Schwartz JT, Bade WG, and Bartle RG (1998) Linear operators: general theory. part. I. Interscience Publishers"},{"key":"357_CR34","doi-asserted-by":"crossref","unstructured":"Yu N, Yu J (2006) Optimal tou decision considering demand response model. In: International conference on power system technology 2006, pp 1\u20135","DOI":"10.1109\/ICPST.2006.321461"}],"container-title":["Memetic Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12293-022-00357-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12293-022-00357-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12293-022-00357-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T02:08:39Z","timestamp":1652407719000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12293-022-00357-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,22]]},"references-count":34,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["357"],"URL":"https:\/\/doi.org\/10.1007\/s12293-022-00357-w","relation":{},"ISSN":["1865-9284","1865-9292"],"issn-type":[{"value":"1865-9284","type":"print"},{"value":"1865-9292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,22]]},"assertion":[{"value":"13 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 February 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}