{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T18:49:58Z","timestamp":1770144598467,"version":"3.49.0"},"reference-count":48,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T00:00:00Z","timestamp":1655078400000},"content-version":"vor","delay-in-days":163,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2022,1,1]],"date-time":"2022-01-01T00:00:00Z","timestamp":1640995200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Complexity"],"published-print":{"date-parts":[[2022,1]]},"abstract":"<jats:p>The area of smart power grids needs to constantly improve its efficiency and resilience, to provide high quality electrical power in a resilient grid, while managing faults and avoiding failures. Achieving this requires high component reliability, adequate maintenance, and a studied failure occurrence. Correct system operation involves those activities and novel methodologies to detect, classify, and isolate faults and failures and model and simulate processes with predictive algorithms and analytics (using data analysis and asset condition to plan and perform activities). In this paper, we showcase the application of a complex\u2010adaptive, self\u2010organizing modeling method, and Probabilistic Boolean Networks (PBNs), as a way towards the understanding of the dynamics of smart grid devices, and to model and characterize their behavior. This work demonstrates that PBNs are equivalent to the standard Reinforcement Learning Cycle, in which the agent\/model has an interaction with its environment and receives feedback from it in the form of a reward signal. Different reward structures were created to characterize preferred behavior. This information can be used to guide the PBN to avoid fault conditions and failures.<\/jats:p>","DOI":"10.1155\/2022\/3652441","type":"journal-article","created":{"date-parts":[[2022,6,13]],"date-time":"2022-06-13T16:05:06Z","timestamp":1655136306000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Reinforcement Learning with Probabilistic Boolean Network Models of Smart Grid Devices"],"prefix":"10.1155","volume":"2022","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3507-1821","authenticated-orcid":false,"given":"Pedro Juan","family":"Rivera Torres","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0193-3067","authenticated-orcid":false,"given":"Carlos","family":"Gershenson Garc\u00eda","sequence":"additional","affiliation":[]},{"given":"Mar\u00eda Fernanda","family":"S\u00e1nchez Puig","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6564-0557","authenticated-orcid":false,"given":"Samir","family":"Kanaan Izquierdo","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2022,6,13]]},"reference":[{"key":"e_1_2_9_1_2","doi-asserted-by":"publisher","DOI":"10.20964\/2018.02.54"},{"key":"e_1_2_9_2_2","doi-asserted-by":"publisher","DOI":"10.1051\/e3sconf\/202129701068"},{"key":"e_1_2_9_3_2","unstructured":"European Union Commission Task Force on Smart Grids European Union Commission Task Force on Smart Grids 2009 https:\/\/www.usef.energy\/implementations\/smart-grids-task-force-eg3."},{"key":"e_1_2_9_4_2","doi-asserted-by":"publisher","DOI":"10.1002\/9780470602782.ch3"},{"key":"e_1_2_9_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIE.2020.2975049"},{"key":"e_1_2_9_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-32817-6_2"},{"key":"e_1_2_9_7_2","unstructured":"TorresR. PedroJ. andKanaan IzquierdoS. Contributions to Reinforcement Learning through Probabilistic Boolean Networks Proceedings of the a Poster Presented in the Conference on Complex Systems October 2020 Thessaloniki Greece."},{"key":"e_1_2_9_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-34409-2_10"},{"key":"e_1_2_9_9_2","first-page":"355","volume-title":"Advances in Intelligent Systems and Computing","author":"Torres R.","year":"2019"},{"key":"e_1_2_9_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10845-016-1226-x"},{"key":"e_1_2_9_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10845-015-1183-9"},{"key":"e_1_2_9_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10845-015-1143-4"},{"key":"e_1_2_9_13_2","unstructured":"TorresR. PedroJ. andLlanes SantiagoO. Model-based Fault Diagnosis of Manufacturing Processes and Machines Using Probabilistic Boolean Networks Proceedings of the 19th Scientific Conference of Engineering and Architecture ISP-CUJAE November 2018d La Habana Cuba."},{"key":"e_1_2_9_14_2","volume-title":"Contribuciones al modelado te\u00f3rico de sistemas de fabricaci\u00f3n mediante Redes Booleanas Probabil\u00edsticas","author":"Torres R.","year":"2017"},{"key":"e_1_2_9_15_2","unstructured":"TorresR. PedroJ. andSerrano MercadoE. I. Probabilistic Boolean Network Modeling as an Aid for DFMEA in Manufacturing Systems Proceedings of the 18th Scientific Conference of Engineering and Architecture ISP-CUJAE November 2016 La Habana Cuba."},{"key":"e_1_2_9_16_2","unstructured":"Rivera-Torres PedroJ. SeguelJ. Rodr\u00edguez-Mart\u00ednezM. Irizarry-Rivera andAgust\u00edn Formal Methods for the Design and Analysis of Electrical Power Distribution Systems Endowed with Intelligent Power Routers 2012 Lockheed-Martin Bethesda MD USA Final Report for PO 4100307844."},{"key":"e_1_2_9_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/0022-5193(69)90015-0"},{"key":"e_1_2_9_18_2","doi-asserted-by":"publisher","DOI":"10.1038\/224177a0"},{"key":"e_1_2_9_19_2","doi-asserted-by":"publisher","DOI":"10.3389\/fgene.2020.00625"},{"key":"e_1_2_9_20_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/18.2.261"},{"key":"e_1_2_9_21_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898717631"},{"key":"e_1_2_9_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/jproc.2002.804686"},{"key":"e_1_2_9_23_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1215732109"},{"key":"e_1_2_9_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/tcbb.2011.61"},{"key":"e_1_2_9_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dam.2012.05.010"},{"key":"e_1_2_9_26_2","volume-title":"Perturbations in Boolean Networks as Model of Gene Regulatory Dynamics (Doctoral Thesis)","author":"Ghanbarnejad F.","year":"2012"},{"key":"e_1_2_9_27_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0069626"},{"key":"e_1_2_9_28_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-syb.2008.0173"},{"key":"e_1_2_9_29_2","unstructured":"KobayashiK.andHiraishiK. Reachability analysis of probabilistic boolean networks using model checking Proceedings of the SICE Annual Conference 2010 August 2010 Taipei Taiwan 829\u2013832."},{"key":"e_1_2_9_30_2","doi-asserted-by":"publisher","DOI":"10.4208\/eajam.030511.060911a"},{"key":"e_1_2_9_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.biosystems.2013.07.008"},{"key":"e_1_2_9_32_2","doi-asserted-by":"publisher","DOI":"10.1186\/1478-811x-11-46"},{"key":"e_1_2_9_33_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cta.2014.0536"},{"key":"e_1_2_9_34_2","doi-asserted-by":"publisher","DOI":"10.1515\/9781400865260"},{"key":"e_1_2_9_35_2","unstructured":"GershensonC. BedauM. HuttonT. KumarS. andSuzukiH. Introduction to random boolean networks Proceedings of the Workshop and Tutorial Proceedings Ninth International Conference on the Simulation and Synthesis of Living Systems (Alife IX) September2004 Boston MA USA 160\u2013173."},{"key":"e_1_2_9_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12064-011-0144-x"},{"key":"e_1_2_9_37_2","doi-asserted-by":"publisher","DOI":"10.17775\/CSEEJPES.2018.00520"},{"key":"e_1_2_9_38_2","doi-asserted-by":"crossref","unstructured":"ZhangH. T. XuF. Y. andZhouL. Artificial neural network for load forecasting in smart grid 6 Proceedings of the 2010 International Conference on Machine Learning and Cybernetics (July2010 Qingdao China IEEE 3200\u20133205 https:\/\/doi.org\/10.1109\/icmlc.2010.5580713 2-s2.0-78149306099.","DOI":"10.1109\/ICMLC.2010.5580713"},{"key":"e_1_2_9_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2014.08.014"},{"key":"e_1_2_9_40_2","article-title":"Machine learning au service de la pr\u00e9diction de la demande d\u2019\u00e9nergie dans les smart grid","volume":"8","author":"Mountassir F.","year":"2018","journal-title":"Mediterranean Telecommunications Journal"},{"key":"e_1_2_9_41_2","volume-title":"Macro-Actions in Reinforcement Learning: An Empirical Analysis\u2019 by Amy McGovern and Richard S","author":"Sutton R. S.","year":"1998"},{"key":"e_1_2_9_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/tnn.1998.712192"},{"key":"e_1_2_9_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/tii.2020.3014599"},{"key":"e_1_2_9_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/tvt.2020.3025627"},{"key":"e_1_2_9_45_2","unstructured":"McCallumA. K. Reinforcement Learning with Selective Perception and Hidden State 1996 University of Rochester Rochester NY USA Ph.D. Dissertation."},{"key":"e_1_2_9_46_2","unstructured":"RummeryG. A. Problem Solving with Reinforcement Learning 1995 Cambridge University Cambridge UK Ph.D. Dissertation."},{"key":"e_1_2_9_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22110-1_47"},{"key":"e_1_2_9_48_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2007.11.009"}],"container-title":["Complexity"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2022\/3652441","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1155\/2022\/3652441","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1155\/2022\/3652441","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T21:28:16Z","timestamp":1769117296000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1155\/2022\/3652441"}},"subtitle":[],"editor":[{"given":"Hiroki","family":"Sayama","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,1]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,1]]}},"alternative-id":["10.1155\/2022\/3652441"],"URL":"https:\/\/doi.org\/10.1155\/2022\/3652441","archive":["Portico"],"relation":{},"ISSN":["1076-2787","1099-0526"],"issn-type":[{"value":"1076-2787","type":"print"},{"value":"1099-0526","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1]]},"assertion":[{"value":"2021-04-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-06-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"3652441"}}