{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T02:58:33Z","timestamp":1763348313550,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,15]],"date-time":"2021-01-15T00:00:00Z","timestamp":1610668800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Games"],"abstract":"<jats:p>Multi-Agent Systems (MAS) have been used to solve several optimization problems in control systems. MAS allow understanding the interactions between agents and the complexity of the system, thus generating functional models that are closer to reality. However, these approaches assume that information between agents is always available, which means the employment of a full-information model. Some tendencies have been growing in importance to tackle scenarios where information constraints are relevant issues. In this sense, game theory approaches appear as a useful technique that use a strategy concept to analyze the interactions of the agents and achieve the maximization of agent outcomes. In this paper, we propose a distributed control method of learning that allows analyzing the effect of the exploration concept in MAS. The dynamics obtained use Q-learning from reinforcement learning as a way to include the concept of exploration into the classic exploration-less Replicator Dynamics equation. Then, the Boltzmann distribution is used to introduce the Boltzmann-Based Distributed Replicator Dynamics as a tool for controlling agents behaviors. This distributed approach can be used in several engineering applications, where communications constraints between agents are considered. The behavior of the proposed method is analyzed using a smart grid application for validation purposes. Results show that despite the lack of full information of the system, by controlling some parameters of the method, it has similar behavior to the traditional centralized approaches.<\/jats:p>","DOI":"10.3390\/g12010008","type":"journal-article","created":{"date-parts":[[2021,1,21]],"date-time":"2021-01-21T02:36:05Z","timestamp":1611196565000},"page":"8","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Boltzmann Distributed Replicator Dynamics: Population Games in a Microgrid Context"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4895-4255","authenticated-orcid":false,"given":"Gustavo","family":"Chica-Pedraza","sequence":"first","affiliation":[{"name":"School of Telecommunications Engineering, Universidad Santo Tom\u00e1s, 110311 Bogot\u00e1 D.C., Colombia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3259-4013","authenticated-orcid":false,"given":"Eduardo","family":"Mojica-Nava","sequence":"additional","affiliation":[{"name":"Department of Electric and Electronic Engineering, Universidad Nacional de Colombia, 111321 Bogot\u00e1 D.C., Colombia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1086-3665","authenticated-orcid":false,"given":"Ernesto","family":"Cadena-Mu\u00f1oz","sequence":"additional","affiliation":[{"name":"Department of Systems and Industrial Engineering, Universidad Nacional de Colombia, 111321 Bogot\u00e1 D.C., Colombia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1109\/MSP.2015.2451994","article-title":"Game theory for networks: A tutorial on game-theoretic tools for emerging signal processing applications","volume":"33","author":"Bacci","year":"2015","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2639","DOI":"10.1007\/s11071-018-4713-0","article-title":"Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism","volume":"95","author":"Mu","year":"2019","journal-title":"Nonlinear Dyn."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.automatica.2015.10.012","article-title":"Distributed robust adaptive equilibrium computation for generalized convex games","volume":"63","author":"Zhu","year":"2016","journal-title":"Automatica"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100962","DOI":"10.1016\/j.phycom.2019.100962","article-title":"Distributed vs centralized game theory-based mode selection and power control for D2D communications","volume":"38","author":"Najeh","year":"2020","journal-title":"Phys. Commun."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.apenergy.2019.04.177","article-title":"Game theory based interactive demand side management responding to dynamic pricing in price-based demand response of smart grids","volume":"250","author":"Tang","year":"2019","journal-title":"Appl. Energy"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Fr\u00e4mling, K. (2020). Decision theory meets explainable ai. International Workshop on Explainable, Transparent Autonomous Agents and Multi-Agent Systems, Springer.","DOI":"10.1007\/978-3-030-51924-7_4"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Navon, A., Ben Yosef, G., Machlev, R., Shapira, S., Roy Chowdhury, N., Belikov, J., Orda, A., and Levron, Y. (2020). Applications of Game Theory to Design and Operation of Modern Power Systems: A Comprehensive Review. Energies, 13.","DOI":"10.3390\/en13153982"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MCS.2016.2621479","article-title":"The role of population games and evolutionary dynamics in distributed control systems: The advantages of evolutionary game theory","volume":"37","author":"Quijano","year":"2017","journal-title":"IEEE Control. Syst. Mag."},{"key":"ref_9","unstructured":"Lanctot, M., Lockhart, E., Lespiau, J.B., Zambaldi, V., Upadhyay, S., P\u00e9rolat, J., Srinivasan, S., Timbers, F., Tuyls, K., and Omidshafiei, S. (2019). OpenSpiel: A framework for reinforcement learning in games. arXiv."},{"key":"ref_10","unstructured":"Sandholm, W.H. (2010). Population Games and Evolutionary Dynamics, MIT Press."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-43102-z","article-title":"Computation and simulation of evolutionary Game Dynamics in Finite populations","volume":"9","author":"Hindersin","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_12","unstructured":"M\u00e4ki, U. (2012). Philosophy of game theory. Handbook of the Philosophy of Economics, Elsevier."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1439","DOI":"10.1109\/JSAC.2006.879350","article-title":"A tutorial on decomposition methods for network utility maximization","volume":"24","author":"Palomar","year":"2006","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3075","DOI":"10.1016\/j.automatica.2012.08.037","article-title":"State based potential games","volume":"48","author":"Marden","year":"2012","journal-title":"Automatica"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MNET.2019.1800221","article-title":"Optimal edge resource allocation in IoT-based smart cities","volume":"33","author":"Zhao","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"114039","DOI":"10.1016\/j.apenergy.2019.114039","article-title":"Microgrids: Overview and guidelines for practical implementations and operation","volume":"258","author":"Cagnano","year":"2020","journal-title":"Appl. Energy"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1109\/TPWRS.2006.873018","article-title":"Defining control strategies for microgrids islanded operation","volume":"21","author":"Lopes","year":"2006","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_18","unstructured":"Ibaraki, T., and Katoh, N. (1988). Resource Allocation Problems: Algorithmic Approaches, MIT Press."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ahn, S.J., and Moon, S.I. (2009, January 26\u201330). Economic scheduling of distributed generators in a microgrid considering various constraints. Proceedings of the 2009 IEEE Power & Energy Society General Meeting, Calgary, AB, Canada.","DOI":"10.1109\/PES.2009.5275938"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4419","DOI":"10.1016\/j.enpol.2008.09.030","article-title":"Demand side management: Benefits and challenges","volume":"36","author":"Strbac","year":"2008","journal-title":"Energy Policy"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Olivares, D.E., Ca\u00f1izares, C.A., and Kazerani, M. (2011, January 24\u201328). A centralized optimal energy management system for microgrids. Proceedings of the 2011 IEEE Power and Energy Society General Meeting, Detroit, MI, USA.","DOI":"10.1109\/PES.2011.6039527"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Quintana-Barcia, P., Dragicevic, T., Garcia, J., Ribas, J., and Guerrero, J.M. (2019). A distributed control strategy for islanded single-phase microgrids with hybrid energy storage systems based on power line signaling. Energies, 12.","DOI":"10.3390\/en12010085"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1634","DOI":"10.1109\/TSMC.2019.2900722","article-title":"Distributed optimal economic dispatch for microgrids considering communication delays","volume":"49","author":"Huang","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/MIE.2010.938720","article-title":"Hierarchical control of intelligent microgrids","volume":"4","author":"Vasquez","year":"2010","journal-title":"IEEE Ind. Electron. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"659","DOI":"10.1613\/jair.4818","article-title":"Evolutionary dynamics of multi-agent learning: A survey","volume":"53","author":"Bloembergen","year":"2015","journal-title":"J. Artif. Intell. Res."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Peters, H. (2015). Game Theory: A Multi-Leveled Approach, Springer.","DOI":"10.1007\/978-3-662-46950-7"},{"key":"ref_27","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ertel, W. (2017). Reinforcement Learning. Introduction to Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-319-58487-4"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1613\/jair.1.11396","article-title":"A survey on transfer learning for multiagent reinforcement learning systems","volume":"64","author":"Costa","year":"2019","journal-title":"J. Artif. Intell. Res."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ba\u015far, T., and Zaccour, G. (2018). Handbook of Dynamic Game Theory, Springer.","DOI":"10.1007\/978-3-319-44374-4"},{"key":"ref_31","unstructured":"Weibull, J.W. (1997). Evolutionary Game Theory, MIT Press."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Newton, J. (2018). Evolutionary game theory: A renaissance. Games, 9.","DOI":"10.3390\/g9020031"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Tuyls, K., Verbeeck, K., and Lenaerts, T. (2003, January 14\u201318). A selection-mutation model for q-learning in multi-agent systems. Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, Melbourne, Australia.","DOI":"10.1145\/860575.860687"},{"key":"ref_34","first-page":"304","article-title":"Distributed population dynamics: Optimization and control applications","volume":"47","author":"Obando","year":"2016","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Eiben, A.E., and Smith, J.E. (2015). Introduction to Evolutionary Computing, Springer.","DOI":"10.1007\/978-3-662-44874-8"},{"key":"ref_36","unstructured":"Stauffer, D. (1999). Life, Love and Death: Models of Biological Reproduction and Aging, Institute for Theoretical Physics."},{"key":"ref_37","unstructured":"Aj, W., and Wollenberg, B. (1996). Power Generation, Operation and Control, John Wiley & Sons."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4559","DOI":"10.1109\/TIE.2011.2107714","article-title":"A population dynamics approach for the dispatch of distributed generators","volume":"58","author":"Pantoja","year":"2011","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_39","first-page":"306","article-title":"Dynamic population games for optimal dispatch on hierarchical microgrid control","volume":"44","author":"Macana","year":"2013","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hofbauer, J., and Sigmund, K. (1998). Evolutionary Games and Population Dynamics, Cambridge University Press.","DOI":"10.1017\/CBO9781139173179"},{"key":"ref_41","unstructured":"Britton, N.F. (2012). Essential Mathematical Biology, Springer Science & Business Media."},{"key":"ref_42","unstructured":"Young, H.P., and Zamir, S. (2015). Handbook of Game Theory with Economic Applications, Elsevier. Technical Report."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2586","DOI":"10.1109\/TSG.2015.2444399","article-title":"Population games methods for distributed control of microgrids","volume":"6","author":"Barreto","year":"2015","journal-title":"IEEE Trans. Smart Grid"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Pantoja, A., Quijano, N., and Passino, K.M. (2014, January 4\u20136). Dispatch of distributed generators under local-information constraints. Proceedings of the 2014 American Control Conference, Portland, OR, USA.","DOI":"10.1109\/ACC.2014.6859190"}],"container-title":["Games"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-4336\/12\/1\/8\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:11:29Z","timestamp":1760159489000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-4336\/12\/1\/8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,15]]},"references-count":44,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["g12010008"],"URL":"https:\/\/doi.org\/10.3390\/g12010008","relation":{},"ISSN":["2073-4336"],"issn-type":[{"type":"electronic","value":"2073-4336"}],"subject":[],"published":{"date-parts":[[2021,1,15]]}}}