{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T06:28:03Z","timestamp":1778567283344,"version":"3.51.4"},"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T00:00:00Z","timestamp":1734393600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union\u2019s Horizon 2020 research and innovation programme under the Marie Sk\u0142odowska-Curie","award":["945045"],"award-info":[{"award-number":["945045"]}]},{"name":"NWO Gravitation","award":["024.002.003"],"award-info":[{"award-number":["024.002.003"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Previous explanations for the persistence of polarization of opinions have typically included modelling assumptions that predispose the possibility of polarization (i.e., assumptions allowing a pair of agents to drift apart in their opinion such as repulsive interactions or bounded confidence). An exception is a recent simulation study showing that polarization is persistent when agents form their opinions using social reinforcement learning. Our goal is to highlight the usefulness of reinforcement learning in the context of modeling opinion dynamics, but that caution is required when selecting the tools used to study such a model. We show that the polarization observed in the model of the simulation study cannot persist indefinitely, and exhibits consensus asymptotically with probability one. By constructing a link between the reinforcement learning model and the voter model, we argue that the observed polarization is metastable. Finally, we show that a slight modification in the learning process of the agents changes the model from being non-ergodic to being ergodic. Our results show that reinforcement learning may be a powerful method for modelling polarization in opinion dynamics, but that the tools (objects to study such as the stationary distribution, or time to absorption for example) appropriate for analysing such models crucially depend on their properties (such as ergodicity, or transience). These properties are determined by the details of the learning process and may be difficult to identify based solely on simulations.<\/jats:p>","DOI":"10.1371\/journal.pone.0313951","type":"journal-article","created":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T18:28:37Z","timestamp":1734460117000},"page":"e0313951","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["How social reinforcement learning can lead to metastable polarisation and the voter model"],"prefix":"10.1371","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2685-6857","authenticated-orcid":true,"given":"Benedikt V.","family":"Meylahn","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4388-805X","authenticated-orcid":true,"given":"Janusz M.","family":"Meylahn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2024,12,17]]},"reference":[{"key":"pone.0313951.ref001","first-page":"1","article-title":"Mathematical models of the distribution of attitudes under controversy","volume":"14","author":"RP Abelson","year":"1964","journal-title":"Contributions to Mathematical Psychology"},{"key":"pone.0313951.ref002","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1142\/S0219525900000078","article-title":"Mixing beliefs among interacting agents","volume":"3","author":"G Deffuant","year":"2000","journal-title":"Advances in Complex Systems"},{"issue":"3","key":"pone.0313951.ref003","article-title":"Opinion dynamics and bounded confidence models, analysis and simulation","volume":"5","author":"R Hegselmann","year":"2000","journal-title":"Journal of Artificial Societies and Social Simulation"},{"key":"pone.0313951.ref004","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1140\/epjb\/e2004-00126-9","article-title":"Bounded confidence and social networks","volume":"38","author":"G Weisbuch","year":"2004","journal-title":"The European Physics Journal B"},{"issue":"2","key":"pone.0313951.ref005","doi-asserted-by":"crossref","DOI":"10.1142\/S0218202511500072","article-title":"The bounded confidence model of opinion dynamics","volume":"22","author":"J G\u00f3mez-Serrano","year":"2012","journal-title":"Mathematical Models and Methods in Applied Sciences"},{"issue":"1-3","key":"pone.0313951.ref006","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1080\/0022250X.2010.532261","article-title":"Small worlds and cultural polarization","volume":"35","author":"A Flache","year":"2011","journal-title":"The Journal of Mathematical Sociology"},{"issue":"9","key":"pone.0313951.ref007","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0044489","article-title":"Discrete Model of Opinion Changes Using Knowledge and Emotions as Control Variables","volume":"7","author":"P Sobkowicz","year":"2012","journal-title":"PLOS ONE"},{"issue":"4","key":"pone.0313951.ref008","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1109\/TAC.2012.2224251","article-title":"Consensus Problems on Networks With Antagonistic Interactions","volume":"58","author":"C Altafini","year":"2013","journal-title":"IEEE Transactions on Automatic Control"},{"issue":"1","key":"pone.0313951.ref009","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/0022250X.2022.2036142","article-title":"A relative approach to opinion formation","volume":"48","author":"KMD Chan","year":"2022","journal-title":"The Journal of Mathematical Sociology"},{"issue":"2","key":"pone.0313951.ref010","first-page":"123","article-title":"Quantitatively modelling opinion dynamics during elections","volume":"38","author":"MF Burke","year":"2022","journal-title":"ORiON"},{"issue":"3","key":"pone.0313951.ref011","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1080\/00222500490480257","article-title":"When will they ever make up their minds? The social structure of unstable decision making","volume":"28","author":"A Flache","year":"2004","journal-title":"The Journal of Mathematical Sociology"},{"issue":"17","key":"pone.0313951.ref012","doi-asserted-by":"crossref","first-page":"3036","DOI":"10.1016\/j.physa.2011.03.021","article-title":"Collective beliefs versus individual inflexibility: The unavoidable biases of a public debate","volume":"390","author":"S Galam","year":"2011","journal-title":"Physica A: Statistical Mechanics and its Applications"},{"issue":"4","key":"pone.0313951.ref013","doi-asserted-by":"crossref","first-page":"19:130","DOI":"10.1145\/2538508","article-title":"Binary opinion dynamics with stubborn agents","volume":"1","author":"E Yildiz","year":"2013","journal-title":"ACM Transactions on Economics and Computation"},{"key":"pone.0313951.ref014","doi-asserted-by":"crossref","first-page":"042303","DOI":"10.1103\/PhysRevE.102.042303","article-title":"Dynamics of opinion expression","volume":"102","author":"F Gaisbauer","year":"2020","journal-title":"Physical Review E"},{"key":"pone.0313951.ref015","first-page":"1","article-title":"Opinion dynamics beyond social influence","author":"BV Meylahn","year":"2024","journal-title":"Network Science"},{"issue":"3","key":"pone.0313951.ref016","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1037\/h0046123","article-title":"A formal theory of social power","volume":"63","author":"JRP French","year":"1956","journal-title":"Psychological Review"},{"key":"pone.0313951.ref017","first-page":"168","volume-title":"Studies in social power","author":"F Harary","year":"1959"},{"issue":"345","key":"pone.0313951.ref018","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1080\/01621459.1974.10480137","article-title":"Reaching a consensus","volume":"69","author":"MH DeGroot","year":"1974","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"pone.0313951.ref019","doi-asserted-by":"crossref","first-page":"89","DOI":"10.2307\/3213262","article-title":"Towards consensus: Some convergence theorems on repeated averaging","volume":"14","author":"S Chatterjee","year":"1977","journal-title":"Journal of Applied Probability"},{"key":"pone.0313951.ref020","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1093\/biomet\/60.3.581","article-title":"A model for spatial conflict","volume":"60","author":"P Clifford","year":"1973","journal-title":"Biometrika"},{"issue":"4","key":"pone.0313951.ref021","doi-asserted-by":"crossref","first-page":"643","DOI":"10.1214\/aop\/1176996306","article-title":"Ergodic theorems for weakly interacting infinite systems and the voter model","volume":"3","author":"RA Holley","year":"1975","journal-title":"The Annals of Probability"},{"issue":"2","key":"pone.0313951.ref022","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1080\/0022250X.2018.1517761","article-title":"Opinion polarization by learning from social feedback","volume":"43","author":"S Banisch","year":"2019","journal-title":"The Journal of Mathematical Sociology"},{"issue":"2","key":"pone.0313951.ref023","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1103\/RevModPhys.81.591","article-title":"Statistical physics models of social dynamics","volume":"81","author":"C Castellano","year":"2009","journal-title":"Review of Modern Physics"},{"issue":"4","key":"pone.0313951.ref024","doi-asserted-by":"crossref","DOI":"10.18564\/jasss.3521","article-title":"Models of social influence: Towards the next frontiers","volume":"20","author":"A Flache","year":"2017","journal-title":"Journal of Artificial Societies and Social Simulation"},{"issue":"10","key":"pone.0313951.ref025","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0258259","article-title":"Modeling the emergence of affective polarization in the social media society","volume":"16","author":"P T\u00f6rnberg","year":"2021","journal-title":"PLOS ONE"},{"issue":"42","key":"pone.0313951.ref026","doi-asserted-by":"crossref","first-page":"e2207159119","DOI":"10.1073\/pnas.2207159119","article-title":"How digital media drive affective polarization through partisan sorting","volume":"119","author":"P T\u00f6rnberg","year":"2022","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"pone.0313951.ref027","article-title":"Modelling adaptive learning behaviours for consensus formation in human societies","volume":"6","author":"C Yu","year":"2016","journal-title":"Scientific Reports"},{"issue":"3","key":"pone.0313951.ref028","doi-asserted-by":"crossref","DOI":"10.3390\/ijerph17030946","article-title":"Public Opinion Polarization by Individual Revenue from the Social Preference Theory","volume":"17","author":"T Chen","year":"2020","journal-title":"International Journal of Environmental Research and Public Health"},{"issue":"4","key":"pone.0313951.ref029","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1037\/rev0000291","article-title":"Individual attitude change and societal dynamics: Computational experiments with psychological theories","volume":"128","author":"J Lorenz","year":"2021","journal-title":"Psychological Review"},{"key":"pone.0313951.ref030","doi-asserted-by":"crossref","first-page":"127423","DOI":"10.1016\/j.physa.2022.127423","article-title":"Clustering and stubbornness regulate the formation of echo chambers in personalised opinion dynamics","volume":"599","author":"N Botte","year":"2022","journal-title":"Physica A: Statistical Mechanics and its Applications"},{"key":"pone.0313951.ref031","doi-asserted-by":"crossref","first-page":"20232011","DOI":"10.1098\/rspb.2023.2011","article-title":"The roots of polarization in the individual reward system","volume":"291","author":"G Lefebvre","year":"2024","journal-title":"Proceedings of the Royal Society B: Biological Sciences"},{"issue":"4","key":"pone.0313951.ref032","first-page":"1333","article-title":"Coalescing random walks and voter model consensus times on the torus in Zd","volume":"17","author":"JT Cox","year":"1989","journal-title":"The Annals of Probability"},{"key":"pone.0313951.ref033","doi-asserted-by":"crossref","first-page":"036132","DOI":"10.1103\/PhysRevE.72.036132","article-title":"Voter model dynamics in complex networks: Role of dimensionality, disorder, and degree distribution","volume":"72","author":"K Suchecki","year":"2005","journal-title":"Physical Review E"},{"key":"pone.0313951.ref034","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1209\/epl\/i2004-10329-8","article-title":"Conservation laws for the voter model in complex networks","volume":"62","author":"K Suchecki","year":"2005","journal-title":"Europhysics Letters"},{"issue":"1","key":"pone.0313951.ref035","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1209\/epl\/i2003-00490-0","article-title":"Incomplete ordering of the voter model on small-world networks","volume":"63","author":"C Castellano","year":"2003","journal-title":"Europhysics Letters"},{"key":"pone.0313951.ref036","doi-asserted-by":"crossref","first-page":"016109","DOI":"10.1103\/PhysRevE.69.016109","article-title":"Solution of voter model dynamics on annealed small-world networks","volume":"69","author":"D Vilone","year":"2004","journal-title":"Physical Review E"},{"key":"pone.0313951.ref037","doi-asserted-by":"crossref","first-page":"00491241231186658","DOI":"10.1177\/00491241231186658","article-title":"Biased Processing and Opinion Polarization: Experimental Refinement of Argument Communication Theory in the Context of the Energy Debate","author":"S Banisch","year":"2023","journal-title":"Sociological Methods & Research"},{"key":"pone.0313951.ref038","doi-asserted-by":"crossref","first-page":"016121","DOI":"10.1103\/PhysRevE.66.016121","article-title":"Random geometric graphs","volume":"66","author":"J Dall","year":"2002","journal-title":"Physical Review E"},{"issue":"6","key":"pone.0313951.ref039","doi-asserted-by":"crossref","first-page":"063119","DOI":"10.1063\/5.0205136","article-title":"Interpersonal trust: Asymptotic analysis of a stochastic coordination game with multi-agent learning","volume":"34","author":"BV Meylahn","year":"2024","journal-title":"Chaos"},{"issue":"1","key":"pone.0313951.ref040","first-page":"279","article-title":"Absolutely expedient algorithms for learning Nash equilibria","volume":"104","author":"VV Phansalkar","year":"1994","journal-title":"Proceedings of the Indian Academy of Sciences: Mathematical Sciences"},{"issue":"5","key":"pone.0313951.ref041","doi-asserted-by":"crossref","DOI":"10.1109\/21.293490","article-title":"Decentralized learning of Nash equilibria in multi-person stochastic games with incomplete information","volume":"24","author":"PS Sastry","year":"1994","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics"},{"issue":"1","key":"pone.0313951.ref042","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1006\/jeth.1997.2319","article-title":"Learning through reinforcement and replicator dynamics","volume":"77","author":"T B\u00f6rgers","year":"1997","journal-title":"Journal of Economic Theory"},{"issue":"7","key":"pone.0313951.ref043","doi-asserted-by":"crossref","first-page":"4748","DOI":"10.1073\/pnas.032086299","article-title":"Chaos in learning a simple two-person game","volume":"99","author":"Y Sato","year":"2002","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pone.0313951.ref044","doi-asserted-by":"crossref","first-page":"015206","DOI":"10.1103\/PhysRevE.67.015206","article-title":"Coupled replicator equations for the dynamics of learning in multiagent systems","volume":"67","author":"Y Sato","year":"2003","journal-title":"Physical Review E"},{"issue":"1","key":"pone.0313951.ref045","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.physd.2005.06.031","article-title":"Stability and diversity in collective adaptation","volume":"210","author":"Y Sato","year":"2005","journal-title":"Physica D: Nonlinear Phenomena"},{"key":"pone.0313951.ref046","doi-asserted-by":"crossref","first-page":"043305","DOI":"10.1103\/PhysRevE.99.043305","article-title":"Deterministic limit of temporal difference reinforcement learning for stochastic games","volume":"99","author":"W Barfuss","year":"2019","journal-title":"Physical Review E"},{"issue":"1","key":"pone.0313951.ref047","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-023-27672-7","article-title":"Intrinsic fluctuations of reinforcement learning promote cooperation","volume":"13","author":"W Barfuss","year":"2023","journal-title":"Scientific Reports"},{"key":"pone.0313951.ref048","doi-asserted-by":"crossref","first-page":"4830491","DOI":"10.1155\/2022\/4830491","article-title":"Limiting dynamics for Q-Learning with memory one in symmetric two-player, two-action games","volume":"2022","author":"JM Meylahn","year":"2022","journal-title":"Complexity"},{"issue":"4","key":"pone.0313951.ref049","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/j.socnet.2012.06.001","article-title":"Agent based models and opinion dynamics as Markov chains","volume":"34","author":"S Banisch","year":"2012","journal-title":"Social Networks"},{"issue":"1","key":"pone.0313951.ref050","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/0304-4149(94)00035-R","article-title":"The noisy voter model","volume":"55","author":"BL Granovsky","year":"1995","journal-title":"Stochastic Processes and their Applications"},{"issue":"1","key":"pone.0313951.ref051","doi-asserted-by":"crossref","first-page":"24775","DOI":"10.1038\/srep24775","article-title":"The noisy voter model on complex networks","volume":"6","author":"A Carro","year":"2016","journal-title":"Scientific Reports"},{"issue":"7","key":"pone.0313951.ref052","doi-asserted-by":"crossref","first-page":"1989","DOI":"10.1073\/pnas.1418838112","article-title":"The spontaneous emergence of conventions: An experimental study of cultural evolution","volume":"112","author":"D Centola","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pone.0313951.ref053","doi-asserted-by":"crossref","first-page":"2222","DOI":"10.1038\/s41598-024-52458-w","article-title":"Bursts of communication increase opinion diversity in the temporal Deffuant model","volume":"14","author":"F Zarei","year":"2024","journal-title":"Scientific Reports"},{"key":"pone.0313951.ref054","doi-asserted-by":"crossref","DOI":"10.3389\/fphy.2015.00064","article-title":"Consensus emerging from the bottom-up: The role of cognitive variables in opinion dynamics","volume":"3","author":"F Giardini","year":"2015","journal-title":"Frontiers in Physics"},{"issue":"3","key":"pone.0313951.ref055","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1093\/ijpor\/edm014","article-title":"The Role of Communication in Public Opinion Processes: Understanding the Impacts of Intrapersonal, Media, and Social Filters","volume":"19","author":"LH Hoffman","year":"2007","journal-title":"International Journal of Public Opinion Research"},{"key":"pone.0313951.ref056","doi-asserted-by":"crossref","first-page":"14600","DOI":"10.1038\/s41598-023-39725-y","article-title":"Mass media impact on opinion evolution in biased digital environments: A bounded confidence model","volume":"13","author":"V Pansanella","year":"2023","journal-title":"Scientific Reports"},{"key":"pone.0313951.ref057","doi-asserted-by":"crossref","first-page":"129929","DOI":"10.1016\/j.physa.2024.129929","article-title":"Social clustering reinforces external influence on the majority opinion model","volume":"648","author":"N Van Santen","year":"2024","journal-title":"Physica A: Statistical Mechanics and its Applications"},{"key":"pone.0313951.ref058","doi-asserted-by":"crossref","unstructured":"Hagberg AA, Schult DA, Swart PJ. Exploring network structure, dynamics, and function using NetworkX. In: Varoquaux G, Vaught T, Millman J, editors. Proceedings of the 7th Python in Science Conference. Pasadena, (CA) USA; 2008. p. 11\u201315.","DOI":"10.25080\/TCWV9851"},{"key":"pone.0313951.ref059","doi-asserted-by":"crossref","unstructured":"Penrose M. Random Geometric Graphs. Oxford University Press; 2003. Available from: https:\/\/doi.org\/10.1093\/acprof:oso\/9780198506263.001.0001.","DOI":"10.1093\/acprof:oso\/9780198506263.001.0001"},{"key":"pone.0313951.ref060","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1146\/annurev.soc.27.1.415","article-title":"Birds of a Feather: Homophily in Social Networks","volume":"27","author":"M McPherson","year":"2001","journal-title":"Annual Review of Sociology"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313951","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,17]],"date-time":"2024-12-17T18:29:17Z","timestamp":1734460157000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0313951"}},"subtitle":[],"editor":[{"given":"William","family":"Ott","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2024,12,17]]},"references-count":60,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0313951","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,17]]}}}