{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,19]],"date-time":"2026-06-19T21:13:05Z","timestamp":1781903585648,"version":"3.54.5"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011840","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T00:00:00Z","timestamp":1707436800000}}],"reference-count":55,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2024,1,30]],"date-time":"2024-01-30T00:00:00Z","timestamp":1706572800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["789240"],"award-info":[{"award-number":["789240"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Learning from past experience is an important adaptation and theoretical models may help to understand its evolution. Many of the existing models study simple phenotypes and do not consider the mechanisms underlying learning while the more complex neural network models often make biologically unrealistic assumptions and rarely consider evolutionary questions. Here, we present a novel way of modelling learning using small neural networks and a simple, biology-inspired learning algorithm. Learning affects only part of the network, and it is governed by the difference between expectations and reality. We use this model to study the evolution of learning under various environmental conditions and different scenarios for the trade-off between exploration (learning) and exploitation (foraging). Efficient learning readily evolves in our individual-based simulations. However, in line with previous studies, the evolution of learning is less likely in relatively constant environments, where genetic adaptation alone can lead to efficient foraging, or in short-lived organisms that cannot afford to spend much of their lifetime on exploration. Once learning does evolve, the characteristics of the learning strategy (i.e. the duration of the learning period and the learning rate) and the average performance after learning are surprisingly little affected by the frequency and\/or magnitude of environmental change. In contrast, an organism\u2019s lifespan and the distribution of resources in the environment have a clear effect on the evolved learning strategy: a shorter lifespan or a broader resource distribution lead to fewer learning episodes and larger learning rates. Interestingly, a longer learning period does not always lead to better performance, indicating that the evolved neural networks differ in the effectiveness of learning. Overall, however, we show that a biologically inspired, yet relatively simple, learning mechanism can evolve to lead to an efficient adaptation in a changing environment.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011840","type":"journal-article","created":{"date-parts":[[2024,1,30]],"date-time":"2024-01-30T13:33:49Z","timestamp":1706621629000},"page":"e1011840","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["A neural network model for the evolution of learning in changing environments"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0137-5379","authenticated-orcid":true,"given":"Magdalena","family":"Kozielska","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3281-663X","authenticated-orcid":true,"given":"Franz J.","family":"Weissing","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,1,30]]},"reference":[{"key":"pcbi.1011840.ref001","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195122343.001.0001","volume-title":"Developmental Plasticity and Evolution.","author":"MJ West-Eberhard","year":"2003"},{"key":"pcbi.1011840.ref002","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.jtbi.2010.06.017","article-title":"The evolution of associative learning: A factor in the Cambrian explosion","volume":"266","author":"S Ginsburg","year":"2010","journal-title":"J Theor Biol"},{"key":"pcbi.1011840.ref003","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1007\/s12038-009-0081-8","article-title":"Epigenetic learning in non-neural organisms","volume":"34","author":"S Ginsburg","year":"2009","journal-title":"J Biosci"},{"key":"pcbi.1011840.ref004","first-page":"1012","volume-title":"Invertebrate Learning.","author":"WC Corning"},{"key":"pcbi.1011840.ref005","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1007\/978-1-4419-1428-6_302","volume-title":"Encyclopedia of the Sciences of Learning","author":"M. Papini","year":"2012"},{"key":"pcbi.1011840.ref006","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1017\/S1138741600005333","article-title":"Contemporary study of Pavlovian conditioning","volume":"6","author":"RA Rescorla","year":"2003","journal-title":"Span J Psychol"},{"key":"pcbi.1011840.ref007","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.anbehav.2018.05.024","article-title":"Components of change and the evolution of learning in theory and experiment","volume":"147","author":"AS Dunlap","year":"2019","journal-title":"Anim Behav"},{"key":"pcbi.1011840.ref008","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.conb.2006.03.006","article-title":"The computational neurobiology of learning and reward","volume":"16","author":"ND Daw","year":"2006","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011840.ref009","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1038\/s42256-019-0025-4","article-title":"Reinforcement learning in artificial and biological systems.","volume":"1","author":"EO Neftci","year":"2019","journal-title":"Nat Mach Intell"},{"key":"pcbi.1011840.ref010","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.tpb.2013.09.003","article-title":"On learning dynamics underlying the evolution of learning rules","volume":"91","author":"S Dridi","year":"2014","journal-title":"Theor Popul Biol"},{"key":"pcbi.1011840.ref011","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.tpb.2013.10.004","article-title":"Evolution of learning strategies in temporally and spatially variable environments: A review of theory","volume":"91","author":"K Aoki","year":"2014","journal-title":"Theor Popul Biol"},{"key":"pcbi.1011840.ref012","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1093\/beheco\/ars085","article-title":"Exposing the behavioral gambit: The evolution of learning and decision rules","volume":"24","author":"TW Fawcett","year":"2013","journal-title":"Behav Ecol"},{"key":"pcbi.1011840.ref013","first-page":"586","article-title":"Evolution of learning in fluctuating environments: When selection favors both social and exploratory individual learning","volume":"62","author":"E Borenstein","year":"2008","journal-title":"Evolution (N Y)."},{"key":"pcbi.1011840.ref014","doi-asserted-by":"crossref","first-page":"3609","DOI":"10.1098\/rspb.2010.0857","article-title":"Learning in a game context: strategy choice by some keeps learning from evolving in others","author":"F Dubois","year":"2010","journal-title":"Proc R Soc B Biol Sci"},{"key":"pcbi.1011840.ref015","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1016\/j.anbehav.2015.01.037","article-title":"A model for the evolution of reinforcement learning in fluctuating games","volume":"104","author":"S Dridi","year":"2015","journal-title":"Anim Behav"},{"key":"pcbi.1011840.ref016","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1098\/rsif.2007.1348","article-title":"Simple learning rules to cope with changing environments","volume":"5","author":"R Gro\u00df","year":"2008","journal-title":"J R Soc Interface"},{"key":"pcbi.1011840.ref017","first-page":"1","article-title":"Neural network models for the evolution of associative learning","author":"E M\u00e9ndez Salinas","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011840.ref018","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-94463-0","volume-title":"Neural Networks and Deep Learning.","author":"CC Aggarwal","year":"2018","edition":"1"},{"key":"pcbi.1011840.ref019","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/B978-1-4832-1448-1.50014-7","article-title":"The evolution of learning: An experiment in genetic connectionism.","author":"DJ Chalmers","year":"1991","journal-title":"Connect Model"},{"key":"pcbi.1011840.ref020","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/fnhum.2020.00050","article-title":"Neural population dynamics and cognitive function","volume":"14","author":"SE Nadeau","year":"2020","journal-title":"Front Hum Neurosci"},{"key":"pcbi.1011840.ref021","doi-asserted-by":"crossref","first-page":"306","DOI":"10.7551\/mitpress\/3115.003.0042","volume-title":"From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior","author":"PM Todd","year":"1991"},{"key":"pcbi.1011840.ref022","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1177\/105971239600500104","article-title":"Learning to adapt to changing environments in evolving neural networks.","volume":"5","author":"S Nolfi","year":"1997","journal-title":"Adapt Behav"},{"key":"pcbi.1011840.ref023","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1007\/s12065-007-0002-4","article-title":"Neuroevolution: From architectures to learning.","volume":"1","author":"D Floreano","year":"2008","journal-title":"Evol Intell"},{"key":"pcbi.1011840.ref024","first-page":"8","article-title":"On the relationships between generative encodings, regularity, and learning abilities when evolving plastic artificial neural networks","author":"P Tonelli","year":"2013","journal-title":"PLoS One"},{"key":"pcbi.1011840.ref025","first-page":"2131","article-title":"Investigating the evolution of a neuroplasticity network for learning. IEEE Trans Syst Man","volume":"49","author":"L Wang","year":"2019","journal-title":"Cybern Syst"},{"key":"pcbi.1011840.ref026","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.cosrev.2009.03.005","article-title":"Reservoir computing approaches to recurrent neural network training","volume":"3","author":"M Luko\u0161evi\u010dius","year":"2009","journal-title":"Comput Sci Rev"},{"key":"pcbi.1011840.ref027","first-page":"374","article-title":"Evolutionary aspects of reservoir computing","author":"LF Seoane","year":"2019","journal-title":"Philos Trans R Soc B Biol Sci."},{"key":"pcbi.1011840.ref028","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1016\/j.conb.2012.06.004","article-title":"Pavlovian valuation systems in learning and decision making","volume":"22","author":"JJ Clark","year":"2012","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011840.ref029","doi-asserted-by":"crossref","first-page":"23","DOI":"10.31887\/DCNS.2016.18.1\/wschultz","article-title":"Dopamine reward prediction error coding.","author":"W. Schultz","year":"2016","journal-title":"Dialogues Clin Neurosci"},{"key":"pcbi.1011840.ref030","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1023\/A:1008973931182","article-title":"Learning and evolution","volume":"7","author":"S Nolfi","year":"1999","journal-title":"Auton Robots"},{"key":"pcbi.1011840.ref031","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1006\/jtbi.2003.3146","article-title":"Carving the cognitive niche: Optimal learning strategies in homogeneous and heterogeneous environments","volume":"220","author":"B Kerr","year":"2003","journal-title":"J Theor Biol"},{"key":"pcbi.1011840.ref032","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1111\/j.2006.0030-1299.15462.x","article-title":"Exploration or exploitation: life expectancy changes the value of learning in foraging strategies","author":"S Eliassen","year":"2007","journal-title":"Oikos"},{"key":"pcbi.1011840.ref033","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-51652-5","article-title":"Need for speed: Short lifespan selects for increased learning ability","volume":"9","author":"J Liedtke","year":"2019","journal-title":"Sci Rep"},{"key":"pcbi.1011840.ref034","volume-title":"Neural Networks & Animal Behaviour. Princeton and Oxford","author":"M Enquist","year":"2005"},{"key":"pcbi.1011840.ref035","doi-asserted-by":"crossref","first-page":"1190","DOI":"10.1111\/j.1460-9568.2011.07986.x","article-title":"Surprise! Neural correlates of Pearce-Hall and Rescorla-Wagner coexist within the brain","volume":"35","author":"MR Roesch","year":"2012","journal-title":"Eur J Neurosci"},{"key":"pcbi.1011840.ref036","article-title":"The joint evolution of movement and competition strategies","author":"PR Gupte","year":"2023","journal-title":"Am Nat"},{"key":"pcbi.1011840.ref037","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1073\/pnas.1408589111","article-title":"Evolutionary tipping points in the capacity to adapt to environmental change","volume":"112","author":"CA Botero","year":"2014","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1011840.ref038","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1086\/605370","article-title":"Quantifying the adaptive value of learning in foraging behavior","volume":"174","author":"S Eliassen","year":"2009","journal-title":"Am Nat"},{"key":"pcbi.1011840.ref039","first-page":"534","article-title":"Rogers\u2019 paradox recast and resolved: Population structure and the evolution of social learning strategies","volume":"64","author":"L Rendell","year":"2010","journal-title":"Evolution (N Y)."},{"key":"pcbi.1011840.ref040","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.tics.2013.12.012","article-title":"The evolution of decision rules in complex environments","volume":"18","author":"TW Fawcett","year":"2014","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011840.ref041","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/S0022-5193(85)80219-8","article-title":"Optimal foraging and learning","volume":"117","author":"JM McNamara","year":"1985","journal-title":"J Theor Biol"},{"key":"pcbi.1011840.ref042","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1016\/S0022-5193(87)80209-6","article-title":"Memory and the efficient use of information","volume":"125","author":"JM McNamara","year":"1987","journal-title":"J Theor Biol"},{"key":"pcbi.1011840.ref043","doi-asserted-by":"crossref","first-page":"R231","DOI":"10.1016\/j.cub.2019.02.034","article-title":"Neural network models and deep learning\u2014a primer for biologists","volume":"29","author":"N Kriegeskorte","year":"2019","journal-title":"Curr Biol"},{"key":"pcbi.1011840.ref044","first-page":"487","article-title":"Interactions Between Learning and Evolution","author":"D Ackley","year":"1992","journal-title":"Artificial Life II, SFI Studies in the Sciences of Complexity"},{"key":"pcbi.1011840.ref045","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1007\/s10682-009-9336-y","article-title":"Behavioural plasticity: An interaction between evolution and experience","volume":"24","author":"F Mery","year":"2010","journal-title":"Evol Ecol"},{"key":"pcbi.1011840.ref046","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1146\/annurev.ento.53.103106.093343","article-title":"Evolutionary biology of insect learning","volume":"53","author":"R. Dukas","year":"2008","journal-title":"Annu Rev Entomol"},{"key":"pcbi.1011840.ref047","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1007\/s10144-009-0174-0","article-title":"Evolutionary ecology of learning: Insights from fruit flies","volume":"52","author":"TJ Kawecki","year":"2010","journal-title":"Popul Ecol"},{"key":"pcbi.1011840.ref048","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.cobeha.2016.09.010","article-title":"Reliability, uncertainty, and costs in the evolution of animal learning","volume":"12","author":"AS Dunlap","year":"2016","journal-title":"Curr Opin Behav Sci"},{"key":"pcbi.1011840.ref049","doi-asserted-by":"crossref","DOI":"10.1016\/S0065-3454(08)60046-7","article-title":"Selective costs and benefits in the evolution of learning","author":"TD Johnston","year":"1982","journal-title":"Advances in the Study of Behavior"},{"key":"pcbi.1011840.ref050","first-page":"9","article-title":"Brain development and the role of experience in the early years","volume":"30","author":"AL Tierney","year":"2009","journal-title":"Zero Three"},{"key":"pcbi.1011840.ref051","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1093\/beheco\/2.1.77","article-title":"Chnage, regularity, and value in the evolution of animal learning","volume":"2","author":"DW Stephens","year":"1991","journal-title":"Behav Ecol"},{"key":"pcbi.1011840.ref052","doi-asserted-by":"crossref","first-page":"842","DOI":"10.1093\/beheco\/arv184","article-title":"Environmental complexity favors the evolution of learning","volume":"27","author":"S Dridi","year":"2016","journal-title":"Behav Ecol"},{"key":"pcbi.1011840.ref053","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1023\/A:1006746727151","article-title":"Implementing behaviour in individual-based models using neural networks and genetic algorithms","volume":"13","author":"G Huse","year":"1999","journal-title":"Evol Ecol"},{"key":"pcbi.1011840.ref054","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1007\/s13218-012-0204-5","article-title":"Reservoir computing trends.","volume":"26","author":"M Luko\u0161evi\u010dius","year":"2012","journal-title":"KI\u2014K\u00fcnstliche Intelligenz"},{"key":"pcbi.1011840.ref055","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pcbi.1004128","article-title":"Neural modularity helps organisms evolve to learn new skills without forgetting old skills","volume":"11","author":"KO Ellefsen","year":"2015","journal-title":"PLoS Comput Biol"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011840","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T00:00:00Z","timestamp":1707436800000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011840","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T13:38:27Z","timestamp":1707485907000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011840"}},"subtitle":[],"editor":[{"given":"Xingru","family":"Chen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,1,30]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1,30]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011840","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.08.18.553831","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,30]]}}}