{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:18:39Z","timestamp":1750220319242,"version":"3.41.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,12,31]],"date-time":"2020-12-31T00:00:00Z","timestamp":1609372800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Research Council of Norway through its Centres of Excellence scheme","award":["262762"],"award-info":[{"award-number":["262762"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"<jats:p>Neural networks have been widely used in agent learning architectures; however, learnings for one task might nullify learnings for another. Behavioural plasticity enables humans and animals alike to respond to environmental changes without degrading learned knowledge; this can be achieved by regulating behaviour with neuromodulation\u2014a biological process found in the brain. We demonstrate that by modulating activity-propagating signals, neurally trained agents evolving to solve tasks in dynamic environments that are prone to change can expect a significantly higher fitness than non-modulatory agents and also achieve their goals more often. Further, we show that while behavioural plasticity can help agents to achieve goals in these variable environments, this ability to overcome environmental changes with greater success comes at the cost of highly volatile evolution.<\/jats:p>","DOI":"10.1145\/3487918","type":"journal-article","created":{"date-parts":[[2021,12,20]],"date-time":"2021-12-20T19:13:58Z","timestamp":1640027638000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Behavioural Plasticity Can Help Evolving Agents in Dynamic Environments but at the Cost of Volatility"],"prefix":"10.1145","volume":"15","author":[{"given":"Chloe M.","family":"Barnes","sequence":"first","affiliation":[{"name":"Aston University, Birmingham, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anik\u00f3","family":"Ek\u00e1rt","sequence":"additional","affiliation":[{"name":"Aston University, Birmingham, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai Olav","family":"Ellefsen","sequence":"additional","affiliation":[{"name":"University of Oslo, Oslo, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyrre","family":"Glette","sequence":"additional","affiliation":[{"name":"University of Oslo, Oslo, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter R.","family":"Lewis","sequence":"additional","affiliation":[{"name":"Ontario Tech University, Oshawa, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jim","family":"T\u00f8rresen","sequence":"additional","affiliation":[{"name":"University of Oslo, Oslo, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,12,20]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.87.23.9241"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/81453"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2012.2202658"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACSOS49614.2020.00034"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2019.00021"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.07.033"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0047507"},{"key":"e_1_3_1_9_2","first-page":"992","volume-title":"Proceedings of the 24th European Conference on Artificial Intelligence (ECAI)","author":"Beaulieu Shawn","year":"2020","unstructured":"Shawn Beaulieu, Lapo Frati, Thomas Miconi, Joel Lehman, Kenneth O. Stanley, Jeff Clune, and Nick Cheney. 2020. Learning to continually learn. In Proceedings of the 24th European Conference on Artificial Intelligence (ECAI). IOS Press, 992\u20131001. DOI:https:\/\/doi.org\/10.3233\/FAIA200193"},{"key":"e_1_3_1_10_2","first-page":"101","volume-title":"Advances in Artificial Life: Proceedings of the 11th European Conference on the Synthesis and Simulation of Living Systems (ECAL 2011)","author":"Borg James M.","year":"2011","unstructured":"James M. Borg, Alastair Channon, Charles Day, et al. 2011. Discovering and maintaining behaviours inaccessible to incremental genetic evolution through transcription errors and cultural transmission. In Advances in Artificial Life: Proceedings of the 11th European Conference on the Synthesis and Simulation of Living Systems (ECAL 2011). The MIT Press, 101\u2013108. DOI:https:\/\/doi.org\/10.7551\/978-978-0-262-29714-1-ch019"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1080\/15326900701399939"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISQED.2019.8697362"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1007334"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1004128"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2017.2776863"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305498"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2017.01.007"},{"key":"e_1_3_1_18_2","first-page":"1","volume-title":"Proceedings of the 28th International Conference on Architecture of Computing Systems","author":"H\u00e4hner J\u00f6rg","year":"2015","unstructured":"J\u00f6rg H\u00e4hner, Uwe Brinkschulte, Paul Lukowicz, Sanaz Mostaghim, Bernhard Sick, and Sven Tomforde. 2015. Runtime self-integration as key challenge for mastering interwoven systems. In Proceedings of the 28th International Conference on Architecture of Computing Systems. VDE, 1\u20138."},{"key":"e_1_3_1_19_2","first-page":"21","volume-title":"Cold Spring Harbor Symposia on Quantitative Biology","author":"Hamood Albert W.","year":"2014","unstructured":"Albert W. Hamood and Eve Marder. 2014. Animal-to-animal variability in neuromodulation and circuit function. In Cold Spring Harbor Symposia on Quantitative Biology, Vol. 79. Cold Spring Harbor Laboratory Press, 21\u201328. DOI:https:\/\/doi.org\/10.1101\/sqb.2014.79.024828"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1111\/jeb.13405"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1155\/2016\/4296356"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454450"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1139\/z97-023"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1177\/1059712308095775"},{"key":"e_1_3_1_26_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Lillicrap Timothy P.","year":"2016","unstructured":"Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2016. Continuous control with deep reinforcement learning. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1803839115"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0079-7421(08)60536-8"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10682-009-9336-y"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jtbi.2009.06.007"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1095-8649.2012.03477.x"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2012.00228"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/1771390.1771431"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.3390\/su5010163"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.anbehav.2012.12.031"},{"key":"e_1_3_1_37_2","volume-title":"Artificial Life XI: Proceedings of the 11th International Conference on the Simulation and Synthesis of Living Systems","author":"Soltoggio Andrea","year":"2008","unstructured":"Andrea Soltoggio, John A. Bullinaria, Claudio Mattiussi, Peter D\u00fcrr, and Dario Floreano. 2008. Evolutionary advantages of neuromodulated plasticity in dynamic, reward-based scenarios. In Artificial Life XI: Proceedings of the 11th International Conference on the Simulation and Synthesis of Living Systems. ISBNx9780262750172"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1111\/brv.12186"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-018-0006-z"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1162\/106365602320169811"},{"issue":"1","key":"e_1_3_1_41_2","article-title":"Brain-inspired replay for continual learning with artificial neural networks","volume":"11","author":"Ven Gido M. van de","year":"2020","unstructured":"Gido M. van de Ven, Hava T. Siegelmann, and Andreas S. Tolias. 2020. Brain-inspired replay for continual learning with artificial neural networks. Nat. Commun. 11, 1 (2020). ISSN20411723. DOI:https:\/\/doi.org\/10.1038\/s41467-020-17866-2","journal-title":"Nat. Commun."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0227922"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0187736"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.4161\/worm.21086"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(99)00103-3"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(99)00113-X"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.784219"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1080\/00949655.2010.520163"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.7551\/978-0-262-32621-6-ch149"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3487918","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3487918","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:56Z","timestamp":1750191116000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3487918"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,31]]},"references-count":48,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3487918"],"URL":"https:\/\/doi.org\/10.1145\/3487918","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"type":"print","value":"1556-4665"},{"type":"electronic","value":"1556-4703"}],"subject":[],"published":{"date-parts":[[2020,12,31]]},"assertion":[{"value":"2021-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-12-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}