{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T19:52:05Z","timestamp":1791921125689,"version":"4.3.4"},"reference-count":93,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T00:00:00Z","timestamp":1791849600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T00:00:00Z","timestamp":1791849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203139\/Z\/16\/Z"],"award-info":[{"award-number":["203139\/Z\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203139\/A\/16\/Z"],"award-info":[{"award-number":["203139\/A\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["214251\/Z\/18\/Z"],"award-info":[{"award-number":["214251\/Z\/18\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203139\/Z\/16\/Z"],"award-info":[{"award-number":["203139\/Z\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004440","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203139\/A\/16\/Z"],"award-info":[{"award-number":["203139\/A\/16\/Z"]}],"id":[{"id":"10.13039\/100004440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"MSIP | Institute for Information and communications Technology Promotion","doi-asserted-by":"publisher","award":["MSIT 426 2019-0-01371"],"award-info":[{"award-number":["MSIT 426 2019-0-01371"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000646","name":"Japan Society for the Promotion of Science London","doi-asserted-by":"publisher","award":["22H04998"],"award-info":[{"award-number":["22H04998"]}],"id":[{"id":"10.13039\/501100000646","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Commun Psychol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Survival depends not only on maximising rewards but on minimising harm. While Reinforcement Learning (RL) provides a normative framework for adaptive decision-making, standard algorithms often fail to account for the catastrophic costs of errors during learning, in the same way as animals do. In this review, we synthesise recent advances connecting the engineering of safe RL with behavioural and systems neuroscience. We adopt a principled taxonomy distinguishing between two strategies: (1) modifying the optimisation criterion to embed safety into the learning objective, and (2) modifying the exploration process to prevent dangerous actions. We show that engineering solutions, such as worst-case optimisation, risk-sensitive objectives (e.g., conditional value at risk, CVaR), and constrained Markov decision processes (MDPs), offer precise computational accounts for anxiety, risk-aversion, and defensive reflexes. Conversely, biological mechanisms, including innate priors, interoception, and meta-control based on uncertainty, reliability and competing needs, offer robust solutions to the problem of safe exploration in artificial agents. By bridging these domains, we highlight a converging computational landscape where safety is achieved through a dynamic interplay between reactive instincts and deliberative planning.<\/jats:p>","DOI":"10.1038\/s44271-026-00542-4","type":"journal-article","created":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T10:52:22Z","timestamp":1791888742000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Natural and engineered safe reinforcement learning"],"prefix":"10.1038","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-2507-5450","authenticated-orcid":false,"given":"Pranav","family":"Mahajan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ben","family":"Seymour","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,10,13]]},"reference":[{"key":"542_CR1","unstructured":"Mahajan, P. Safe Learning in Humans and Machines. Ph.D. thesis, University of Oxford (United Kingdom) (2025)."},{"key":"542_CR2","first-page":"1437","volume":"16","author":"J Garc\u0131a","year":"2015","unstructured":"Garc\u0131a, J. & Fern\u00e1ndez, F. A comprehensive survey on safe reinforcement learning. J. Mach. Learn. Res. 16, 1437\u20131480 (2015).","journal-title":"J. Mach. Learn. Res."},{"key":"542_CR3","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1177\/2167702614562040","volume":"3","author":"QJ Huys","year":"2015","unstructured":"Huys, Q. J., Guitart-Masip, M., Dolan, R. J. & Dayan, P. Decision-theoretic psychiatry. Clin. Psychol. Sci. 3, 400\u2013421 (2015).","journal-title":"Clin. Psychol. Sci."},{"key":"542_CR4","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1101\/lm.039180.115","volume":"22","author":"JN Perusini","year":"2015","unstructured":"Perusini, J. N. & Fanselow, M. S. Neurobehavioral perspectives on the distinction between fear and anxiety. Learn. Mem. 22, 417\u2013425 (2015).","journal-title":"Learn. Mem."},{"key":"542_CR5","doi-asserted-by":"crossref","unstructured":"Schulz, E. & Dayan, P. Computational psychiatry for computers. Iscience 23, 101772 (2020).","DOI":"10.1016\/j.isci.2020.101772"},{"key":"542_CR6","unstructured":"Fanselow, M. S. & Lester, L. S. A functional behavioristic approach to aversively motivated behavior:: Predatory imminence as a determinant of the topography of defensive behavior. In Evolution and Learning (Psychology Press, 2013)."},{"key":"542_CR7","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.tics.2019.12.016","volume":"24","author":"D Mobbs","year":"2020","unstructured":"Mobbs, D., Headley, D. B., Ding, W. & Dayan, P. Space, time, and fear: survival computations along defensive circuits. Trends Cogn. Sci. 24, 228\u2013241 (2020).","journal-title":"Trends Cogn. Sci."},{"key":"542_CR8","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.tics.2019.01.012","volume":"23","author":"DA Evans","year":"2019","unstructured":"Evans, D. A., Stempel, A. V., Vale, R. & Branco, T. Cognitive control of escape behaviour. Trends Cogn. Sci. 23, 334\u2013348 (2019).","journal-title":"Trends Cogn. Sci."},{"key":"542_CR9","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1038\/s41583-023-00699-5","volume":"24","author":"B Seymour","year":"2023","unstructured":"Seymour, B., Crook, R. J. & Chen, Z. S. Post-injury pain and behaviour: a control theory perspective. Nat. Rev. Neurosci. 24, 378\u2013392, https:\/\/doi.org\/10.1038\/s41583-023-00699-5 (2023).","journal-title":"Nat. Rev. Neurosci."},{"key":"542_CR10","unstructured":"Watkins, C. J. C. H. Learning from delayed rewards. (1989)."},{"key":"542_CR11","unstructured":"Sutton, R. S. & Barto, A. G.Reinforcement learning: An introduction (MIT press, 2018)."},{"key":"542_CR12","unstructured":"Dickinson, A. & Balleine, B. The role of learning in motivation in gallistel cr (ed.), steven\u2019s handbook of experimental psychology: Learning, motivation and emotion (vol. 3, pp. 497\u2013534) (2002)."},{"key":"542_CR13","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","volume":"8","author":"ND Daw","year":"2005","unstructured":"Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704\u20131711 (2005).","journal-title":"Nat. Neurosci."},{"key":"542_CR14","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","volume":"81","author":"SW Lee","year":"2014","unstructured":"Lee, S. W., Shimojo, S. & O\u2019Doherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687\u2013699 (2014).","journal-title":"Neuron"},{"key":"542_CR15","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1145\/122344.122377","volume":"2","author":"RS Sutton","year":"1991","unstructured":"Sutton, R. S. Dyna, an integrated architecture for learning, planning, and reacting. ACM Sigart Bull. 2, 160\u2013163 (1991).","journal-title":"ACM Sigart Bull."},{"key":"542_CR16","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1162\/neco.1993.5.4.613","volume":"5","author":"P Dayan","year":"1993","unstructured":"Dayan, P. Improving generalization for temporal difference learning: the successor representation. Neural Comput. 5, 613\u2013624 (1993).","journal-title":"Neural Comput."},{"key":"542_CR17","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1038\/s41562-017-0180-8","volume":"1","author":"I Momennejad","year":"2017","unstructured":"Momennejad, I. et al. The successor representation in human reinforcement learning. Nat. Hum. Behav. 1, 680\u2013692 (2017).","journal-title":"Nat. Hum. Behav."},{"key":"542_CR18","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1016\/j.neunet.2006.03.002","volume":"19","author":"P Dayan","year":"2006","unstructured":"Dayan, P., Niv, Y., Seymour, B. & Daw, N. D. The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153\u20131160 (2006).","journal-title":"Neural Netw."},{"key":"542_CR19","unstructured":"Lengyel, M. & Dayan, P. Hippocampal contributions to control: the third way. Advances in neural information processing systems20 (2007)."},{"key":"542_CR20","doi-asserted-by":"crossref","first-page":"RP101371","DOI":"10.7554\/eLife.101371.3","volume":"13","author":"P Mahajan","year":"2025","unstructured":"Mahajan, P., Tong, S., Lee, S. W. & Seymour, B. Balancing safety and efficiency in human decision-making. Elife 13, RP101371 (2025).","journal-title":"Elife"},{"key":"542_CR21","doi-asserted-by":"crossref","first-page":"e1013538","DOI":"10.1371\/journal.pcbi.1013538","volume":"22.1","author":"P Mahajan","year":"2026","unstructured":"Mahajan, P., Dayan, P. & Seymour, B. Homeostasis after injury: How intertwined inference and control underpin post-injury pain and behaviour. PLOS Comput. Biol. 22, e1013538 (2026).","journal-title":"PLOS Comput. Biol."},{"key":"542_CR22","doi-asserted-by":"crossref","first-page":"14431","DOI":"10.1073\/pnas.1003111107","volume":"107","author":"CJ Burke","year":"2010","unstructured":"Burke, C. J., Tobler, P. N., Baddeley, M. & Schultz, W. Neural mechanisms of observational learning. Proc. Natl. Acad. Sci. 107, 14431\u201314436 (2010).","journal-title":"Proc. Natl. Acad. Sci."},{"key":"542_CR23","doi-asserted-by":"crossref","first-page":"541","DOI":"10.1016\/j.tics.2024.01.006","volume":"28","author":"AF Hall","year":"2024","unstructured":"Hall, A. F., Browning, M. & Huys, Q. J. The computational structure of consummatory anhedonia. Trends Cogn. Sci. 28, 541\u2013553 (2024).","journal-title":"Trends Cogn. Sci."},{"key":"542_CR24","doi-asserted-by":"crossref","unstructured":"Huys, Q. J. & Browning, M. A computational view on the nature of reward and value in anhedonia. In Anhedonia: Preclinical, Translational, and Clinical Integration, 421\u2013441 (Springer, 2021).","DOI":"10.1007\/7854_2021_290"},{"key":"542_CR25","doi-asserted-by":"crossref","unstructured":"Heger, M. Consideration of risk in reinforcement learning. In Machine Learning Proceedings 1994, 105\u2013111 (Elsevier, 1994).","DOI":"10.1016\/B978-1-55860-335-6.50021-0"},{"key":"542_CR26","unstructured":"Gaskett, C. Reinforcement learning under circumstances beyond its control. (2003)."},{"key":"542_CR27","unstructured":"Tamar, A., Mannor, S., & Xu, H. (2014, June). Scaling up robust MDPs using function approximation. In International conference on machine learning (pp. 181\u2013189). PMLR."},{"key":"542_CR28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/CPSY_a_00026","volume":"4","author":"S Zorowitz","year":"2020","unstructured":"Zorowitz, S., Momennejad, I. & Daw, N. D. Anxiety, avoidance, and sequential evaluation. Comput. Psychiatry 4, 1\u201317 (2020).","journal-title":"Comput. Psychiatry"},{"key":"542_CR29","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1287\/mnsc.18.7.356","volume":"18","author":"RA Howard","year":"1972","unstructured":"Howard, R. A. & Matheson, J. E. Risk-sensitive markov decision processes. Manag. Sci. 18, 356\u2013369 (1972).","journal-title":"Manag. Sci."},{"key":"542_CR30","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1287\/moor.27.2.294.324","volume":"27","author":"VS Borkar","year":"2002","unstructured":"Borkar, V. S. Q-learning for risk-sensitive control. Math. Oper. Res. 27, 294\u2013311 (2002).","journal-title":"Math. Oper. Res."},{"key":"542_CR31","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1023\/A:1017940631555","volume":"49","author":"O Mihatsch","year":"2002","unstructured":"Mihatsch, O. & Neuneier, R. Risk-sensitive reinforcement learning. Mach. Learn. 49, 267\u2013290 (2002).","journal-title":"Mach. Learn."},{"key":"542_CR32","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1523\/JNEUROSCI.5498-10.2012","volume":"32","author":"Y Niv","year":"2012","unstructured":"Niv, Y., Edlund, J. A., Dayan, P. & O\u2019Doherty, J. P. Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. J. Neurosci. 32, 551\u2013562 (2012).","journal-title":"J. Neurosci."},{"key":"542_CR33","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1613\/jair.1666","volume":"24","author":"P Geibel","year":"2005","unstructured":"Geibel, P. & Wysotzki, F. Risk-sensitive reinforcement learning applied to control under constraints. J. Artif. Intell. Res. 24, 81\u2013108 (2005).","journal-title":"J. Artif. Intell. Res."},{"key":"542_CR34","unstructured":"Morimura, T., Sugiyama, M., Kashima, H., Hachiya, H. & Tanaka, T. Parametric return density estimation for reinforcement learning. arXiv preprint arXiv:1203.3497 (2012)."},{"key":"542_CR35","doi-asserted-by":"crossref","unstructured":"Tamar, A., Glassner, Y. & Mannor, S. Optimizing the cvar via sampling. In Twenty-Ninth AAAI Conference on Artificial Intelligence (2015).","DOI":"10.1609\/aaai.v29i1.9561"},{"key":"542_CR36","first-page":"22209","volume":"34","author":"C Gagne","year":"2021","unstructured":"Gagne, C. & Dayan, P. Two steps to risk sensitivity. Adv. Neural Inf. Process. Syst. 34, 22209\u201322220 (2021).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"542_CR37","unstructured":"Chow, Y., Tamar, A., Mannor, S. & Pavone, M. Risk-sensitive and robust decision-making: a cvar optimization approach. Advances in neural information processing systems28 (2015)."},{"key":"542_CR38","doi-asserted-by":"crossref","unstructured":"Shapiro, A., Dentcheva, D. & Ruszczynski, A.Lectures on stochastic programming: modeling and theory (SIAM, 2021).","DOI":"10.1137\/1.9781611976595"},{"key":"542_CR39","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","volume":"69","author":"ND Daw","year":"2011","unstructured":"Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans\u2019 choices and striatal prediction errors. Neuron 69, 1204\u20131215 (2011).","journal-title":"Neuron"},{"key":"542_CR40","doi-asserted-by":"crossref","first-page":"102617","DOI":"10.1016\/j.jmp.2021.102617","volume":"106","author":"C Gagne","year":"2022","unstructured":"Gagne, C. & Dayan, P. Peril, prudence and planning as risk, avoidance and worry. J. Math. Psychol. 106, 102617 (2022).","journal-title":"J. Math. Psychol."},{"key":"542_CR41","doi-asserted-by":"publisher","first-page":"1609","DOI":"10.1038\/s41593-018-0232-z","volume":"21","author":"MG Mattar","year":"2018","unstructured":"Mattar, M. G. & Daw, N. D. Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21, 1609\u20131617 (2018).","journal-title":"Nat. Neurosci."},{"key":"542_CR42","doi-asserted-by":"crossref","unstructured":"Shen, T. & Dayan, P. Risking your tail: Modeling individual differences in risk-sensitive exploration using bayes adaptive markov decision processes. eLife13 (2024).","DOI":"10.7554\/eLife.100366.3"},{"key":"542_CR43","unstructured":"Sui, X., Dayan, P. & Lloyd, K. Exploring optimal risk-sensitive behavior in the balloon analogue risk task (bart). In Computational Psychiatry Conference 2023 (2023)."},{"key":"542_CR44","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1038\/s41586-019-1924-6","volume":"577","author":"W Dabney","year":"2020","unstructured":"Dabney, W. et al. A distributional code for value in dopamine-based reinforcement learning. Nature 577, 671\u2013675 (2020).","journal-title":"Nature"},{"key":"542_CR45","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1038\/s41593-023-01535-w","volume":"27","author":"TH Muller","year":"2024","unstructured":"Muller, T. H. et al. Distributional reinforcement learning in prefrontal cortex. Nat. Neurosci. 27, 403\u2013408 (2024).","journal-title":"Nat. Neurosci."},{"key":"542_CR46","first-page":"13662","volume":"33","author":"P Tano","year":"2020","unstructured":"Tano, P., Dayan, P. & Pouget, A. A local temporal difference code for distributional reinforcement learning. Adv. Neural Inf. Process. Syst. 33, 13662\u201313673 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"542_CR47","unstructured":"Altman, E. Constrained Markov decision processes, vol. 7 (CRC Press, 1999)."},{"key":"542_CR48","unstructured":"Achiam, J., Held, D., Tamar, A. & Abbeel, P. Constrained policy optimization. In International Conference on Machine Learning, 22\u201331 (PMLR, 2017)."},{"key":"542_CR49","doi-asserted-by":"crossref","first-page":"e2221180120","DOI":"10.1073\/pnas.2221180120","volume":"120","author":"Z Dulberg","year":"2023","unstructured":"Dulberg, Z., Dubey, R., Berwian, I. M. & Cohen, J. D. Having multiple selves helps learning agents explore and adapt in complex changing worlds. Proc. Natl. Acad. Sci. USA 120, e2221180120 (2023).","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"542_CR50","doi-asserted-by":"crossref","unstructured":"Dulberg, Z. & Cohen, J. D. On the duality of pain and pleasure processing: Why two dimensions of valence may be better than one. bioRxiv 2025\u201301 (2025).","DOI":"10.1101\/2025.01.22.634365"},{"key":"542_CR51","doi-asserted-by":"crossref","unstructured":"Elfwing, S. & Seymour, B. Parallel reward and punishment control in humans and robots: Safe reinforcement learning using the maxpain algorithm. In 2017 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 140\u2013147 (IEEE, 2017).","DOI":"10.1109\/DEVLRN.2017.8329799"},{"key":"542_CR52","doi-asserted-by":"crossref","unstructured":"Gehring, C. & Precup, D. Smart exploration in reinforcement learning using absolute temporal difference errors. In Proceedings of the 2013 international conference on Autonomous agents and multi-agent systems, 1037\u20131044 (2013).","DOI":"10.65109\/PWAM3332"},{"key":"542_CR53","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1016\/j.cognition.2008.11.014","volume":"110","author":"KA Krueger","year":"2009","unstructured":"Krueger, K. A. & Dayan, P. Flexible shaping: how learning in small steps helps. Cognition 110, 380\u2013394, https:\/\/doi.org\/10.1016\/j.cognition.2008.11.014 (2009).","journal-title":"Cognition"},{"key":"542_CR54","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-13737-7","volume":"10","author":"HM Dorfman","year":"2019","unstructured":"Dorfman, H. M. & Gershman, S. J. Controllability governs the balance between Pavlovian and instrumental action selection. Nat. Commun. 10, 1\u20138 (2019).","journal-title":"Nat. Commun."},{"key":"542_CR55","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.cub.2015.10.066","volume":"26","author":"S Zhang","year":"2016","unstructured":"Zhang, S., Mano, H., Ganesh, G., Robbins, T. & Seymour, B. Dissociable learning processes underlie human pain conditioning. Curr. Biol. 26, 52\u201358 (2016).","journal-title":"Curr. Biol."},{"key":"542_CR56","doi-asserted-by":"crossref","first-page":"e31949","DOI":"10.7554\/eLife.31949","volume":"7","author":"S Zhang","year":"2018","unstructured":"Zhang, S. et al. The control of tonic pain by active relief learning. Elife 7, e31949 (2018).","journal-title":"Elife"},{"key":"542_CR57","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/S0304-3959(99)00242-0","volume":"85","author":"JW Vlaeyen","year":"2000","unstructured":"Vlaeyen, J. W. & Linton, S. J. Fear-avoidance and its consequences in chronic musculoskeletal pain: a state of the art. Pain 85, 317\u2013332 (2000).","journal-title":"Pain"},{"key":"542_CR58","doi-asserted-by":"crossref","first-page":"1588","DOI":"10.1097\/j.pain.0000000000000574","volume":"157","author":"JW Vlaeyen","year":"2016","unstructured":"Vlaeyen, J. W., Crombez, G. & Linton, S. J. The fear-avoidance model of pain. Pain 157, 1588\u20131589 (2016).","journal-title":"Pain"},{"key":"542_CR59","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1038\/s41386-021-01123-1","volume":"47","author":"A Soltani","year":"2022","unstructured":"Soltani, A. & Koechlin, E. Computational models of adaptive behavior and prefrontal cortex. Neuropsychopharmacology 47, 58\u201371 (2022).","journal-title":"Neuropsychopharmacology"},{"key":"542_CR60","doi-asserted-by":"crossref","unstructured":"Woo, J. H. et al. Contribution of amygdala to dynamic model arbitration under uncertainty. Nature Communications (2025).","DOI":"10.1101\/2024.09.13.612869"},{"key":"542_CR61","doi-asserted-by":"crossref","first-page":"e1012580","DOI":"10.1371\/journal.pcbi.1012580","volume":"20","author":"B Millidge","year":"2024","unstructured":"Millidge, B., Song, Y., Lak, A., Walton, M. E. & Bogacz, R. Reward bases: a simple mechanism for adaptive acquisition of multiple reward types. PLOS Comput. Biol. 20, e1012580 (2024).","journal-title":"PLOS Comput. Biol."},{"key":"542_CR62","doi-asserted-by":"crossref","unstructured":"Mahajan, P. & Seymour, B. Composing the value signal for dopamine-mediated learning. (2025).","DOI":"10.1101\/2025.10.10.681616"},{"key":"542_CR63","doi-asserted-by":"crossref","unstructured":"Lin, B., Bouneffouf, D. & Cecchi, G. Split q learning: reinforcement learning with two-stream rewards. arXiv preprint arXiv:1906.12350 (2019).","DOI":"10.24963\/ijcai.2019\/913"},{"key":"542_CR64","doi-asserted-by":"crossref","unstructured":"Wang, J., Elfwing, S. & Uchibe, E. Deep reinforcement learning by parallelizing reward and punishment using the maxpain architecture. In 2018 Joint IEEE 8th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), 175\u2013180 (IEEE, 2018).","DOI":"10.1109\/DEVLRN.2018.8761044"},{"key":"542_CR65","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.neunet.2020.12.001","volume":"135","author":"J Wang","year":"2021","unstructured":"Wang, J., Elfwing, S. & Uchibe, E. Modular deep reinforcement learning from reward and punishment for robot navigation. Neural Netw. 135, 115\u2013126 (2021).","journal-title":"Neural Netw."},{"key":"542_CR66","unstructured":"Maire, F. & Bulitko, V. Apprenticeship learning for initial value functions in reinforcement learning. Planning and Learning in A Priori Unknown or Dynamic Domains 23 (2005)."},{"key":"542_CR67","unstructured":"Wang, J. X. et al. Learning to reinforcement learn. arXiv preprint arXiv:1611.05763 (2016)."},{"key":"542_CR68","doi-asserted-by":"publisher","first-page":"1421","DOI":"10.1038\/s41593-018-0222-1","volume":"21","author":"W Menegas","year":"2018","unstructured":"Menegas, W., Akiti, K., Amo, R., Uchida, N. & Watabe-Uchida, M. Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli. Nat. Neurosci. 21, 1421\u20131430 (2018).","journal-title":"Nat. Neurosci."},{"key":"542_CR69","doi-asserted-by":"crossref","first-page":"3789","DOI":"10.1016\/j.neuron.2022.08.022","volume":"110","author":"K Akiti","year":"2022","unstructured":"Akiti, K. et al. Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction. Neuron 110, 3789\u20133804 (2022).","journal-title":"Neuron"},{"key":"542_CR70","doi-asserted-by":"crossref","unstructured":"Tsutsui-Kimura, I. et al. Dopamine in the tail of the striatum facilitates avoidance in threat\u2013reward conflicts. Nat. Neurosci. 1\u201316 (2025).","DOI":"10.1038\/s41593-025-01902-9"},{"key":"542_CR71","doi-asserted-by":"crossref","unstructured":"Mahajan, P. & Seymour, B. Entropy regularised reinforcement learning reconciles aversive and action prediction errors in the tail of the striatum. bioRxiv 2026\u201307 (2026).","DOI":"10.64898\/2026.07.09.737461"},{"key":"542_CR72","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1016\/S0893-6080(02)00048-5","volume":"15","author":"S Kakade","year":"2002","unstructured":"Kakade, S. & Dayan, P. Dopamine: generalization and bonuses. Neural Netw. 15, 549\u2013559 (2002).","journal-title":"Neural Netw."},{"key":"542_CR73","doi-asserted-by":"crossref","first-page":"101441","DOI":"10.1016\/j.cobeha.2024.101441","volume":"59","author":"I Green","year":"2024","unstructured":"Green, I., Amo, R. & Watabe-Uchida, M. Shifting attention to orient or avoid: a unifying account of the tail of the striatum and its dopaminergic inputs. Curr. Opin. Behav. Sci. 59, 101441 (2024).","journal-title":"Curr. Opin. Behav. Sci."},{"key":"542_CR74","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1523\/JNEUROSCI.3692-12.2013","volume":"33","author":"V Miskovic","year":"2013","unstructured":"Miskovic, V. & Keil, A. Perceiving threat in the face of safety: excitation and inhibition of conditioned fear in human visual cortex. J. Neurosci. 33, 72\u201378 (2013).","journal-title":"J. Neurosci."},{"key":"542_CR75","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms8823","volume":"6","author":"LM McTeague","year":"2015","unstructured":"McTeague, L. M., Gruss, L. F. & Keil, A. Aversive learning shapes neuronal orientation tuning in human visual cortex. Nat. Commun. 6, 7823 (2015).","journal-title":"Nat. Commun."},{"key":"542_CR76","doi-asserted-by":"crossref","unstructured":"Dayan, P., Roiser, J. & Viding, E. The first steps on long marches: The costs of active observation. In Psychiatry Reborn: Biopsychosocial psychiatry in modern medicine, 213\u2013228 (Oxford University Press, 2020).","DOI":"10.1093\/med\/9780198789697.003.0014"},{"key":"542_CR77","doi-asserted-by":"crossref","unstructured":"Pezzato, C. et al. Active inference for fault tolerant control of robot manipulators with sensory faults. In International Workshop on Active Inference, 20\u201327 (Springer, 2020).","DOI":"10.1007\/978-3-030-64919-7_3"},{"key":"542_CR78","doi-asserted-by":"crossref","unstructured":"Pastor, P., Kalakrishnan, M., Righetti, L. & Schaal, S. Towards associative skill memories. In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), 309\u2013315 (IEEE, 2012).","DOI":"10.1109\/HUMANOIDS.2012.6651537"},{"key":"542_CR79","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/j.robot.2012.09.017","volume":"61","author":"P Pastor","year":"2013","unstructured":"Pastor, P. et al. From dynamic movement primitives to associative skill memories. Robot. Autonomous Syst. 61, 351\u2013361 (2013).","journal-title":"Robot. Autonomous Syst."},{"key":"542_CR80","unstructured":"Lee, S. et al. Life-inspired interoceptive artificial intelligence for autonomous and adaptive agents. arXiv preprint arXiv:2309.05999 (2023)."},{"key":"542_CR81","doi-asserted-by":"crossref","first-page":"e04811","DOI":"10.7554\/eLife.04811","volume":"3","author":"M Keramati","year":"2014","unstructured":"Keramati, M. & Gutkin, B. Homeostatic reinforcement learning for integrating reward collection and physiological stability. Elife 3, e04811 (2014).","journal-title":"Elife"},{"key":"542_CR82","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1016\/j.neubiorev.2018.04.017","volume":"90","author":"AP Owens","year":"2018","unstructured":"Owens, A. P., Allen, M., Ondobaka, S. & Friston, K. J. Interoceptive inference: From computational neuroscience to clinic. Neurosci. Biobehav. Rev. 90, 174\u2013183 (2018).","journal-title":"Neurosci. Biobehav. Rev."},{"key":"542_CR83","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO.a.1475","volume":"38","author":"P Mahajan","year":"2025","unstructured":"Mahajan, P., Tang, M., Li, T. E., Havoutis, I. & Seymour, B. Neural associative skill memories for safer robotics and modeling human sensorimotor repertoires. Neural Comput. 38, 1\u201327 (2025).","journal-title":"Neural Comput."},{"key":"542_CR84","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1011183","volume":"20","author":"B Millidge","year":"2024","unstructured":"Millidge, B., Tang, M., Osanlouy, M., Harper, N. S. & Bogacz, R. Predictive coding networks for temporal prediction. PLOS Comput. Biol. 20, 1\u201331, https:\/\/doi.org\/10.1371\/journal.pcbi.1011183 (2024).","journal-title":"PLOS Comput. Biol."},{"key":"542_CR85","first-page":"440","volume":"2","author":"D Ha","year":"2018","unstructured":"Ha, D. & Schmidhuber, J. World models. arXiv Prepr. arXiv:1803. 10122 2, 440 (2018).","journal-title":"arXiv Prepr. arXiv:1803. 10122"},{"key":"542_CR86","unstructured":"Goldstein, N. et al. A parabrachial hub for need-state control of enduring pain. Nature 1\u20139 (2025)."},{"key":"542_CR87","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1037\/a0037015","volume":"121","author":"AG Collins","year":"2014","unstructured":"Collins, A. G. & Frank, M. J. Opponent actor learning (opal): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. Psychol. Rev. 121, 337 (2014).","journal-title":"Psychol. Rev."},{"key":"542_CR88","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","volume":"275","author":"W Schultz","year":"1997","unstructured":"Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593\u20131599 (1997).","journal-title":"Science"},{"key":"542_CR89","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1038\/35044563","volume":"1","author":"W Schultz","year":"2000","unstructured":"Schultz, W. Multiple reward signals in the brain. Nat. Rev. Neurosci. 1, 199\u2013207 (2000).","journal-title":"Nat. Rev. Neurosci."},{"key":"542_CR90","doi-asserted-by":"publisher","first-page":"664","DOI":"10.1038\/nature02581","volume":"429","author":"B Seymour","year":"2004","unstructured":"Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664\u2013667, https:\/\/doi.org\/10.1038\/nature02581 (2004).","journal-title":"Nature"},{"key":"542_CR91","unstructured":"Moskovitz, T. et al. Reload: Reinforcement learning with optimistic ascent-descent for last-iterate convergence in constrained mdps. In International Conference on Machine Learning, 25303\u201325336 (PMLR, 2023)."},{"key":"542_CR92","first-page":"22384","volume":"33","author":"Y Fei","year":"2020","unstructured":"Fei, Y., Yang, Z., Chen, Y., Wang, Z. & Xie, Q. Risk-sensitive reinforcement learning: Near-optimal risk-sample tradeoff in regret. Adv. Neural Inf. Process. Syst. 33, 22384\u201322395 (2020).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"542_CR93","doi-asserted-by":"crossref","unstructured":"Ball, T. M. & Gunaydin, L. A. Measuring maladaptive avoidance: from animal models to clinical anxiety. Neuropsychopharmacology 1\u20139 (2022).","DOI":"10.31234\/osf.io\/mkax4"}],"container-title":["Communications Psychology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s44271-026-00542-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s44271-026-00542-4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s44271-026-00542-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T19:38:28Z","timestamp":1791920308000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s44271-026-00542-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,13]]},"references-count":93,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["542"],"URL":"https:\/\/doi.org\/10.1038\/s44271-026-00542-4","relation":{},"ISSN":["2731-9121"],"issn-type":[{"value":"2731-9121","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,13]]},"assertion":[{"value":"1 April 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 September 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 October 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Ethics"}}],"article-number":"138"}}