{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,2]],"date-time":"2026-09-02T03:32:32Z","timestamp":1788319952403,"version":"build-2803163510"},"reference-count":72,"publisher":"eLife Sciences Publications, Ltd","license":[{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2025,12,1]],"date-time":"2025-12-01T00:00:00Z","timestamp":1764547200000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Max Planck Society"},{"id":[{"id":"https:\/\/ror.org\/01hhn8329","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005156","name":"Alexander von Humboldt Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100005156","id-type":"DOI","asserted-by":"crossref"}]},{"id":[{"id":"https:\/\/ror.org\/012kf4317","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"crossref","award":["Machine Learning Cluster of Excellence, EXC number 2064\/1 \u2013 Project number 39072764"],"award-info":[{"award-number":["Machine Learning Cluster of Excellence, EXC number 2064\/1 \u2013 Project number 39072764"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"award":["Machine Learning Cluster of Excellence, EXC number 2064\/1 \u2013 Project number 39072764"],"award-info":[{"award-number":["Machine Learning Cluster of Excellence, EXC number 2064\/1 \u2013 Project number 39072764"]}],"id":[{"id":"https:\/\/ror.org\/018mejw64","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003042","name":"Else Kr\u00f6ner-Fresenius-Stiftung","doi-asserted-by":"crossref","award":["ClinbrAIn: Artificial Intelligence for Clinical Brain Research"],"award-info":[{"award-number":["ClinbrAIn: Artificial Intelligence for Clinical Brain Research"]}],"id":[{"id":"10.13039\/501100003042","id-type":"DOI","asserted-by":"crossref"}]},{"award":["ClinbrAIn: Artificial Intelligence for Clinical Brain Research"],"award-info":[{"award-number":["ClinbrAIn: Artificial Intelligence for Clinical Brain Research"]}],"id":[{"id":"https:\/\/ror.org\/03zcxha54","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":["elifesciences.org"],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Novelty is a double-edged sword for agents and animals alike: they might benefit from untapped resources or face unexpected costs or dangers such as predation. The conventional exploration\/exploitation tradeoff is thus colored by risk sensitivity. A wealth of experiments has shown how animals solve this dilemma, for example, using intermittent approach. However, there are large individual differences in the nature of approach, and modeling has yet to elucidate how this might be based on animals\u2019 differing prior expectations about reward and threat, and differing degrees of risk aversion. To capture these factors, we built a Bayes-adaptive Markov decision process model with three key components: an adaptive hazard function capturing potential predation, an intrinsic reward function providing the urge to explore, and a conditional value at risk (CVaR) objective, which is a contemporary measure of trait risk sensitivity. We fit this model to a coarse-grain abstraction of the behavior of 26 animals who freely explored a novel object in an open-field arena. We show that the model captures both quantitative (frequency, duration of exploratory bouts) and qualitative (with distinguished, cautious, tail-behind approach) features of behavior, including the substantial idiosyncrasies that were observed. Some animals begin with cautious exploration and quickly transition to a confident approach to maximize exploration for reward; we classify them as potentially more risk neutral and enjoying a flexible hazard prior. By contrast, other animals only ever approach in a cautious manner and display a form of self-censoring; they are characterized by potential risk aversion and high and inflexible hazard priors. Explaining risk-sensitive exploration using factorized parameters of reinforcement learning models could aid in the understanding, diagnosis, and treatment of psychiatric abnormalities such as anxiety disorders.<\/jats:p>","DOI":"10.7554\/elife.100366","type":"journal-article","created":{"date-parts":[[2024,11,11]],"date-time":"2024-11-11T07:25:17Z","timestamp":1731309917000},"update-policy":"https:\/\/doi.org\/10.7554\/elife.100366","source":"Crossref","is-referenced-by-count":6,"title":["Individual differences in tail risk sensitive exploration using Bayes-adaptive Markov decision processes"],"prefix":"10.7554","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-0286-5663","authenticated-orcid":true,"given":"Tingke","family":"Shen","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/026nmvv73","id-type":"ROR","asserted-by":"publisher"}],"name":"Max Planck Institute for Biological Cybernetics","place":["T\u00fcbingen, Germany"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"corresponding-author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3476-1839","authenticated-orcid":true,"given":"Peter","family":"Dayan","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/026nmvv73","id-type":"ROR","asserted-by":"publisher"}],"name":"Max Planck Institute for Biological Cybernetics","place":["T\u00fcbingen, Germany"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"4374","published-online":{"date-parts":[[2025,12,1]]},"reference":[{"key":"bib1","first-page":"51","type":"journal-article","article-title":"Patterns of attachment behavior shown by the infant in interaction with his mother","volume":"10","author":"Ainsworth","year":"1964","journal-title":"Merrill-Palmer Quarterly of Behavior and Development"},{"key":"bib2","doi-asserted-by":"publisher","first-page":"3789","DOI":"10.1016\/j.neuron.2022.08.022","type":"journal-article","article-title":"Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction","volume":"110","author":"Akiti","year":"2022","journal-title":"Neuron"},{"key":"bib3","doi-asserted-by":"publisher","DOI":"10.1101\/2023.01.27.525847","type":"preprint","article-title":"Exploring replay","volume-title":"bioRxiv","author":"Antonov","year":"2023"},{"key":"bib4","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1037\/h0062815","type":"journal-article","article-title":"Young children in an insecure situation","volume":"38","author":"Arsenian","year":"1943","journal-title":"The Journal of Abnormal and Social Psychology"},{"key":"bib5","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1111\/1467-9965.00068","type":"journal-article","article-title":"Coherent measures of risk","volume":"9","author":"Artzner","year":"1999","journal-title":"Mathematical Finance"},{"key":"bib6","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/14207.001.0001","type":"book","volume-title":"Distributional Reinforcement Learning","author":"Bellemare","year":"2023"},{"key":"bib7","doi-asserted-by":"publisher","first-page":"439","DOI":"10.7551\/mitpress\/11442.003.0049","type":"journal-article","article-title":"Opening burton\u2019s clock: Psychiatric insights from computational cognitive models","volume":"2020","author":"Bennett","year":"2020","journal-title":"The Cognitive Neurosciences"},{"key":"bib8","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1146\/annurev-neuro-080317-062007","type":"journal-article","article-title":"Anxiety, depression, and decision making: a computational perspective","volume":"41","author":"Bishop","year":"2018","journal-title":"Annual Review of Neuroscience"},{"key":"bib9","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1177\/146642405507600912","type":"journal-article","article-title":"(b) The growth of independence in the young child","volume":"76","author":"Bowlby","year":"1955","journal-title":"Journal Royal Society of Health"},{"key":"bib10","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.neuron.2009.06.009","type":"journal-article","article-title":"Midbrain dopamine neurons signal preference for advance information about upcoming rewards","volume":"63","author":"Bromberg-Martin","year":"2009","journal-title":"Neuron"},{"key":"bib11","doi-asserted-by":"publisher","first-page":"1175","DOI":"10.1006\/anbe.2002.3024","type":"journal-article","article-title":"Predator inspection behaviour and attack cone avoidance in a characin fish: the effects of predator diet and prey experience","volume":"63","author":"Brown","year":"2002","journal-title":"Animal Behaviour"},{"key":"bib12","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/0146-6402(83)90015-2","type":"journal-article","article-title":"Cognitive processes in anxiety","volume":"5","author":"Butler","year":"1983","journal-title":"Advances in Behaviour Research and Therapy"},{"key":"bib13","doi-asserted-by":"publisher","first-page":"1014","DOI":"10.1016\/j.biopsych.2016.12.010","type":"journal-article","article-title":"Enhanced risk aversion, but not loss aversion, in unmedicated pathological anxiety","volume":"81","author":"Charpentier","year":"2017","journal-title":"Biological Psychiatry"},{"key":"bib14","volume-title":"Risk-Sensitive and Robust Decision-Making: A Cvar Optimization Approach","author":"Chow","year":"2015","type":"book"},{"key":"bib15","doi-asserted-by":"publisher","first-page":"294","DOI":"10.2307\/1382883","type":"journal-article","article-title":"Individual variation in the antisnake behavior of california ground squirrels (Spermophilus beecheyi)","volume":"78","author":"Coss","year":"1997","journal-title":"Journal of Mammalogy"},{"key":"bib16","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1038\/s41586-019-1924-6","type":"journal-article","article-title":"A distributional code for value in dopamine-based reinforcement learning","volume":"577","author":"Dabney","year":"2020","journal-title":"Nature"},{"key":"bib17","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1018357105171","type":"journal-article","article-title":"Exploration bonuses and dual control","volume":"25","author":"Dayan","year":"1996","journal-title":"Machine Learning"},{"key":"bib18","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1038\/81504","type":"journal-article","article-title":"Learning and selective attention","volume":"3 Suppl","author":"Dayan","year":"2000","journal-title":"Nature Neuroscience"},{"key":"bib19","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1093\/med\/9780198789697.003.0014","type":"book-chapter","article-title":"The first steps on long marches: the costs of active observation","volume-title":"Psychiatry Reborn: Biopsychosocial Psychiatry in Modern Medicine","author":"Dayan","year":"2020"},{"key":"bib20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1301.6690","type":"preprint","article-title":"Model-based bayesian exploration","volume-title":"arXiv","author":"Dearden","year":"2013"},{"key":"bib21","type":"dissertation","article-title":"Optimal learning: Computational procedures for Bayes-adaptive Markov decision processes","author":"Duff","year":"2002","unstructured":"Duff M. 2002a. Optimal learning: Computational procedures for Bayes-adaptive Markov decision processes. University of Massachusetts Amherst."},{"key":"bib22","volume-title":"Optimal Learning: Computational Procedures for Bayes-Adaptive Markov Decision Processes","author":"Duff","year":"2002","type":"book"},{"key":"bib23","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1007\/BF00303714","type":"journal-article","article-title":"Do guppies play TIT FOR TAT during predator inspection visits?","volume":"23","author":"Dugatkin","year":"1988","journal-title":"Behavioral Ecology and Sociobiology"},{"key":"bib24","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1007\/s00442-020-04773-y","type":"journal-article","article-title":"Among-individual differences in foraging modulate resource exploitation under perceived predation risk","volume":"194","author":"Eccard","year":"2020","journal-title":"Oecologia"},{"key":"bib25","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/j.tics.2015.07.010","type":"journal-article","article-title":"Mood as representation of momentum","volume":"20","author":"Eldar","year":"2016","journal-title":"Trends in Cognitive Sciences"},{"key":"bib26","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/BF00164184","type":"journal-article","article-title":"The costs and benefits of predator inspection behaviour in Thomson\u2019s gazelles","volume":"34","author":"FitzGibbon","year":"1994","journal-title":"Behavioral Ecology and Sociobiology"},{"key":"bib27","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.1701381","type":"journal-article","article-title":"Risk preference shares the psychometric structure of major psychological traits","volume":"3","author":"Frey","year":"2017","journal-title":"Science Advances"},{"key":"bib28","first-page":"22209","type":"conference-paper","article-title":"Two steps to risk sensitivity","author":"Gagne","year":"2021","unstructured":"Gagne C, Dayan P. 2021. Two steps to risk sensitivity. Advances in Neural Information Processing Systems 34 (NeurIPS 2021)."},{"key":"bib29","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmp.2021.102617","type":"journal-article","article-title":"Peril, prudence and planning as risk, avoidance and worry","volume":"106","author":"Gagne","year":"2022","journal-title":"Journal of Mathematical Psychology"},{"key":"bib30","doi-asserted-by":"publisher","first-page":"1373","DOI":"10.1037\/a0029119","type":"journal-article","article-title":"Reduced risk-taking behavior as a trait feature of anxiety","volume":"12","author":"Giorgetta","year":"2012","journal-title":"Emotion"},{"key":"bib31","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1111\/j.2517-6161.1979.tb01068.x","type":"journal-article","article-title":"Bandit processes and dynamic allocation indices","volume":"41","author":"Gittins","year":"1979","journal-title":"Journal of the Royal Statistical Society Series B"},{"key":"bib32","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1613\/jair.1579","type":"journal-article","article-title":"A framework for sequential planning in multi-agent settings","volume":"24","author":"Gmytrasiewicz","year":"2005","journal-title":"Journal of Artificial Intelligence Research"},{"key":"bib33","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.tics.2013.09.001","type":"journal-article","article-title":"Information-seeking, curiosity, and attention: computational and neural mechanisms","volume":"17","author":"Gottlieb","year":"2013","journal-title":"Trends in Cognitive Sciences"},{"key":"bib34","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.cobeha.2015.10.007","type":"journal-article","article-title":"Neophobia is not only avoidance: improving neophobia tests by combining cognition and ecology","volume":"6","author":"Greggor","year":"2015","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"bib35","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1613\/jair.4117","type":"journal-article","article-title":"Scalable and efficient bayes-adaptive reinforcement learning based on monte-carlo tree search","volume":"48","author":"Guez","year":"2013","journal-title":"Journal of Artificial Intelligence Research"},{"key":"bib36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.12477","type":"preprint","article-title":"On dynamic program decompositions of static risk measures","volume-title":"arXiv","author":"Hau","year":"2023"},{"key":"bib37","doi-asserted-by":"publisher","DOI":"10.5334\/cpsy.81","type":"book","volume-title":"Components of Behavioral Activation Therapy for Depression Engage Specific Reinforcement Learning Mechanisms in a Pilot Study","author":"Huys","year":"2022"},{"key":"bib38","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1613\/jair.583","type":"journal-article","article-title":"Variational probabilistic inference and the QMR-DT Network","volume":"10","author":"Jaakkola","year":"1999","journal-title":"Journal of Artificial Intelligence Research"},{"key":"bib39","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","type":"journal-article","article-title":"Planning and acting in partially observable stochastic domains","volume":"101","author":"Kaelbling","year":"1998","journal-title":"Artificial Intelligence"},{"key":"bib40","doi-asserted-by":"publisher","first-page":"549","DOI":"10.1016\/s0893-6080(02)00048-5","type":"journal-article","article-title":"Dopamine: generalization and bonuses","volume":"15","author":"Kakade","year":"2002","journal-title":"Neural Networks"},{"key":"bib41","doi-asserted-by":"publisher","DOI":"10.46867\/IJCP.2011.24.01.02","type":"journal-article","article-title":"Individual modulation of anti-predator responses in common marmosets","volume":"24","author":"Kemp","year":"2011","journal-title":"International Journal of Comparative Psychology"},{"key":"bib42","doi-asserted-by":"publisher","first-page":"12868","DOI":"10.1073\/pnas.1609094113","type":"journal-article","article-title":"Adaptive integration of habits into depth-limited planning defines a habitual-goal\u2013directed spectrum","volume":"113","author":"Keramati","year":"2016","journal-title":"PNAS"},{"key":"bib43","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1005916","type":"journal-article","article-title":"Interrupting behaviour: Minimizing decision costs via temporal commitment and low-level interrupts","volume":"14","author":"Lloyd","year":"2018","journal-title":"PLOS Computational Biology"},{"key":"bib44","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/BF00300641","type":"journal-article","article-title":"Predator inspection behaviour in minnow shoals: differences between populations and individuals","volume":"19","author":"Magurran","year":"1986","journal-title":"Behavioral Ecology and Sociobiology"},{"key":"bib45","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1016\/S0003-3472(05)80524-X","type":"journal-article","article-title":"Population differences in predator recognition and attack cone avoidance in the guppy Poecilia reticulata","volume":"40","author":"Magurran","year":"1990","journal-title":"Animal Behaviour"},{"key":"bib46","doi-asserted-by":"publisher","first-page":"665","DOI":"10.1016\/j.paid.2006.08.016","type":"journal-article","article-title":"Dispositional anxiety and risk-avoidant decision-making","volume":"42","author":"Maner","year":"2007","journal-title":"Personality and Individual Differences"},{"key":"bib47","doi-asserted-by":"publisher","DOI":"10.1101\/2023.11.12.566754","type":"preprint","article-title":"Multi-timescale reinforcement learning in the brain","volume-title":"bioRxiv","author":"Masset","year":"2023"},{"key":"bib48","doi-asserted-by":"publisher","first-page":"1281","DOI":"10.1038\/s41593-018-0209-y","type":"journal-article","article-title":"DeepLabCut: markerless pose estimation of user-defined body parts with deep learning","volume":"21","author":"Mathis","year":"2018","journal-title":"Nature Neuroscience"},{"key":"bib49","doi-asserted-by":"publisher","first-page":"1609","DOI":"10.1038\/s41593-018-0232-z","type":"journal-article","article-title":"Prioritized memory access explains planning and hippocampal replay","volume":"21","author":"Mattar","year":"2018","journal-title":"Nature Neuroscience"},{"key":"bib50","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-46582-1","type":"journal-article","article-title":"Individual variation in cognitive style reflects foraging and anti-predator strategies in a small mammal","volume":"9","author":"Mazza","year":"2019","journal-title":"Scientific Reports"},{"key":"bib51","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.21886","type":"journal-article","article-title":"Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice","volume":"6","author":"Menegas","year":"2017","journal-title":"eLife"},{"key":"bib52","doi-asserted-by":"publisher","first-page":"1421","DOI":"10.1038\/s41593-018-0222-1","type":"journal-article","article-title":"Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli","volume":"21","author":"Menegas","year":"2018","journal-title":"Nature Neuroscience"},{"key":"bib53","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/j.tics.2019.12.016","type":"journal-article","article-title":"Space, time, and fear: survival computations along defensive circuits","volume":"24","author":"Mobbs","year":"2020","journal-title":"Trends in Cognitive Sciences"},{"key":"bib54","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1038\/s41593-021-00950-1","type":"journal-article","article-title":"A primate temporal cortex-zona incerta pathway for novelty seeking","volume":"25","author":"Ogasawara","year":"2022","journal-title":"Nature Neuroscience"},{"key":"bib55","doi-asserted-by":"publisher","DOI":"10.3389\/neuro.12.006.2007","type":"journal-article","article-title":"What is intrinsic motivation? a typology of computational approaches","volume":"1","author":"Oudeyer","year":"2007","journal-title":"Frontiers in Neurorobotics"},{"key":"bib56","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1016\/j.tics.2012.07.009","type":"journal-article","article-title":"Emotion and decision-making: affect-driven belief systems in anxiety and depression","volume":"16","author":"Paulus","year":"2012","journal-title":"Trends in Cognitive Sciences"},{"key":"bib57","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1016\/j.conb.2019.03.011","type":"journal-article","article-title":"State representation in mental illness","volume":"55","author":"Radulescu","year":"2019","journal-title":"Current Opinion in Neurobiology"},{"key":"bib58","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1111\/j.1469-185X.2007.00010.x","type":"journal-article","article-title":"Integrating animal temperament within ecology and evolution","volume":"82","author":"R\u00e9ale","year":"2007","journal-title":"Biological Reviews of the Cambridge Philosophical Society"},{"key":"bib59","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1287\/mnsc.2013.1869","type":"journal-article","article-title":"Risk preferences around the world","volume":"61","author":"Rieger","year":"2015","journal-title":"Management Science"},{"key":"bib60","first-page":"1142","type":"conference-paper","article-title":"Risk-averse bayes-adaptive reinforcement learning","author":"Rigter","year":"2021","unstructured":"Rigter M, Lacerda B, Hawes N. 2021. Risk-averse bayes-adaptive reinforcement learning. Advances in Neural Information Processing Systems 34."},{"key":"bib61","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell","year":"2016","type":"book"},{"key":"bib62","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.06295","type":"preprint","article-title":"Policy Distillation","volume-title":"arXiv","author":"Rusu","year":"2016"},{"key":"bib63","type":"software","article-title":"Risking-your-tail-modeling-individual-differences-in-risk-sensitive-exploration","volume-title":"Software Heritage","author":"Shen","year":"2025","unstructured":"Shen T. 2025. Risking-your-tail-modeling-individual-differences-in-risk-sensitive-exploration. Software Heritage. swh:1:rev:73126ab1952b96842a588f6fa4fc4eed6a80b264. https:\/\/archive.softwareheritage.org\/swh:1:dir:1e57b66b4cea20aa9aeac42046646c38d4919648;origin=https:\/\/github.com\/shenkev\/Risking-your-Tail-Modeling-Individual-Differences-in-Risk-sensitive-Exploration;visit=swh:1:snp:4e617ceb034057cfab1c6ec8b831e5a6eeef3360;anchor=swh:1:rev:73126ab1952b96842a588f6fa4fc4eed6a80b264."},{"key":"bib64","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/0166-4328(94)90008-6","type":"journal-article","article-title":"Thigmotaxis as an index of anxiety in mice. Influence of dopaminergic transmissions","volume":"61","author":"Simon","year":"1994","journal-title":"Behavioural Brain Research"},{"key":"bib65","doi-asserted-by":"publisher","DOI":"10.1101\/2023.11.12.566727","type":"preprint","article-title":"Dopamine neurons encode a multidimensional probabilistic map of future reward","volume-title":"bioRxiv","author":"Sousa","year":"2023"},{"key":"bib66","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1023\/A:1022633531479","type":"journal-article","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Machine Learning"},{"key":"bib67","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1098\/rsif.2008.0172","type":"journal-article","article-title":"Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems","volume":"6","author":"Toni","year":"2009","journal-title":"Journal of the Royal Society, Interface"},{"key":"bib68","first-page":"9","type":"journal-article","article-title":"Learning to reinforcement learn","volume":"3","author":"Wang","year":"2017","journal-title":"Machine Learning"},{"key":"bib69","doi-asserted-by":"publisher","DOI":"10.1214\/aoap\/1177005588","type":"journal-article","article-title":"On the gittins index for multiarmed bandits","volume":"2","author":"Weber","year":"1992","journal-title":"The Annals of Applied Probability"},{"key":"bib70","doi-asserted-by":"publisher","first-page":"2074","DOI":"10.1037\/a0038199","type":"journal-article","article-title":"Humans use directed and random exploration to solve the explore-exploit dilemma","volume":"143","author":"Wilson","year":"2014","journal-title":"Journal of Experimental Psychology. General"},{"key":"bib71","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1038\/s41593-020-00706-3","type":"journal-article","article-title":"Revealing the structure of pharmacobehavioral space through motion sequencing","volume":"23","author":"Wiltschko","year":"2020","journal-title":"Nature Neuroscience"},{"key":"dataset1","doi-asserted-by":"publisher","DOI":"10.5061\/dryad.41ns1rnh2","type":"dataset","article-title":"Dopamine activity in the tail of the striatum, DeepLabCut and MoSeq during novel object exploration","volume-title":"Dryad Digital Repository","author":"Watabe-Uchida","year":"2023","unstructured":"Watabe-Uchida M, Akiti K, Tsutsui-Kimura I, Xie Y, Mathis A, Markowitz J, Anyoha R, Datta S, Weygandt Mathis M, Uchida N. 2023. Dopamine activity in the tail of the striatum, DeepLabCut and MoSeq during novel object exploration. Dryad Digital Repository."}],"container-title":["eLife"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/cdn.elifesciences.org\/articles\/100366\/elife-100366-v1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/cdn.elifesciences.org\/articles\/100366\/elife-100366-v1.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/elifesciences.org\/articles\/100366","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T09:14:25Z","timestamp":1787562865000},"score":1,"resource":{"primary":{"URL":"https:\/\/elifesciences.org\/articles\/100366"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,1]]},"references-count":72,"alternative-id":["10.7554\/eLife.100366"],"URL":"https:\/\/doi.org\/10.7554\/elife.100366","archive":["CLOCKSS"],"relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.01.07.574574","asserted-by":"subject"}],"references":[{"id-type":"doi","id":"10.5061\/dryad.41ns1rnh2","asserted-by":"subject"}],"is-same-as":[{"id-type":"doi","id":"10.7554\/eLife.100366.3","asserted-by":"subject"}],"is-version-of":[{"id-type":"doi","id":"10.7554\/eLife.100366.1","asserted-by":"subject"},{"id-type":"doi","id":"10.7554\/eLife.100366.2","asserted-by":"subject"}],"is-supplemented-by":[{"id-type":"uri","id":"https:\/\/archive.softwareheritage.org\/swh:1:dir:1e57b66b4cea20aa9aeac42046646c38d4919648;origin=https:\/\/github.com\/shenkev\/Risking-your-Tail-Modeling-Individual-Differences-in-Risk-sensitive-Exploration;visit=swh:1:snp:4e617ceb034057cfab1c6ec8b831e5a6eeef3360;anchor=swh:1:rev:73126ab1952b96842a588f6fa4fc4eed6a80b264","asserted-by":"subject"}]},"ISSN":["2050-084X"],"issn-type":[{"value":"2050-084X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,1]]},"assertion":[{"value":"single anonymised","name":"peer_review_transparency","label":"Peer review transparency","group":{"name":"peer_review_taxonomy"}},{"value":"other reviewer(s), editor","name":"peer_review_interaction","label":"Peer review interaction","group":{"name":"peer_review_taxonomy"}},{"value":"review summaries, review reports, author\/editor communication, reviewer identities reviewer opt in, editor identities","name":"peer_review_published","label":"Peer review published","group":{"name":"peer_review_taxonomy"}},{"value":"open (sign in with ORCID iD required)","name":"post_publication_commenting","label":"Post publication commenting","group":{"name":"post_publication_commenting"}}],"article-number":"RP100366"}}