{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T14:56:34Z","timestamp":1770562594457,"version":"3.49.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009213","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000}}],"reference-count":48,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T00:00:00Z","timestamp":1626393600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MC_UU_12024\/5"],"award-info":[{"award-number":["MC_UU_12024\/5"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MC_UU_00003\/1"],"award-info":[{"award-number":["MC_UU_00003\/1"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000268","name":"Biotechnology and Biological Sciences Research Council","doi-asserted-by":"publisher","award":["BB\/S006338\/1"],"award-info":[{"award-number":["BB\/S006338\/1"]}],"id":[{"id":"10.13039\/501100000268","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MR\/P00878X"],"award-info":[{"award-number":["MR\/P00878X"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MR\/K501256\/1"],"award-info":[{"award-number":["MR\/K501256\/1"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","award":["MR\/N013468\/1"],"award-info":[{"award-number":["MR\/N013468\/1"]}],"id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Reward prediction errors (RPEs) and risk preferences have two things in common: both can shape decision making behavior, and both are commonly associated with dopamine. RPEs drive value learning and are thought to be represented in the phasic release of striatal dopamine. Risk preferences bias choices towards or away from uncertainty; they can be manipulated with drugs that target the dopaminergic system. Based on the common neural substrate, we hypothesize that RPEs and risk preferences are linked on the level of behavior as well. Here, we develop this hypothesis theoretically and test it empirically. First, we apply a recent theory of learning in the basal ganglia to predict how RPEs influence risk preferences. We find that positive RPEs should cause increased risk-seeking, while negative RPEs should cause risk-aversion. We then test our behavioral predictions using a novel bandit task in which value and risk vary independently across options. Critically, conditions are included where options vary in risk but are matched for value. We find that our prediction was correct: participants become more risk-seeking if choices are preceded by positive RPEs, and more risk-averse if choices are preceded by negative RPEs. These findings cannot be explained by other known effects, such as nonlinear utility curves or dynamic learning rates.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009213","type":"journal-article","created":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T17:46:01Z","timestamp":1626457561000},"page":"e1009213","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":19,"title":["An association between prediction errors and risk-seeking: Theory and behavioral evidence"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0399-574X","authenticated-orcid":true,"given":"Moritz","family":"Moeller","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9233-1066","authenticated-orcid":true,"given":"Jan","family":"Grohn","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0735-4349","authenticated-orcid":true,"given":"Sanjay","family":"Manohar","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8994-1661","authenticated-orcid":true,"given":"Rafal","family":"Bogacz","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2021,7,16]]},"reference":[{"key":"pcbi.1009213.ref001","first-page":"64","article-title":"A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement","volume":"2","author":"RA Rescorla","year":"1972","journal-title":"Classical conditioning II: Current research and theory"},{"issue":"5306","key":"pcbi.1009213.ref002","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"issue":"6","key":"pcbi.1009213.ref003","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1038\/nrn1406","article-title":"Dopamine, learning and motivation","volume":"5","author":"RA Wise","year":"2004","journal-title":"Nature reviews neuroscience"},{"issue":"5","key":"pcbi.1009213.ref004","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.tins.2007.03.006","article-title":"Dopamine neuron systems in the brain: an update","volume":"30","author":"A Bj\u00f6rklund","year":"2007","journal-title":"Trends in neurosciences"},{"issue":"6851","key":"pcbi.1009213.ref005","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1038\/35092560","article-title":"A cellular mechanism of reward-related learning","volume":"413","author":"JN Reynolds","year":"2001","journal-title":"Nature"},{"issue":"7","key":"pcbi.1009213.ref006","doi-asserted-by":"crossref","first-page":"966","DOI":"10.1038\/nn.3413","article-title":"A causal link between prediction errors, dopamine neurons and learning","volume":"16","author":"EE Steinberg","year":"2013","journal-title":"Nature neuroscience"},{"issue":"3","key":"pcbi.1009213.ref007","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1037\/a0037015","article-title":"Opponent actor learning (OpAL): Modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive","volume":"121","author":"AG Collins","year":"2014","journal-title":"Psychological review"},{"issue":"9","key":"pcbi.1009213.ref008","doi-asserted-by":"crossref","first-page":"e1005062","DOI":"10.1371\/journal.pcbi.1005062","article-title":"Learning reward uncertainty in the basal ganglia","volume":"12","author":"JG Mikhael","year":"2016","journal-title":"PLoS computational biology"},{"issue":"2","key":"pcbi.1009213.ref009","doi-asserted-by":"crossref","first-page":"e1006285","DOI":"10.1371\/journal.pcbi.1006285","article-title":"Learning the payoffs and costs of actions","volume":"15","author":"M M\u00f6ller","year":"2019","journal-title":"PLoS computational biology"},{"key":"pcbi.1009213.ref010","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1146\/annurev-neuro-061010-113641","article-title":"Modulation of striatal projection systems by dopamine","volume":"34","author":"CR Gerfen","year":"2011","journal-title":"Annual review of neuroscience"},{"issue":"11","key":"pcbi.1009213.ref011","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1212\/01.wnl.0000218206.20920.4d","article-title":"Prospective prevalence of pathologic gambling and medication association in Parkinson disease","volume":"66","author":"V Voon","year":"2006","journal-title":"Neurology"},{"issue":"12","key":"pcbi.1009213.ref012","doi-asserted-by":"crossref","first-page":"1757","DOI":"10.1002\/mds.21611","article-title":"Pathological gambling in Parkinson\u2019s disease: risk factors and differences from dopamine dysregulation. An analysis of published case series","volume":"22","author":"DA Gallagher","year":"2007","journal-title":"Movement disorders: official journal of the Movement Disorder Society"},{"issue":"5","key":"pcbi.1009213.ref013","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/archneurol.2010.65","article-title":"Impulse control disorders in Parkinson disease: a cross-sectional study of 3090 patients","volume":"67","author":"D Weintraub","year":"2010","journal-title":"Archives of neurology"},{"issue":"3","key":"pcbi.1009213.ref014","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1038\/npp.2008.121","article-title":"Dopaminergic modulation of risk-based decision making","volume":"34","author":"JR St Onge","year":"2009","journal-title":"Neuropsychopharmacology"},{"issue":"37","key":"pcbi.1009213.ref015","doi-asserted-by":"crossref","first-page":"18732","DOI":"10.1073\/pnas.1900872116","article-title":"Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability","volume":"116","author":"B Chew","year":"2019","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"pcbi.1009213.ref016","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1196\/annals.1390.018","article-title":"Cost, benefit, tonic, phasic: what do response rates tell us about dopamine and motivation?","volume":"1104","author":"Y Niv","year":"2007","journal-title":"Annals of the New York Academy of Sciences"},{"issue":"6","key":"pcbi.1009213.ref017","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1038\/s41593-018-0152-y","article-title":"What does dopamine mean?","volume":"21","author":"JD Berke","year":"2018","journal-title":"Nature neuroscience"},{"key":"pcbi.1009213.ref018","doi-asserted-by":"crossref","first-page":"e53262","DOI":"10.7554\/eLife.53262","article-title":"Dopamine role in learning and action inference","volume":"9","author":"R Bogacz","year":"2020","journal-title":"Elife"},{"issue":"6992","key":"pcbi.1009213.ref019","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1038\/nature02581","article-title":"Temporal difference models describe higher-order learning in humans","volume":"429","author":"B Seymour","year":"2004","journal-title":"Nature"},{"issue":"7106","key":"pcbi.1009213.ref020","doi-asserted-by":"crossref","first-page":"1042","DOI":"10.1038\/nature05051","article-title":"Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans","volume":"442","author":"M Pessiglione","year":"2006","journal-title":"Nature"},{"issue":"2","key":"pcbi.1009213.ref021","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1523\/JNEUROSCI.5498-10.2012","article-title":"Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain","volume":"32","author":"Y Niv","year":"2012","journal-title":"Journal of Neuroscience"},{"issue":"6","key":"pcbi.1009213.ref022","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1007\/PL00007984","article-title":"A computational model of action selection in the basal ganglia. I. A new functional anatomy","volume":"84","author":"K Gurney","year":"2001","journal-title":"Biological cybernetics"},{"issue":"8","key":"pcbi.1009213.ref023","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1016\/j.neunet.2006.03.006","article-title":"Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making","volume":"19","author":"MJ Frank","year":"2006","journal-title":"Neural networks"},{"key":"pcbi.1009213.ref024","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"2018"},{"issue":"7","key":"pcbi.1009213.ref025","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1038\/s41562-019-0597-3","article-title":"Positive reward prediction errors during decision-making strengthen memory encoding","volume":"3","author":"AI Jang","year":"2019","journal-title":"Nature human behaviour"},{"issue":"1","key":"pcbi.1009213.ref026","article-title":"Trial-by-trial data analysis using computational models","volume":"23","author":"ND Daw","year":"2011","journal-title":"Decision making, affect, and learning: Attention and performance XXIII"},{"issue":"6","key":"pcbi.1009213.ref027","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.tics.2017.03.011","article-title":"The importance of falsification in computational cognitive modeling","volume":"21","author":"S Palminteri","year":"2017","journal-title":"Trends in cognitive sciences"},{"key":"pcbi.1009213.ref028","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1142\/9789814417358_0006","volume-title":"Handbook of the fundamentals of financial decision making: Part I","author":"D Kahneman","year":"2013"},{"issue":"5","key":"pcbi.1009213.ref029","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.3758\/s13423-014-0790-3","article-title":"Do learning rates adapt to the distribution of rewards?","volume":"22","author":"SJ Gershman","year":"2015","journal-title":"Psychonomic Bulletin & Review"},{"issue":"1819","key":"pcbi.1009213.ref030","doi-asserted-by":"crossref","first-page":"20190665","DOI":"10.1098\/rstb.2019.0665","article-title":"The description\u2013experience gap: a challenge for the neuroeconomics of decision-making under uncertainty","volume":"376","author":"B Garcia","year":"2021","journal-title":"Philosophical Transactions of the Royal Society B"},{"issue":"3","key":"pcbi.1009213.ref031","doi-asserted-by":"crossref","first-page":"1628","DOI":"10.1152\/jn.00483.2015","article-title":"Scaling prediction errors to reward variability benefits error-driven learning in humans","volume":"114","author":"KM Diederen","year":"2015","journal-title":"Journal of Neurophysiology"},{"issue":"5","key":"pcbi.1009213.ref032","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1016\/j.neuron.2016.04.019","article-title":"Adaptive prediction error coding in the human midbrain and striatum facilitates behavioral adaptation and learning efficiency","volume":"90","author":"KM Diederen","year":"2016","journal-title":"Neuron"},{"issue":"5715","key":"pcbi.1009213.ref033","doi-asserted-by":"crossref","first-page":"1642","DOI":"10.1126\/science.1105370","article-title":"Adaptive coding of reward value by dopamine neurons","volume":"307","author":"PN Tobler","year":"2005","journal-title":"Science"},{"issue":"2","key":"pcbi.1009213.ref034","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1002\/wics.199","article-title":"The Bayesian information criterion: background, derivation, and applications","volume":"4","author":"AA Neath","year":"2012","journal-title":"Wiley Interdisciplinary Reviews: Computational Statistics"},{"key":"pcbi.1009213.ref035","article-title":"Gambling on an empty stomach: Hunger modulates preferences for learned but not described risks","author":"MM van Swieten","year":"2021","journal-title":"bioRxiv"},{"issue":"1","key":"pcbi.1009213.ref036","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1038\/nn.4173","article-title":"Mesolimbic dopamine signals the value of work","volume":"19","author":"AA Hamid","year":"2016","journal-title":"Nature neuroscience"},{"issue":"7691","key":"pcbi.1009213.ref037","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1038\/nature25457","article-title":"Dopamine neuron activity before action initiation gates and invigorates future movements","volume":"554","author":"JA da Silva","year":"2018","journal-title":"Nature"},{"key":"pcbi.1009213.ref038","doi-asserted-by":"crossref","first-page":"e63166","DOI":"10.7554\/eLife.63166","article-title":"Tonic firing mode of midbrain dopamine neurons continuously tracks reward values changing moment-by-moment","volume":"10","author":"Y Wang","year":"2021","journal-title":"Elife"},{"issue":"2","key":"pcbi.1009213.ref039","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1037\/bul0000115","article-title":"A meta-analytic review of two modes of learning and the description-experience gap","volume":"144","author":"DU Wulff","year":"2018","journal-title":"Psychological bulletin"},{"issue":"3","key":"pcbi.1009213.ref040","doi-asserted-by":"crossref","first-page":"629","DOI":"10.3758\/s13423-013-0542-9","article-title":"Remembering the best and worst of times: Memories for extreme outcomes bias risky decisions","volume":"21","author":"CR Madan","year":"2014","journal-title":"Psychonomic bulletin & review"},{"issue":"9","key":"pcbi.1009213.ref041","first-page":"1430","article-title":"Dissociable effects of surprising rewards on learning and memory","volume":"44","author":"N Rouhani","year":"2018","journal-title":"Journal of Experimental Psychology: Learning, Memory, and Cognition"},{"issue":"7","key":"pcbi.1009213.ref042","doi-asserted-by":"crossref","first-page":"3146","DOI":"10.1523\/JNEUROSCI.3653-14.2015","article-title":"Economic choices reveal probability distortion in macaque monkeys","volume":"35","author":"WR Stauffer","year":"2015","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1009213.ref043","doi-asserted-by":"crossref","unstructured":"Guyaguler B, Horne RN, editors. Uncertainty assessment of well placement optimization. SPE annual technical conference and exhibition; 2001: Society of Petroleum Engineers.","DOI":"10.2118\/71625-MS"},{"issue":"8","key":"pcbi.1009213.ref044","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41562-017-0145","article-title":"Selective overweighting of larger magnitudes during noisy numerical comparison","volume":"1","author":"B Spitzer","year":"2017","journal-title":"Nature Human Behaviour"},{"issue":"7095","key":"pcbi.1009213.ref045","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1038\/nature04766","article-title":"Cortical substrates for exploratory decisions in humans","volume":"441","author":"ND Daw","year":"2006","journal-title":"Nature"},{"issue":"9","key":"pcbi.1009213.ref046","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"TE Behrens","year":"2007","journal-title":"Nature neuroscience"},{"key":"pcbi.1009213.ref047","volume-title":"An introduction to the Kalman filter","author":"G Welch","year":"1995"},{"issue":"1","key":"pcbi.1009213.ref048","doi-asserted-by":"crossref","first-page":"e1003441","DOI":"10.1371\/journal.pcbi.1003441","article-title":"VBA: a probabilistic treatment of nonlinear models for neurobiological and behavioural data.","volume":"10","author":"J Daunizeau","year":"2014","journal-title":"PLoS Comput Biol"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009213","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T00:00:00Z","timestamp":1627430400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009213","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,28]],"date-time":"2021-07-28T17:43:41Z","timestamp":1627494221000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009213"}},"subtitle":[],"editor":[{"given":"Alireza","family":"Soltani","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,7,16]]},"references-count":48,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2021,7,16]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009213","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1009213","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,16]]}}}