{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T11:40:08Z","timestamp":1774352408340,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1008553","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000}}],"reference-count":51,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2021,2,10]],"date-time":"2021-02-10T00:00:00Z","timestamp":1612915200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Pavlovian associations drive approach towards reward-predictive cues, and avoidance of punishment-predictive cues. These associations \u201cmisbehave\u201d when they conflict with correct instrumental behavior. This raises the question of how Pavlovian and instrumental influences on behavior are arbitrated. We test a computational theory according to which Pavlovian influence will be stronger when inferred controllability of outcomes is low. Using a model-based analysis of a Go\/NoGo task with human subjects, we show that theta-band oscillatory power in frontal cortex tracks inferred controllability, and that these inferences predict Pavlovian action biases. Functional MRI data revealed an inferior frontal gyrus correlate of action probability and a ventromedial prefrontal correlate of outcome valence, both of which were modulated by inferred controllability.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1008553","type":"journal-article","created":{"date-parts":[[2021,2,11]],"date-time":"2021-02-11T16:59:40Z","timestamp":1613062780000},"page":"e1008553","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":36,"title":["Neural signatures of arbitration between Pavlovian and instrumental action selection"],"prefix":"10.1371","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6546-3298","authenticated-orcid":true,"given":"Samuel J.","family":"Gershman","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2294-6492","authenticated-orcid":true,"given":"Marc","family":"Guitart-Masip","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2428-3562","authenticated-orcid":true,"given":"James F.","family":"Cavanagh","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2021,2,10]]},"reference":[{"key":"pcbi.1008553.ref001","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1037\/h0040090","article-title":"The misbehavior of organisms","volume":"16","author":"K Breland","year":"1961","journal-title":"American Psychologist"},{"key":"pcbi.1008553.ref002","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1016\/j.neunet.2006.03.002","article-title":"The misbehavior of value and the discipline of the will","volume":"19","author":"P Dayan","year":"2006","journal-title":"Neural Networks"},{"key":"pcbi.1008553.ref003","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1901\/jeab.1969.12-511","article-title":"Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement","volume":"12","author":"DR Williams","year":"1969","journal-title":"Journal of the experimental analysis of behavior"},{"key":"pcbi.1008553.ref004","doi-asserted-by":"crossref","first-page":"443","DOI":"10.3758\/BF03200092","article-title":"An approach through the looking-glass","volume":"14","author":"WA Hershberger","year":"1986","journal-title":"Animal Learning & Behavior"},{"key":"pcbi.1008553.ref005","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1037\/h0027780","article-title":"Effects of appetitive discriminative stimuli on avoidance behavior","volume":"81","author":"N Grossen","year":"1969","journal-title":"Journal of Experimental Psychology"},{"key":"pcbi.1008553.ref006","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/0023-9690(70)90124-4","article-title":"An interaction between appetitive Pavlovian CSs and instrumental avoidance responding","volume":"1","author":"JA Bull","year":"1970","journal-title":"Learning and Motivation"},{"key":"pcbi.1008553.ref007","doi-asserted-by":"crossref","first-page":"390","DOI":"10.1037\/h0062283","article-title":"Some quantitative properties of anxiety","volume":"29","author":"W Estes","year":"1941","journal-title":"Journal of Experimental Psychology"},{"key":"pcbi.1008553.ref008","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1037\/h0042199","article-title":"The conditioned emotional response as a function of intensity of the US","volume":"54","author":"Z Annau","year":"1961","journal-title":"Journal of Comparative and Physiological Psychology"},{"key":"pcbi.1008553.ref009","article-title":"Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees","volume":"8","author":"QJ Huys","year":"2012","journal-title":"PLoS Computational Biology"},{"key":"pcbi.1008553.ref010","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1037\/h0024475","article-title":"Two-process learning theory: relationships between Pavlovian conditioning and instrumental learning","volume":"74","author":"R Rescorla","year":"1967","journal-title":"Psychological Review"},{"key":"pcbi.1008553.ref011","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.tics.2014.01.003","article-title":"Action versus valence in decision making","volume":"18","author":"M Guitart-Masip","year":"2014","journal-title":"Trends in Cognitive Sciences"},{"key":"pcbi.1008553.ref012","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1016\/S0893-6080(02)00047-3","article-title":"Actor\u2013critic models of the basal ganglia: New anatomical and computational perspectives","volume":"15","author":"D Joel","year":"2002","journal-title":"Neural Networks"},{"key":"pcbi.1008553.ref013","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1126\/science.1094285","article-title":"Dissociable roles of ventral and dorsal striatum in instrumental conditioning","volume":"304","author":"J O\u2019Doherty","year":"2004","journal-title":"Science"},{"key":"pcbi.1008553.ref014","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"ND Daw","year":"2005","journal-title":"Nature Neuroscience"},{"key":"pcbi.1008553.ref015","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1002055","article-title":"Speed\/accuracy trade-off between the habitual and the goal-directed processes","volume":"7","author":"M Keramati","year":"2011","journal-title":"PLoS Computational Biology"},{"key":"pcbi.1008553.ref016","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"SW Lee","year":"2014","journal-title":"Neuron"},{"key":"pcbi.1008553.ref017","doi-asserted-by":"crossref","first-page":"1321","DOI":"10.1177\/0956797617708288","article-title":"Cost-benefit arbitration between multiple reinforcement-learning systems","volume":"28","author":"W Kool","year":"2017","journal-title":"Psychological Science"},{"key":"pcbi.1008553.ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-019-13737-7","article-title":"Controllability governs the balance between Pavlovian and instrumental action selection","volume":"10","author":"HM Dorfman","year":"2019","journal-title":"Nature Communications"},{"key":"pcbi.1008553.ref019","doi-asserted-by":"crossref","first-page":"11993","DOI":"10.1523\/JNEUROSCI.2513-09.2009","article-title":"Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans","volume":"29","author":"MJ Crockett","year":"2009","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref020","doi-asserted-by":"crossref","first-page":"7867","DOI":"10.1523\/JNEUROSCI.6376-10.2011","article-title":"Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain","volume":"31","author":"M Guitart-Masip","year":"2011","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref021","doi-asserted-by":"crossref","first-page":"2244","DOI":"10.1038\/npp.2012.75","article-title":"Serotonin modulates the effects of Pavlovian aversive predictions on response vigor","volume":"37","author":"MJ Crockett","year":"2012","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1008553.ref022","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.neuroimage.2012.04.024","article-title":"Go and no-go learning in reward and punishment: interactions between affect and effect","volume":"62","author":"M Guitart-Masip","year":"2012","journal-title":"NeuroImage"},{"key":"pcbi.1008553.ref023","doi-asserted-by":"crossref","first-page":"8541","DOI":"10.1523\/JNEUROSCI.5754-12.2013","article-title":"Frontal theta overrides pavlovian learning biases","volume":"33","author":"JF Cavanagh","year":"2013","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref024","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1073\/pnas.1816704116","article-title":"Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning","volume":"116","author":"L de Boer","year":"2019","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1008553.ref025","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1162\/jocn_a_01515","article-title":"Intermittent absence of control during reinforcement learning interferes with Pavlovian bias in action selection","volume":"32","author":"G Csifcs\u00e1k","year":"2020","journal-title":"Journal of Cognitive Neuroscience"},{"key":"pcbi.1008553.ref026","doi-asserted-by":"crossref","first-page":"225","DOI":"10.3758\/PBR.16.2.225","article-title":"Bayesian t tests for accepting and rejecting the null hypothesis","volume":"16","author":"JN Rouder","year":"2009","journal-title":"Psychonomic Bulletin & Review"},{"key":"pcbi.1008553.ref027","doi-asserted-by":"crossref","first-page":"e2005979","DOI":"10.1371\/journal.pbio.2005979","article-title":"Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action","volume":"16","author":"JC Swart","year":"2018","journal-title":"PLoS Biology"},{"key":"pcbi.1008553.ref028","doi-asserted-by":"crossref","first-page":"3198","DOI":"10.1016\/j.neuroimage.2009.11.080","article-title":"Frontal theta links prediction errors to behavioral adaptation in reinforcement learning","volume":"49","author":"JF Cavanagh","year":"2010","journal-title":"Neuroimage"},{"key":"pcbi.1008553.ref029","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1093\/cercor\/bhr332","article-title":"Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation","volume":"22","author":"JF Cavanagh","year":"2012","journal-title":"Cerebral Cortex"},{"key":"pcbi.1008553.ref030","doi-asserted-by":"crossref","first-page":"4106","DOI":"10.1162\/jocn_a_00110","article-title":"Frontal oscillatory dynamics predict feedback learning and action adjustment","volume":"23","author":"I Van de Vijver","year":"2011","journal-title":"Journal of Cognitive Neuroscience"},{"key":"pcbi.1008553.ref031","doi-asserted-by":"crossref","first-page":"1566","DOI":"10.1152\/jn.00290.2019","article-title":"Frontal scalp potentials foretell perceptual choice confidence","volume":"123","author":"K Lim","year":"2020","journal-title":"Journal of Neurophysiology"},{"key":"pcbi.1008553.ref032","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.tics.2014.04.012","article-title":"Frontal theta as a mechanism for cognitive control","volume":"18","author":"JF Cavanagh","year":"2014","journal-title":"Trends in Cognitive Sciences"},{"key":"pcbi.1008553.ref033","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.jphysparis.2014.04.003","article-title":"Frontal midline theta reflects anxiety and cognitive control: meta-analytic evidence","volume":"109","author":"JF Cavanagh","year":"2015","journal-title":"Journal of Physiology-Paris"},{"key":"pcbi.1008553.ref034","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: an integrative theory of anterior cingulate cortex function","volume":"79","author":"A Shenhav","year":"2013","journal-title":"Neuron"},{"key":"pcbi.1008553.ref035","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/S1053-8119(03)00275-1","article-title":"Right inferior prefrontal cortex mediates response inhibition while mesial prefrontal cortex is responsible for error detection","volume":"20","author":"K Rubia","year":"2003","journal-title":"NeuroImage"},{"key":"pcbi.1008553.ref036","doi-asserted-by":"crossref","first-page":"2424","DOI":"10.1523\/JNEUROSCI.4682-05.2006","article-title":"Cortical and subcortical contributions to stop signal response inhibition: role of the subthalamic nucleus","volume":"26","author":"AR Aron","year":"2006","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref037","first-page":"190","article-title":"The neurobiology of personal control during reward learning and its relationship to mood","volume":"4","author":"L Romaniuk","year":"2019","journal-title":"Biological Psychiatry: Cognitive Neuroscience and Neuroimaging"},{"key":"pcbi.1008553.ref038","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.neuroimage.2015.05.024","article-title":"Subjective illusion of control modulates striatal reward anticipation in adolescence","volume":"117","author":"RC Lorenz","year":"2015","journal-title":"NeuroImage"},{"key":"pcbi.1008553.ref039","doi-asserted-by":"crossref","first-page":"11379","DOI":"10.1523\/JNEUROSCI.1640-06.2006","article-title":"Choosing the lesser of two evils, the better of two goods: specifying the roles of ventromedial prefrontal cortex and dorsal anterior cingulate in object choice","volume":"26","author":"K Blair","year":"2006","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref040","doi-asserted-by":"crossref","first-page":"10318","DOI":"10.1523\/JNEUROSCI.1801-12.2012","article-title":"Regionally distinct processing of rewards and punishments by the primate ventromedial prefrontal cortex","volume":"32","author":"IE Monosov","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008553.ref041","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1038\/nn1399","article-title":"Medial prefrontal cortex determines how stressor controllability affects behavior and dorsal raphe nucleus","volume":"8","author":"J Amat","year":"2005","journal-title":"Nature Neuroscience"},{"key":"pcbi.1008553.ref042","first-page":"557","article-title":"Controllability modulates the anticipatory response in the human ventromedial prefrontal cortex","volume":"3","author":"DL Kerr","year":"2012","journal-title":"Frontiers in Psychology"},{"key":"pcbi.1008553.ref043","doi-asserted-by":"crossref","first-page":"1241","DOI":"10.1093\/cercor\/bht317","article-title":"How self-determined choice facilitates performance: A key role of the ventromedial prefrontal cortex","volume":"25","author":"K Murayama","year":"2015","journal-title":"Cerebral Cortex"},{"key":"pcbi.1008553.ref044","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1016\/j.neuron.2014.08.012","article-title":"Perceived control influences neural responses to setbacks and promotes persistence","volume":"83","author":"JP Bhanji","year":"2014","journal-title":"Neuron"},{"key":"pcbi.1008553.ref045","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.1016\/j.neubiorev.2010.03.007","article-title":"Pavlovian to instrumental transfer: a neurobehavioural perspective","volume":"34","author":"NM Holmes","year":"2010","journal-title":"Neuroscience & Biobehavioral Reviews"},{"key":"pcbi.1008553.ref046","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1111\/psyp.12040","article-title":"Frontal midline theta and N 200 amplitude reflect complementary information about expectancy and outcome evaluation","volume":"50","author":"A Hajihosseini","year":"2013","journal-title":"Psychophysiology"},{"key":"pcbi.1008553.ref047","doi-asserted-by":"crossref","first-page":"2752","DOI":"10.1152\/jn.00479.2013","article-title":"Midfrontal conflict-related theta-band power reflects neural oscillations that predict behavior","volume":"110","author":"MX Cohen","year":"2013","journal-title":"Journal of Neurophysiology"},{"key":"pcbi.1008553.ref048","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1111\/j.1469-8986.2011.01293.x","article-title":"Theta lingua franca: A common mid-frontal substrate for action monitoring processes","volume":"49","author":"JF Cavanagh","year":"2012","journal-title":"Psychophysiology"},{"key":"pcbi.1008553.ref049","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1016\/j.neuroimage.2006.07.029","article-title":"Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: a whole-brain analysis at 3 T and 1.5 T","volume":"33","author":"N Weiskopf","year":"2006","journal-title":"NeuroImage"},{"key":"pcbi.1008553.ref050","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1016\/j.neuroimage.2013.08.065","article-title":"Bayesian model selection for group studies\u2014revisited","volume":"84","author":"L Rigoux","year":"2014","journal-title":"NeuroImage"},{"key":"pcbi.1008553.ref051","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1004567","article-title":"A unifying probabilistic view of associative learning","volume":"11","author":"SJ Gershman","year":"2015","journal-title":"PLoS Computational Biology"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1008553","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1008553","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T15:02:53Z","timestamp":1614092573000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1008553"}},"subtitle":[],"editor":[{"given":"Daniele","family":"Marinazzo","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,2,10]]},"references-count":51,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,2,10]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1008553","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.06.02.129700","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,10]]}}}