{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T17:22:29Z","timestamp":1777569749752,"version":"3.51.4"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011692","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000}}],"reference-count":131,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T00:00:00Z","timestamp":1701993600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["NRF-2018R1C1B300731315"],"award-info":[{"award-number":["NRF-2018R1C1B300731315"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["NRF-2018R1A4A102589113"],"award-info":[{"award-number":["NRF-2018R1A4A102589113"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["2021M3E5D2A0102249311"],"award-info":[{"award-number":["2021M3E5D2A0102249311"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100002551","name":"Seoul National University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002551","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["5199990314123"],"award-info":[{"award-number":["5199990314123"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Research suggests that a fast, capacity-limited working memory (WM) system and a slow, incremental reinforcement learning (RL) system jointly contribute to instrumental learning. Thus, situations that strain WM resources alter instrumental learning: under WM loads, learning becomes slow and incremental, the reliance on computationally efficient learning increases, and action selection becomes more random. It is also suggested that Pavlovian learning influences people\u2019s behavior during instrumental learning by providing hard-wired instinctive responses including approach to reward predictors and avoidance of punishment predictors. However, it remains unknown how constraints on WM resources affect instrumental learning under Pavlovian influence. Thus, we conducted a functional magnetic resonance imaging (fMRI) study (N = 49) in which participants completed an instrumental learning task with Pavlovian\u2013instrumental conflict (the orthogonalized go\/no-go task) both with and without extra WM load. Behavioral and computational modeling analyses revealed that WM load reduced the learning rate and increased random choice, without affecting Pavlovian bias. Model-based fMRI analysis revealed that WM load strengthened RPE signaling in the striatum. Moreover, under WM load, the striatum showed weakened connectivity with the ventromedial and dorsolateral prefrontal cortex when computing reward expectations. These results suggest that the limitation of cognitive resources by WM load promotes slow and incremental learning through the weakened cooperation between WM and RL; such limitation also makes action selection more random, but it does not directly affect the balance between instrumental and Pavlovian systems.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011692","type":"journal-article","created":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T13:38:27Z","timestamp":1702042707000},"page":"e1011692","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["The neurocognitive role of working memory load when Pavlovian motivational control affects instrumental learning"],"prefix":"10.1371","volume":"19","author":[{"given":"Heesun","family":"Park","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7302-2716","authenticated-orcid":true,"given":"Hoyoung","family":"Doh","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eunhwi","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Harhim","family":"Park","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5900-8432","authenticated-orcid":true,"given":"Woo-Young","family":"Ahn","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2023,12,8]]},"reference":[{"key":"pcbi.1011692.ref001","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1111\/j.1460-9568.2011.07980.x","article-title":"How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis","volume":"35","author":"AGE Collins","year":"2012","journal-title":"Eur J Neurosci"},{"key":"pcbi.1011692.ref002","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1038\/s41386-020-0746-4","article-title":"Advances in the computational understanding of mental illness","volume":"46","author":"QJM Huys","year":"2021","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1011692.ref003","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.cobeha.2020.10.003","article-title":"The role of executive function in shaping reinforcement learning","author":"M Rmus","year":"2021","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"pcbi.1011692.ref004","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"2018"},{"key":"pcbi.1011692.ref005","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1016\/j.conb.2011.02.009","article-title":"Multiplicity of control in the basal ganglia: computational roles of striatal subregions","volume":"21","author":"AM Bornstein","year":"2011","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011692.ref006","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1177\/105971230501300205","article-title":"Actor\u2014Critic models of reinforcement learning in the basal ganglia: from natural to artificial rats.","volume":"13","author":"M Khamassi","year":"2005","journal-title":"Adapt Behav"},{"key":"pcbi.1011692.ref007","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive Hebbian learning","volume":"16","author":"PR Montague","year":"1996","journal-title":"Journal of neuroscience"},{"key":"pcbi.1011692.ref008","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.jmp.2008.12.005","article-title":"Reinforcement learning in the brain","volume":"53","author":"Y. Niv","year":"2009","journal-title":"J Math Psychol"},{"key":"pcbi.1011692.ref009","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/S0959-4388(97)80007-4","article-title":"Dopamine neurons and their role in reward mechanisms","volume":"7","author":"W. Schultz","year":"1997","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011692.ref010","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1152\/jn.1998.80.1.1","article-title":"Predictive reward signal of dopamine neurons","volume":"80","author":"W. Schultz","year":"1998","journal-title":"J Neurophysiol"},{"key":"pcbi.1011692.ref011","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A Neural Substrate of Prediction and Reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"key":"pcbi.1011692.ref012","doi-asserted-by":"crossref","first-page":"1297","DOI":"10.1016\/j.neubiorev.2013.03.023","article-title":"Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies.","volume":"37","author":"J Garrison","year":"2013","journal-title":"Neurosci Biobehav Rev"},{"key":"pcbi.1011692.ref013","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1126\/science.1094285","article-title":"Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning","volume":"304","author":"J O\u2019Doherty","year":"2004","journal-title":"Science"},{"key":"pcbi.1011692.ref014","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1016\/S0896-6273(03)00169-7","article-title":"Temporal Difference Models and Reward-Related Learning in the Human Brain","volume":"38","author":"JP O\u2019Doherty","year":"2003","journal-title":"Neuron"},{"key":"pcbi.1011692.ref015","doi-asserted-by":"crossref","first-page":"4332","DOI":"10.1523\/JNEUROSCI.2700-16.2017","article-title":"Working Memory Load Strengthens Reward Prediction Errors","volume":"37","author":"AGE Collins","year":"2017","journal-title":"J Neurosci"},{"key":"pcbi.1011692.ref016","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1162\/jocn_a_01238","article-title":"The Tortoise and the Hare: Interactions between Reinforcement Learning and Working Memory.","volume":"30","author":"AGE Collins","year":"2018","journal-title":"J Cogn Neurosci."},{"key":"pcbi.1011692.ref017","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1162\/jocn_a_01808","article-title":"How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective.","volume":"34","author":"AH Yoo","year":"2022","journal-title":"J Cogn Neurosci."},{"key":"pcbi.1011692.ref018","first-page":"201720963","article-title":"Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory","volume":"115","author":"AGE Collins","year":"2018","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1011692.ref019","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.neunet.2009.03.004","article-title":"Goal-directed control and its antipodes.","volume":"22","author":"P. Dayan","year":"2009","journal-title":"Neural Netw"},{"key":"pcbi.1011692.ref020","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.neuron.2013.09.007","article-title":"Goals and habits in the brain","author":"RJ Dolan","year":"2013","journal-title":"Neuron."},{"key":"pcbi.1011692.ref021","doi-asserted-by":"crossref","first-page":"6750","DOI":"10.1523\/JNEUROSCI.1808-08.2008","article-title":"Calculating consequences: brain systems that encode the causal effects of actions","volume":"28","author":"SC Tanaka","year":"2008","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref022","doi-asserted-by":"crossref","first-page":"4019","DOI":"10.1523\/JNEUROSCI.0564-07.2007","article-title":"Determining the neural substrates of goal-directed learning in the human brain","volume":"27","author":"VV Valentin","year":"2007","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref023","first-page":"1","article-title":"Advances in modeling learning and decision-making in neuroscience","author":"AGE Collins","year":"2021","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1011692.ref024","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/S0079-7421(08)60452-1","article-title":"Working Memory.","volume":"8","author":"AD Baddeley","year":"1974","journal-title":"Psychol Learn Motiv"},{"key":"pcbi.1011692.ref025","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1016\/j.cortex.2012.05.022","article-title":"Dorsolateral prefrontal contributions to human working memory.","volume":"49","author":"AK Barbey","year":"2013","journal-title":"Cortex"},{"key":"pcbi.1011692.ref026","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1016\/S1364-6613(03)00197-9","article-title":"Persistent activity in the prefrontal cortex during working memory.","volume":"7","author":"CE Curtis","year":"2003","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011692.ref027","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.neuroscience.2005.07.003","article-title":"Prefrontal cortex and working memory processes","volume":"139","author":"S. Funahashi","year":"2006","journal-title":"Neuroscience"},{"key":"pcbi.1011692.ref028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0168-0102(94)90063-9","article-title":"Working memory and prefrontal cortex","volume":"21","author":"S Funahashi","year":"1994","journal-title":"Neurosci Res"},{"key":"pcbi.1011692.ref029","doi-asserted-by":"crossref","first-page":"830","DOI":"10.1016\/j.neuroimage.2011.11.050","article-title":"Modelling neural correlates of working memory: A coordinate-based meta-analysis.","volume":"60","author":"C Rottschy","year":"2012","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref030","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1016\/j.biopsych.2017.05.017","article-title":"Interactions Among Working Memory, Reinforcement Learning, and Effort in Value-Based Choice: A New Paradigm and Selective Deficits in Schizophrenia","volume":"82","author":"AGE Collins","year":"2017","journal-title":"Biol Psychiatry"},{"key":"pcbi.1011692.ref031","first-page":"1","article-title":"Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning.","author":"SD McDougle","year":"2020","journal-title":"Psychon Bull Rev."},{"key":"pcbi.1011692.ref032","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1177\/0956797612463080","article-title":"The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive.","volume":"24","author":"AR Otto","year":"2013","journal-title":"Psychol Sci"},{"key":"pcbi.1011692.ref033","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1038\/nrn2357","article-title":"A framework for studying the neurobiology of value-based decision making.","volume":"9","author":"A Rangel","year":"2008","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011692.ref034","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1037\/0278-7393.32.2.443","article-title":"Does Working Memory Load Lead to Greater Impulsivity? Commentary on Hinson, Jameson, and Whitney (2003).","volume":"32","author":"AM Franco-Watkins","year":"2006","journal-title":"J Exp Psychol Learn Mem Cogn."},{"key":"pcbi.1011692.ref035","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1027\/1618-3169\/a000024","article-title":"Taxing Executive Processes Does Not Necessarily Increase Impulsive Decision Making.","volume":"57","author":"AM Franco-Watkins","year":"2010","journal-title":"Exp Psychol."},{"key":"pcbi.1011692.ref036","doi-asserted-by":"crossref","first-page":"462","DOI":"10.1037\/xge0000403","article-title":"Taxing Cognitive Capacities Reduces Choice Consistency Rather Than Preference: A Model-Based Test","volume":"147","author":"S Olschewski","year":"2018","journal-title":"J Exp Psychol Gen"},{"key":"pcbi.1011692.ref037","author":"A Dickinson","year":"2002","journal-title":"The role of learning in the operation of motivational systems"},{"key":"pcbi.1011692.ref038","volume-title":"Conditioning and associative learning","author":"NJ Mackintosh","year":"1983"},{"key":"pcbi.1011692.ref039","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1037\/h0036171","article-title":"Pavlovian appetitive contingencies and approach versus withdrawal to conditioned stimuli in pigeons","volume":"86","author":"EA Wasserman","year":"1974","journal-title":"J Comp Physiol Psychol"},{"key":"pcbi.1011692.ref040","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1146\/annurev.psych.48.1.573","article-title":"What\u2019s elementary about associative learning?","volume":"48","author":"EA Wasserman","year":"1997","journal-title":"Annu Rev Psychol"},{"key":"pcbi.1011692.ref041","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1037\/h0040090","article-title":"The misbehavior of organisms.","volume":"16","author":"K Breland","year":"1961","journal-title":"Am Psychol"},{"key":"pcbi.1011692.ref042","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1016\/j.neunet.2006.03.002","article-title":"The misbehavior of value and the discipline of the will.","volume":"19","author":"P Dayan","year":"2006","journal-title":"Neural Netw"},{"key":"pcbi.1011692.ref043","doi-asserted-by":"crossref","first-page":"443","DOI":"10.3758\/BF03200092","article-title":"An approach through the looking-glass.","volume":"14","author":"WA Hershberger","year":"1986","journal-title":"Anim Learn Behav"},{"key":"pcbi.1011692.ref044","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1901\/jeab.1969.12-511","article-title":"AUTO-MAINTENANCE IN THE PIGEON: SUSTAINED PECKING DESPITE CONTINGENT NON-REINFORCEMENT 2.","volume":"12","author":"DR Williams","year":"1969","journal-title":"J Exp Anal Behav."},{"key":"pcbi.1011692.ref045","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.neuroimage.2012.04.024","article-title":"Go and no-go learning in reward and punishment: Interactions between affect and effect.","volume":"62","author":"M Guitart-Masip","year":"2012","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref046","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.tics.2014.01.003","article-title":"Action versus valence in decision making.","volume":"18","author":"M Guitart-Masip","year":"2014","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011692.ref047","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1038\/nn1579","article-title":"Neural systems of reinforcement for drug addiction: from actions to habits to compulsion","volume":"8","author":"BJ Everitt","year":"2005","journal-title":"Nat Neurosci"},{"key":"pcbi.1011692.ref048","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1159\/000363507","article-title":"Pavlovian-to-Instrumental Transfer in Alcohol Dependence: A Pilot Study.","volume":"70","author":"M Garbusow","year":"2014","journal-title":"Neuropsychobiology"},{"key":"pcbi.1011692.ref049","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1111\/adb.12243","article-title":"Pavlovian-to-instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence","volume":"21","author":"M Garbusow","year":"2016","journal-title":"Addict Biol"},{"key":"pcbi.1011692.ref050","doi-asserted-by":"crossref","first-page":"53","DOI":"10.15288\/jsa.2005.66.53","article-title":"The role of Pavlovian cues in alcohol seeking in dependent and nondependent rats","volume":"66","author":"SV Glasner","year":"2005","journal-title":"J Stud Alcohol"},{"key":"pcbi.1011692.ref051","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1038\/s41583-020-0289-z","article-title":"The transition to compulsion in addiction.","volume":"21","author":"C L\u00fcscher","year":"2020","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011692.ref052","first-page":"201816704","article-title":"Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning","volume":"116","author":"L de Boer","year":"2018","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1011692.ref053","doi-asserted-by":"crossref","first-page":"2261","DOI":"10.1016\/j.neurobiolaging.2013.03.030","article-title":"Structural integrity of the substantia nigra and subthalamic nucleus predicts flexibility of instrumental learning in older-age individuals","volume":"34","author":"R Chowdhury","year":"2013","journal-title":"Neurobiol Aging"},{"key":"pcbi.1011692.ref054","doi-asserted-by":"crossref","first-page":"1940","DOI":"10.1126\/science.1102941","article-title":"By carrot or by stick: cognitive reinforcement learning in parkinsonism","volume":"306","author":"MJ Frank","year":"2004","journal-title":"Science"},{"key":"pcbi.1011692.ref055","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1002\/wcs.57","article-title":"Model-based approaches to neuroimaging: combining reinforcement learning theory with fMRI data.","volume":"1","author":"JP Gl\u00e4scher","year":"2010","journal-title":"Wiley Interdiscip Rev Cogn Sci"},{"key":"pcbi.1011692.ref056","doi-asserted-by":"crossref","first-page":"3573","DOI":"10.1093\/cercor\/bhz327","article-title":"Variability in Action Selection Relates to Striatal Dopamine 2\/3 Receptor Availability in Humans: A PET Neuroimaging Study Using Reinforcement Learning and Active Inference Models","volume":"30","author":"RA Adams","year":"2020","journal-title":"Cereb Cortex"},{"key":"pcbi.1011692.ref057","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.neurobiolaging.2020.08.011","article-title":"Learning in anticipation of reward and punishment: perspectives across the human lifespan","volume":"96","author":"MJ Betts","year":"2020","journal-title":"Neurobiol Aging"},{"key":"pcbi.1011692.ref058","doi-asserted-by":"crossref","first-page":"3416","DOI":"10.1038\/s41598-021-82889-8","article-title":"Overcoming Pavlovian bias in semantic space.","volume":"11","author":"S Ereira","year":"2021","journal-title":"Sci Rep."},{"key":"pcbi.1011692.ref059","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1007\/s00213-013-3313-4","article-title":"Differential, but not opponent, effects of l-DOPA and citalopram on action learning with reward and punishment","volume":"231","author":"M Guitart-Masip","year":"2014","journal-title":"Psychopharmacology"},{"key":"pcbi.1011692.ref060","doi-asserted-by":"crossref","first-page":"3340","DOI":"10.1093\/cercor\/bhz313","article-title":"The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study","volume":"30","author":"V Perosa","year":"2020","journal-title":"Cereb Cortex"},{"key":"pcbi.1011692.ref061","doi-asserted-by":"crossref","first-page":"140","DOI":"10.3389\/fnsys.2014.00140","article-title":"Valenced action\/inhibition learning in humans is modulated by a genetic variant linked to dopamine D2 receptor expression","volume":"8","author":"A Richter","year":"2014","journal-title":"Front Syst Neurosci"},{"key":"pcbi.1011692.ref062","first-page":"1","article-title":"Motivational learning biases are differentially modulated by genetic determinants of striatal and prefrontal dopamine function","author":"A Richter","year":"2021","journal-title":"J Neural Transm"},{"key":"pcbi.1011692.ref063","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1007\/s11222-016-9696-4","article-title":"Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC.","volume":"27","author":"A Vehtari","year":"2017","journal-title":"Stat Comput."},{"key":"pcbi.1011692.ref064","doi-asserted-by":"crossref","first-page":"8541","DOI":"10.1523\/JNEUROSCI.5754-12.2013","article-title":"Frontal Theta Overrides Pavlovian Learning Biases","volume":"33","author":"JF Cavanagh","year":"2013","journal-title":"J Neurosci"},{"key":"pcbi.1011692.ref065","author":"J. Kruschke","year":"2014","journal-title":"Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan."},{"key":"pcbi.1011692.ref066","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1006\/nimg.1997.0291","article-title":"Psychophysiological and modulatory interactions in neuroimaging.","volume":"6","author":"KJ Friston","year":"1997","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref067","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.1016\/j.neuroimage.2012.03.068","article-title":"A generalized form of context-dependent psychophysiological interactions (gPPI): A comparison to standard approaches.","volume":"61","author":"DG McLaren","year":"2012","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref068","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1093\/cercor\/bhg090","article-title":"Neuronal representation of response\u2014outcome in the primate prefrontal cortex","volume":"14","author":"S Tsujimoto","year":"2004","journal-title":"Cereb Cortex"},{"key":"pcbi.1011692.ref069","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"TE Behrens","year":"2007","journal-title":"Nat Neurosci"},{"key":"pcbi.1011692.ref070","doi-asserted-by":"crossref","first-page":"10340","DOI":"10.1523\/JNEUROSCI.0895-11.2011","article-title":"Dorsolateral Prefrontal Cortex Drives Mesolimbic Dopaminergic Regions to Initiate Motivated Behavior","volume":"31","author":"IC Ballard","year":"2011","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref071","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1016\/j.neuroimage.2010.05.058","article-title":"Decoding different roles for vmPFC and dlPFC in multi-attribute decision making.","volume":"56","author":"T Kahnt","year":"2011","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref072","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1196\/annals.1390.007","article-title":"Mechanisms of reinforcement learning and decision making in the primate dorsolateral prefrontal cortex","volume":"1104","author":"D Lee","year":"2007","journal-title":"Ann N Y Acad Sci"},{"key":"pcbi.1011692.ref073","doi-asserted-by":"crossref","first-page":"2069","DOI":"10.1046\/j.1460-9568.2003.02922.x","article-title":"Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task","volume":"18","author":"JD Wallis","year":"2003","journal-title":"Eur J Neurosci"},{"key":"pcbi.1011692.ref074","doi-asserted-by":"crossref","first-page":"7749","DOI":"10.1523\/JNEUROSCI.5587-09.2010","article-title":"Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence","volume":"30","author":"SQ Park","year":"2010","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref075","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1038\/nature03287","article-title":"Different time courses of learning-related activity in the prefrontal cortex and striatum","volume":"433","author":"A Pasupathy","year":"2005","journal-title":"Nature"},{"key":"pcbi.1011692.ref076","doi-asserted-by":"crossref","first-page":"3340","DOI":"10.1523\/JNEUROSCI.4313-13.2014","article-title":"Anterior Cingulate Cortex Instigates Adaptive Switches in Choice by Integrating Immediate and Delayed Components of Value in Ventromedial Prefrontal Cortex","volume":"34","author":"M Economides","year":"2014","journal-title":"J Neurosci"},{"key":"pcbi.1011692.ref077","doi-asserted-by":"crossref","first-page":"13214","DOI":"10.1523\/JNEUROSCI.1246-11.2011","article-title":"The Decision Value Computations in the vmPFC and Striatum Use a Relative Value Code That is Guided by Visual Attention","volume":"31","author":"S-L Lim","year":"2011","journal-title":"J Neurosci"},{"key":"pcbi.1011692.ref078","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1111\/j.1749-6632.2011.06290.x","article-title":"Contributions of the ventromedial prefrontal cortex to goal-directed action selection","volume":"1239","author":"JP O\u2019Doherty","year":"2011","journal-title":"Ann N Y Acad Sci"},{"key":"pcbi.1011692.ref079","doi-asserted-by":"crossref","first-page":"2490","DOI":"10.1523\/JNEUROSCI.3319-09.2010","article-title":"Distinct Value Signals in Anterior and Posterior Ventromedial Prefrontal Cortex","volume":"30","author":"DV Smith","year":"2010","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref080","doi-asserted-by":"crossref","first-page":"638","DOI":"10.1016\/j.biopsych.2017.10.030","article-title":"The Multifaceted Role of the Ventromedial Prefrontal Cortex in Emotion, Decision Making, Social Cognition, and Psychopathology","volume":"83","author":"J Hiser","year":"2018","journal-title":"Biol Psychiatry"},{"key":"pcbi.1011692.ref081","doi-asserted-by":"crossref","first-page":"5047","DOI":"10.1523\/JNEUROSCI.4236-15.2016","article-title":"Ventromedial Prefrontal Cortex Damage Is Associated with Decreased Ventral Striatum Volume and Response to Reward","volume":"36","author":"MS Pujara","year":"2016","journal-title":"J Neurosci"},{"key":"pcbi.1011692.ref082","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1038\/npp.2009.131","article-title":"Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action","volume":"35","author":"BW Balleine","year":"2010","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1011692.ref083","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"ND Daw","year":"2005","journal-title":"Nat Neurosci"},{"key":"pcbi.1011692.ref084","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans\u2019 choices and striatal prediction errors","volume":"69","author":"ND Daw","year":"2011","journal-title":"Neuron"},{"key":"pcbi.1011692.ref085","doi-asserted-by":"crossref","first-page":"e1002055","DOI":"10.1371\/journal.pcbi.1002055","article-title":"Speed\/accuracy trade-off between the habitual and the goal-directed processes.","volume":"7","author":"M Keramati","year":"2011","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1011692.ref086","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1098\/rstb.1985.0010","article-title":"Actions and habits: the development of behavioural autonomy.","volume":"308","author":"A Dickinson","year":"1985","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"key":"pcbi.1011692.ref087","doi-asserted-by":"crossref","first-page":"5826","DOI":"10.1038\/s41467-019-13737-7","article-title":"Controllability governs the balance between Pavlovian and instrumental action selection.","volume":"10","author":"HM Dorfman","year":"2019","journal-title":"Nat Commun"},{"key":"pcbi.1011692.ref088","doi-asserted-by":"crossref","first-page":"e2005979","DOI":"10.1371\/journal.pbio.2005979","article-title":"Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action","volume":"16","author":"JC Swart","year":"2018","journal-title":"PLoS Biol"},{"key":"pcbi.1011692.ref089","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1016\/j.tics.2014.04.012","article-title":"Frontal theta as a mechanism for cognitive control.","volume":"18","author":"JF Cavanagh","year":"2014","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011692.ref090","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.tics.2013.12.003","article-title":"Inhibition and the right inferior frontal cortex: one decade on.","volume":"18","author":"AR Aron","year":"2014","journal-title":"Trends Cogn Sci."},{"key":"pcbi.1011692.ref091","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/978-3-642-59794-7_4","article-title":"Segregation of working memory functions within the dorsolateral prefrontal cortex.","author":"R Levy","year":"2000","journal-title":"Executive control and the frontal lobe: Current issues."},{"key":"pcbi.1011692.ref092","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1007\/s002210000399","article-title":"The role of the mid-dorsolateral prefrontal cortex in working memory","volume":"133","author":"M. Petrides","year":"2000","journal-title":"Exp Brain Res"},{"key":"pcbi.1011692.ref093","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.tics.2018.12.006","article-title":"Dopamine and Cognitive Control in Prefrontal Cortex.","volume":"23","author":"T Ott","year":"2019","journal-title":"Trends Cogn Sci."},{"key":"pcbi.1011692.ref094","doi-asserted-by":"crossref","first-page":"e0152781","DOI":"10.1371\/journal.pone.0152781","article-title":"Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients.","volume":"11","author":"MA Albrecht","year":"2016","journal-title":"PLoS One."},{"key":"pcbi.1011692.ref095","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1002\/wcs.1401","article-title":"The costs and benefits of brain dopamine for cognitive control","volume":"7","author":"R. Cools","year":"2016","journal-title":"Wiley Interdiscip Rev Cogn Sci"},{"key":"pcbi.1011692.ref096","first-page":"1","article-title":"Depression is associated with enhanced aversive Pavlovian control over instrumental behaviour","volume":"8","author":"CL Nord","year":"2018","journal-title":"Sci Rep"},{"key":"pcbi.1011692.ref097","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1037\/0278-7393.33.3.570","article-title":"Time and Cognitive Load in Working Memory.","volume":"33","author":"P Barrouillet","year":"2007","journal-title":"J Exp Psychol Learn Mem Cogn."},{"key":"pcbi.1011692.ref098","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1016\/j.neubiorev.2004.09.006","article-title":"Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates.","volume":"28","author":"JW Dalley","year":"2004","journal-title":"Neurosci Biobehav Rev"},{"key":"pcbi.1011692.ref099","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1037\/0735-7044.108.5.883","article-title":"Working memory, response selection, and effortful processing in rats with medial prefrontal lesions.","volume":"108","author":"S Granon","year":"1994","journal-title":"Behav Neurosci"},{"key":"pcbi.1011692.ref100","doi-asserted-by":"crossref","first-page":"36","DOI":"10.5334\/joc.58","article-title":"Working Memory and Attention\u2013A Conceptual Analysis and Review","volume":"2","author":"K. Oberauer","year":"2019","journal-title":"Journal of Cognition"},{"key":"pcbi.1011692.ref101","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.bandc.2004.09.016","article-title":"Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning","volume":"56","author":"KR Ridderinkhof","year":"2004","journal-title":"Brain Cogn"},{"key":"pcbi.1011692.ref102","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.brainres.2012.08.034","article-title":"Reward processing and intertemporal decision making in adults and adolescents: The role of impulsivity and decision consistency","volume":"1478","author":"S Ripke","year":"2012","journal-title":"Brain Res"},{"key":"pcbi.1011692.ref103","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1016\/j.neuron.2012.03.037","article-title":"Action selection and action value in frontal-striatal circuits","volume":"74","author":"M Seo","year":"2012","journal-title":"Neuron"},{"key":"pcbi.1011692.ref104","doi-asserted-by":"crossref","first-page":"531","DOI":"10.3758\/BF03193069","article-title":"Response selection involves executive control: Evidence from the selective interference paradigm.","volume":"33","author":"A Szmalec","year":"2005","journal-title":"Mem Cognit."},{"key":"pcbi.1011692.ref105","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.cortex.2014.08.026","article-title":"Transcranial direct current stimulation over the left prefrontal cortex increases randomness of choice in instrumental learning.","volume":"63","author":"Z Turi","year":"2015","journal-title":"Cortex"},{"key":"pcbi.1011692.ref106","doi-asserted-by":"crossref","first-page":"1655","DOI":"10.1111\/joms.12528","article-title":"The Manager\u2019s Notepad: Working Memory, Exploration, and Performance","volume":"56","author":"D Laureiro-Martinez","year":"2019","journal-title":"Journal of Management Studies"},{"key":"pcbi.1011692.ref107","doi-asserted-by":"crossref","first-page":"100732","DOI":"10.1016\/j.dcn.2019.100732","article-title":"Disentangling the systems contributing to changes in learning during adolescence.","volume":"41","author":"SL Master","year":"2020","journal-title":"Dev Cogn Neurosci"},{"key":"pcbi.1011692.ref108","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cobeha.2016.04.003","article-title":"Taming the beast: extracting generalizable knowledge from computational models of cognition.","volume":"11","author":"MR Nassar","year":"2016","journal-title":"Current opinion in behavioral sciences."},{"key":"pcbi.1011692.ref109","doi-asserted-by":"crossref","first-page":"14617","DOI":"10.1523\/JNEUROSCI.2026-09.2009","article-title":"How humans integrate the prospects of pain and reward during choice","volume":"29","author":"D Talmi","year":"2009","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011692.ref110","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1038\/nn.4238","article-title":"Computational psychiatry as a bridge from neuroscience to clinical applications","volume":"19","author":"QJM Huys","year":"2016","journal-title":"Nat Neurosci"},{"key":"pcbi.1011692.ref111","doi-asserted-by":"crossref","first-page":"13681","DOI":"10.1073\/pnas.1502829112","article-title":"Emergence of system roles in normative neurodevelopment","volume":"112","author":"S Gu","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1011692.ref112","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1016\/j.neuroimage.2016.08.009","article-title":"A simple but useful way to assess fMRI scan qualities.","volume":"154","author":"JD Power","year":"2017","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref113","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1017\/S003329171700174X","article-title":"The impact of traumatic stress on Pavlovian biases.","volume":"48","author":"OT Ousdal","year":"2018","journal-title":"Psychol Med"},{"key":"pcbi.1011692.ref114","doi-asserted-by":"crossref","first-page":"e1006679","DOI":"10.1371\/journal.pcbi.1006679","article-title":"Change, stability, and instability in the Pavlovian guidance of behaviour from adolescence to young adulthood","volume":"14","author":"M Moutoussis","year":"2018","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1011692.ref115","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1162\/CPSY_a_00002","article-title":"Revealing Neurocomputational Mechanisms of Reinforcement Learning and Decision-Making With the hBayesDM Package.","volume":"1","author":"W-Y Ahn","year":"2017","journal-title":"Computational Psychiatry."},{"key":"pcbi.1011692.ref116","first-page":"59","article-title":"A general framework for the parametrization of hierarchical models.","author":"O Papaspiliopoulos","year":"2007","journal-title":"Stat Sci."},{"key":"pcbi.1011692.ref117","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.jmp.2008.12.001","article-title":"Bayesian parameter estimation in the Expectancy Valence model of the Iowa gambling task.","volume":"54","author":"R Wetzels","year":"2010","journal-title":"J Math Psychol"},{"key":"pcbi.1011692.ref118","article-title":"Rank-normalization, folding, and localization: An improved $\\widehatR$ for assessing convergence of MCMC.","author":"A Vehtari","year":"2019","journal-title":"arXiv preprint arXiv:190308008"},{"key":"pcbi.1011692.ref119","doi-asserted-by":"crossref","DOI":"10.1214\/ss\/1177011136","article-title":"Inference from Iterative Simulation Using Multiple Sequences.","volume":"7","author":"A Gelman","year":"1992","journal-title":"Stat Sci."},{"key":"pcbi.1011692.ref120","article-title":"fMRIPrep.","author":"O Esteban","year":"2018","journal-title":"Softw Pract Exp"},{"key":"pcbi.1011692.ref121","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1038\/s41592-018-0235-4","article-title":"fMRIPrep: a robust preprocessing pipeline for functional MRI","volume":"16","author":"O Esteban","year":"2019","journal-title":"Nat Methods"},{"key":"pcbi.1011692.ref122","doi-asserted-by":"crossref","first-page":"13","DOI":"10.3389\/fninf.2011.00013","article-title":"Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in Python.","volume":"5","author":"K Gorgolewski","year":"2011","journal-title":"Front Neuroinform"},{"key":"pcbi.1011692.ref123","article-title":"Nipype.","author":"KJ Gorgolewski","year":"2018","journal-title":"Softw Pract Exp"},{"key":"pcbi.1011692.ref124","doi-asserted-by":"crossref","first-page":"435","DOI":"10.3758\/s13415-015-0338-7","article-title":"Reinforcement learning models and their neural correlates: An activation likelihood estimation meta-analysis.","volume":"15","author":"HW Chase","year":"2015","journal-title":"Cogn Affect Behav Neurosci"},{"key":"pcbi.1011692.ref125","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1016\/j.neuron.2009.05.014","article-title":"How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action","volume":"62","author":"ED Boorman","year":"2009","journal-title":"Neuron"},{"key":"pcbi.1011692.ref126","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural Computations Underlying Arbitration between Model-Based and Model-free Learning","volume":"81","author":"SW Lee","year":"2014","journal-title":"Neuron"},{"key":"pcbi.1011692.ref127","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1016\/j.neuroimage.2012.04.014","article-title":"Multiple testing corrections, nonparametric methods, and random field theory.","volume":"62","author":"TE Nichols","year":"2012","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref128","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1016\/j.neuroimage.2013.12.058","article-title":"Cluster-extent based thresholding in fMRI analyses: pitfalls and recommendations.","volume":"91","author":"C-W Woo","year":"2014","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref129","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.bandc.2019.01.002","article-title":"A coordinate-based meta-analysis of the n-back working memory paradigm using activation likelihood estimation","volume":"132","author":"H Wang","year":"2019","journal-title":"Brain Cogn"},{"key":"pcbi.1011692.ref130","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1016\/j.neuroimage.2013.02.063","article-title":"The valuation system: A coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value.","volume":"76","author":"O Bartra","year":"2013","journal-title":"Neuroimage"},{"key":"pcbi.1011692.ref131","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.1016\/j.neubiorev.2010.12.012","article-title":"Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies.","volume":"35","author":"X Liu","year":"2011","journal-title":"Neurosci Biobehav Rev"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011692","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011692","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T13:48:12Z","timestamp":1703080092000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011692"}},"subtitle":[],"editor":[{"given":"Alireza","family":"Soltani","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2023,12,8]]},"references-count":131,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,12,8]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011692","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.08.01.502269","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,8]]}}}