{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T10:25:19Z","timestamp":1782210319246,"version":"3.54.5"},"reference-count":46,"publisher":"Public Library of Science (PLoS)","issue":"5","license":[{"start":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T00:00:00Z","timestamp":1651708800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008510","name":"University of Maryland-College Park","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100008510","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>A large literature has accumulated suggesting that human and animal decision making is driven by at least two systems, and that important functions of these systems can be captured by reinforcement learning algorithms. The \u201cmodel-free\u201d system caches and uses stimulus\u2013value or stimulus\u2013response associations, and the \u201cmodel-based\u201d system implements more flexible planning using a model of the world. However, it is not clear how the two systems interact during deliberation and how a single decision emerges from this process, especially when they disagree. Most previous work has assumed that while the systems operate in parallel, they do so independently, and they combine linearly to influence decisions. Using an integrated reinforcement learning\/drift-diffusion model, we tested the hypothesis that the two systems interact in a non-linear fashion similar to other situations with cognitive conflict. We differentiated two forms of conflict:<jats:italic>action conflict<\/jats:italic>, a binary state representing whether the systems disagreed on the best action, and<jats:italic>value conflict<\/jats:italic>, a continuous measure of the extent to which the two systems disagreed on the difference in value between the available options. We found that decisions with greater value conflict were characterized by reduced model-based control and increased caution both with and without action conflict. Action conflict itself (the binary state) acted in the opposite direction, although its effects were less prominent. We also found that between-system conflict was highly correlated with within-system conflict, and although it is less clear a priori why the latter might influence the strength of each system above its standard linear contribution, we could not rule it out. Our work highlights the importance of non-linear conflict effects, and provides new constraints for more detailed process models of decision making. It also presents new avenues to explore with relation to disorders of compulsivity, where an imbalance between systems has been implicated.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010047","type":"journal-article","created":{"date-parts":[[2022,5,5]],"date-time":"2022-05-05T17:35:20Z","timestamp":1651772120000},"page":"e1010047","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":3,"title":["Conflict and competition between model-based and model-free control"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4294-6577","authenticated-orcid":true,"given":"Yuqing","family":"Lei","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8851-2200","authenticated-orcid":true,"given":"Alec","family":"Solway","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2022,5,5]]},"reference":[{"issue":"3","key":"pcbi.1010047.ref001","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1016\/j.conb.2011.02.009","article-title":"Multiplicity of control in the basal ganglia: computational roles of striatal subregions","volume":"21","author":"AM Bornstein","year":"2011","journal-title":"Current Opinion in Neurobiology"},{"issue":"45","key":"pcbi.1010047.ref002","doi-asserted-by":"crossref","first-page":"13817","DOI":"10.1073\/pnas.1506367112","article-title":"Habitual control of goal selection in humans","volume":"112","author":"F Cushman","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"12","key":"pcbi.1010047.ref003","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"ND Daw","year":"2005","journal-title":"Nature Neuroscience"},{"issue":"6","key":"pcbi.1010047.ref004","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans\u2019 choices and striatal prediction errors","volume":"69","author":"ND Daw","year":"2011","journal-title":"Neuron"},{"issue":"12","key":"pcbi.1010047.ref005","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1003364","article-title":"Actions, action sequences and habits: evidence that goal-directed and habitual action control are hierarchically organized","volume":"9","author":"A Dezfouli","year":"2013","journal-title":"PLOS Computational Biology"},{"issue":"4","key":"pcbi.1010047.ref006","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"J Gl\u00e4scher","year":"2010","journal-title":"Neuron"},{"issue":"8","key":"pcbi.1010047.ref007","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1005090","article-title":"When does model-based control pay off?","volume":"12","author":"W Kool","year":"2016","journal-title":"PLOS Computational Biology"},{"issue":"4","key":"pcbi.1010047.ref008","article-title":"Neural computations mediating one-shot learning in the human brain","volume":"13","author":"SW Lee","year":"2015","journal-title":"PLOS Biology"},{"issue":"14","key":"pcbi.1010047.ref009","doi-asserted-by":"crossref","first-page":"5526","DOI":"10.1523\/JNEUROSCI.4647-10.2011","article-title":"Neural correlates of forward planning in a spatial decision task in humans","volume":"31","author":"DA Simon","year":"2011","journal-title":"Journal of Neuroscience"},{"issue":"3","key":"pcbi.1010047.ref010","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.jmp.2008.12.005","article-title":"Reinforcement learning in the brain","volume":"53","author":"Y Niv","year":"2009","journal-title":"Journal of Mathematical Psychology"},{"issue":"5306","key":"pcbi.1010047.ref011","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"issue":"3","key":"pcbi.1010047.ref012","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"SW Lee","year":"2014","journal-title":"Neuron"},{"issue":"5","key":"pcbi.1010047.ref013","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1038\/nn.3981","article-title":"Model-based choices involve prospective neural activity","volume":"18","author":"BB Doll","year":"2015","journal-title":"Nature Neuroscience"},{"key":"pcbi.1010047.ref014","doi-asserted-by":"crossref","first-page":"915","DOI":"10.3389\/fnins.2019.00915","article-title":"Loss aversion correlates with the propensity to deploy model-based control","volume":"13","author":"A Solway","year":"2019","journal-title":"Frontiers in Neuroscience"},{"key":"pcbi.1010047.ref015","doi-asserted-by":"crossref","first-page":"e11305","DOI":"10.7554\/eLife.11305","article-title":"Characterizing a psychiatric symptom dimension related to deficits in goal-directed control","volume":"5","author":"CM Gillan","year":"2016","journal-title":"eLife"},{"issue":"1","key":"pcbi.1010047.ref016","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1001\/jamapsychiatry.2019.2998","article-title":"Comparison of the association between goal-directed planning and self-reported compulsivity vs obsessive-compulsive disorder diagnosis","volume":"77","author":"CM Gillan","year":"2020","journal-title":"JAMA Psychiatry"},{"issue":"3","key":"pcbi.1010047.ref017","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1038\/mp.2014.44","article-title":"Disorders of compulsivity: a common bias towards learning habits","volume":"20","author":"V Voon","year":"2015","journal-title":"Molecular Psychiatry"},{"issue":"5","key":"pcbi.1010047.ref018","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1177\/0956797612463080","article-title":"The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive","volume":"24","author":"AR Otto","year":"2013","journal-title":"Psychological Science"},{"issue":"52","key":"pcbi.1010047.ref019","doi-asserted-by":"crossref","first-page":"20941","DOI":"10.1073\/pnas.1312011110","article-title":"Working-memory capacity protects model-based learning from stress","volume":"110","author":"AR Otto","year":"2013","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"3","key":"pcbi.1010047.ref020","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1037\/0033-295X.108.3.624","article-title":"Conflict monitoring and cognitive control","volume":"108","author":"MM Botvinick","year":"2001","journal-title":"Psychological Review"},{"issue":"2","key":"pcbi.1010047.ref021","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: an integrative theory of anterior cingulate cortex function","volume":"79","author":"A Shenhav","year":"2013","journal-title":"Neuron"},{"issue":"3","key":"pcbi.1010047.ref022","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1037\/0033-295X.97.3.332","article-title":"On the control of automatic processes: a parallel distributed processing account of the Stroop effect","volume":"97","author":"JD Cohen","year":"1990","journal-title":"Psychological Review"},{"issue":"2","key":"pcbi.1010047.ref023","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1037\/0033-2909.109.2.163","article-title":"Half a century of research on the Stroop effect: an integrative review","volume":"109","author":"CM MacLeod","year":"1991","journal-title":"Psychological Bulletin"},{"issue":"2","key":"pcbi.1010047.ref024","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1037\/0033-295X.85.2.59","article-title":"A theory of memory retrieval","volume":"85","author":"R Ratcliff","year":"1978","journal-title":"Psychological Review"},{"issue":"4","key":"pcbi.1010047.ref025","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1162\/neco.2008.12-06-420","article-title":"The diffusion decision model: theory and data for two-choice decision tasks","volume":"20","author":"R Ratcliff","year":"2008","journal-title":"Neural Computation"},{"key":"pcbi.1010047.ref026","article-title":"The neural basis of decision making","volume":"30","author":"JI Gold","year":"2007","journal-title":"Annual Review of Neuroscience"},{"issue":"3","key":"pcbi.1010047.ref027","doi-asserted-by":"crossref","first-page":"697","DOI":"10.1037\/a0033152","article-title":"Modeling confidence judgments, response times, and multiple choices in decision making: recognition memory and motion discrimination","volume":"120","author":"R Ratcliff","year":"2013","journal-title":"Psychological Review"},{"issue":"11","key":"pcbi.1010047.ref028","doi-asserted-by":"crossref","first-page":"1462","DOI":"10.1038\/nn.2925","article-title":"Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold","volume":"14","author":"JF Cavanagh","year":"2011","journal-title":"Nature Neuroscience"},{"issue":"2","key":"pcbi.1010047.ref029","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1523\/JNEUROSCI.2036-14.2015","article-title":"fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning","volume":"35","author":"MJ Frank","year":"2015","journal-title":"Journal of Neuroscience"},{"issue":"10","key":"pcbi.1010047.ref030","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1038\/nn.2635","article-title":"Visual fixations and the computation and comparison of value in simple choice","volume":"13","author":"I Krajbich","year":"2010","journal-title":"Nature Neuroscience"},{"issue":"6","key":"pcbi.1010047.ref031","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1017\/S1930297500001285","article-title":"The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure","volume":"5","author":"MM Mormann","year":"2010","journal-title":"Judgment and Decision Making"},{"issue":"4","key":"pcbi.1010047.ref032","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1257\/jep.25.4.3","article-title":"Neuroeconomic Foundations of Economic Choice\u2013Recent Advances","volume":"25","author":"E Fehr","year":"2011","journal-title":"Journal of Economic Perspectives"},{"issue":"2","key":"pcbi.1010047.ref033","doi-asserted-by":"crossref","first-page":"e1006803","DOI":"10.1371\/journal.pcbi.1006803","article-title":"Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling","volume":"15","author":"N Shahar","year":"2019","journal-title":"PLOS Computational Biology"},{"issue":"37","key":"pcbi.1010047.ref034","doi-asserted-by":"crossref","first-page":"11708","DOI":"10.1073\/pnas.1505483112","article-title":"Evidence integration in model-based tree search","volume":"112","author":"A Solway","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"7","key":"pcbi.1010047.ref035","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1016\/j.cub.2016.01.051","article-title":"Neural correlates of decision thresholds in the human subthalamic nucleus","volume":"26","author":"DM Herz","year":"2016","journal-title":"Current Biology"},{"key":"pcbi.1010047.ref036","volume-title":"On-line Q-learning using connectionist systems","author":"GA Rummery","year":"1994"},{"issue":"1","key":"pcbi.1010047.ref037","doi-asserted-by":"crossref","DOI":"10.18637\/jss.v076.i01","article-title":"Stan: A probabilistic programming language","volume":"76","author":"B Carpenter","year":"2017","journal-title":"Journal of Statistical Software"},{"key":"pcbi.1010047.ref038","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511790942","volume-title":"Data analysis using regression and multilevel\/hierarchical models","author":"A Gelman","year":"2006"},{"issue":"1324","key":"pcbi.1010047.ref039","article-title":"Model complexity in diffusion modeling: Benefits of making the model more parsimonious","volume":"7","author":"V Lerche","year":"2016","journal-title":"Frontiers in Psychology"},{"issue":"12","key":"pcbi.1010047.ref040","doi-asserted-by":"crossref","first-page":"e1004648","DOI":"10.1371\/journal.pcbi.1004648","article-title":"Simple plans or sophisticated habits? State, transition and learning interactions in the two-step task","volume":"11","author":"T Akam","year":"2015","journal-title":"PLOS Computational Biology"},{"issue":"4","key":"pcbi.1010047.ref041","doi-asserted-by":"crossref","first-page":"700","DOI":"10.1037\/0033-295X.113.4.700","article-title":"The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks","volume":"113","author":"R Bogacz","year":"2006","journal-title":"Psychological Review"},{"issue":"3","key":"pcbi.1010047.ref042","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.tics.2010.01.003","article-title":"Statistically optimal perception and learning: from behavior to neural representations","volume":"14","author":"J Fiser","year":"2010","journal-title":"Trends in Cognitive Sciences"},{"issue":"1","key":"pcbi.1010047.ref043","doi-asserted-by":"crossref","first-page":"286","DOI":"10.3758\/s13423-017-1271-2","article-title":"Testing the validity of conflict drift-diffusion models for use in estimating cognitive processes: A parameter-recovery study","volume":"25","author":"CN White","year":"2018","journal-title":"Psychonomic Bulletin & Review"},{"issue":"2","key":"pcbi.1010047.ref044","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1037\/rev0000087","article-title":"A dynamic dual process model of risky decision making","volume":"125","author":"A Diederich","year":"2018","journal-title":"Psychological Review"},{"issue":"2","key":"pcbi.1010047.ref045","doi-asserted-by":"crossref","first-page":"730","DOI":"10.3758\/s13428-017-0901-y","article-title":"Bayesian analysis of the piecewise diffusion decision model","volume":"50","author":"WR Holmes","year":"2018","journal-title":"Behavior Research Methods"},{"issue":"12438","key":"pcbi.1010047.ref046","article-title":"Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning","volume":"7","author":"A Konovalov","year":"2016","journal-title":"Nature Communications"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010047","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,4]],"date-time":"2023-02-04T13:33:32Z","timestamp":1675517612000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010047"}},"subtitle":[],"editor":[{"given":"Tobias U","family":"Hauser","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,5,5]]},"references-count":46,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5,5]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010047","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,5]]}}}