{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T08:14:54Z","timestamp":1787904894453,"version":"build-2784847793"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T00:00:00Z","timestamp":1685059200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["31671135"],"award-info":[{"award-number":["31671135"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["31871113"],"award-info":[{"award-number":["31871113"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["31920103009"],"award-info":[{"award-number":["31920103009"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32071049"],"award-info":[{"award-number":["32071049"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["32171081"],"award-info":[{"award-number":["32171081"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100021171","name":"Basic and Applied Basic Research Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["2022A1515012185"],"award-info":[{"award-number":["2022A1515012185"]}],"id":[{"id":"10.13039\/501100021171","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100018537","name":"National Science and Technology Major Project","doi-asserted-by":"publisher","award":["2021ZD0203800"],"award-info":[{"award-number":["2021ZD0203800"]}],"id":[{"id":"10.13039\/501100018537","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Psychiatry"],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Obsessive\u2013compulsive disorder (OCD) is characterized by an imbalance between goal-directed and habitual learning systems in behavioral control, but it is unclear whether these impairments are due to a single system abnormality of the goal-directed system or due to an impairment in a separate arbitration mechanism that selects which system controls behavior at each point in time.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>A total of 30 OCD patients and 120 healthy controls performed a 2-choice, 3-stage Markov decision-making paradigm. Reinforcement learning models were used to estimate goal-directed learning (as model-based reinforcement learning) and habitual learning (as model-free reinforcement learning). In general, 29 high Obsessive\u2013Compulsive Inventory-Revised (OCI-R) score controls, 31 low OCI-R score controls, and all 30 OCD patients were selected for the analysis.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Obsessive\u2013compulsive disorder (OCD) patients showed less appropriate strategy choices than controls regardless of whether the OCI-R scores in the control subjects were high (<jats:italic>p<\/jats:italic> = 0.012) or low (<jats:italic>p<\/jats:italic> &amp;lt; 0.001), specifically showing a greater model-free strategy use in task conditions where the model-based strategy was optimal. Furthermore, OCD patients (<jats:italic>p<\/jats:italic> = 0.001) and control subjects with high OCI-R scores (H-OCI-R; <jats:italic>p<\/jats:italic> = 0.009) both showed greater system switching rather than consistent strategy use in task conditions where model-free use was optimal.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>These findings indicated an impaired arbitration mechanism for flexible adaptation to environmental demands in both OCD patients and healthy individuals reporting high OCI-R scores.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fpsyt.2023.1162800","type":"journal-article","created":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T04:46:44Z","timestamp":1685076404000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Impairment of arbitration between model-based and model-free reinforcement learning in obsessive\u2013compulsive disorder"],"prefix":"10.3389","volume":"14","author":[{"given":"Zhongqiang","family":"Ruan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carol A.","family":"Seger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiong","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dongjae","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sang Wan","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qi","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziwen","family":"Peng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,5,26]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.neuron.2013.09.007","article-title":"Goals and habits in the brain","volume":"80","author":"Dolan","year":"2013","journal-title":"Neuron."},{"key":"B2","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1038\/npp.2009.131","article-title":"Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action","volume":"35","author":"Balleine","year":"2010","journal-title":"Neuropsychopharmacol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"718","DOI":"10.1176\/appi.ajp.2011.10071062","article-title":"Disruption in the balance between goal-directed behavior and habit learning in obsessive-compulsive disorder","volume":"168","author":"Gillan","year":"2011","journal-title":"Am J Psychiat."},{"key":"B4","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1177\/1073858414568317","article-title":"Arbitration between action strategies in obsessive-compulsive disorder","volume":"22","author":"Gruner","year":"2016","journal-title":"Neuroscientist."},{"key":"B5","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nat Neurosci."},{"key":"B6","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans' choices and striatal prediction errors","volume":"69","author":"Daw","year":"2011","journal-title":"Neuron."},{"key":"B7","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1016\/j.tics.2011.11.018","article-title":"Computational psychiatry","volume":"16","author":"Montague","year":"2012","journal-title":"Trends Cogn Sci."},{"key":"B8","doi-asserted-by":"publisher","first-page":"1497","DOI":"10.1038\/s41593-018-0258-2","article-title":"Are we of two minds?","volume":"21","author":"Daw","year":"2018","journal-title":"Nat Neurosci."},{"key":"B9","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"Lee","year":"2014","journal-title":"Neuron."},{"key":"B10","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1146\/annurev-psych-010416-044216","article-title":"Learning, reward, and decision making","volume":"68","author":"O'Doherty","year":"2017","journal-title":"Annu Rev Psychol."},{"key":"B11","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.neubiorev.2020.10.022","article-title":"Why and how the brain weights contributions from a mixture of experts","volume":"123","author":"O'Doherty","year":"2021","journal-title":"Neurosci Biobehav R."},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7554\/eLife.11305","article-title":"Characterizing a psychiatric symptom dimension related to deficits in goal-directed control","volume":"5","author":"Gillan","year":"2016","journal-title":"Elife."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/mp.2014.44","article-title":"Disorders of compulsivity: a common bias towards learning habits","volume":"1","author":"Voon","year":"2014","journal-title":"Mol Psychiatr."},{"key":"B14","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1016\/j.biopsych.2017.04.006","article-title":"Model-based control in dimensional psychiatry","volume":"82","author":"Voon","year":"2017","journal-title":"Biol Psychiat."},{"key":"B15","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1001\/jamapsychiatry.2019.2998","article-title":"Comparison of the association between goal-directed planning and self-reported compulsivity vs obsessive-compulsive disorder diagnosis","volume":"77","author":"Gillan","year":"2020","journal-title":"JAMA Psychiat."},{"key":"B16","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1176\/appi.ajp.2014.14040525","article-title":"Functional neuroimaging of avoidance habits in obsessive-compulsive disorder","volume":"172","author":"Gillan","year":"2015","journal-title":"Am J Psychiat."},{"key":"B17","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1016\/j.biopsych.2013.02.002","article-title":"Enhanced avoidance habits in obsessive-compulsive disorder","volume":"75","author":"Gillan","year":"2014","journal-title":"Biol Psychiat"},{"key":"B18","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1093\/brain\/awu379","article-title":"Imbalance in habitual versus goal directed neural systems during symptom provocation in obsessive-compulsive disorder","volume":"138","author":"Banca","year":"2015","journal-title":"Brain."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1098\/rstb.2013.0475","article-title":"Goal-directed learning and obsessive-compulsive disorder","volume":"369","author":"Gillan","year":"2014","journal-title":"Philos T R Soc B."},{"key":"B20","doi-asserted-by":"publisher","first-page":"992","DOI":"10.1016\/j.bpsc.2020.12.004","article-title":"White matter alterations of the goal-directed system in patients with obsessive-compulsive disorder and their unaffected first-degree relatives","volume":"6","author":"Peng","year":"2021","journal-title":"Biol Psychiat-Cogn N."},{"key":"B21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/cercor\/bhab441","article-title":"Imbalance in functional and structural connectivity underlying goal-directed and habitual learning systems in obsessive-compulsive disorder","volume":"1","author":"Xu","year":"2021","journal-title":"Cereb Cortex."},{"key":"B22","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.1002\/jnr.24594","article-title":"The two-step task, avoidance, and OCD","volume":"98","author":"Geramita","year":"2020","journal-title":"J Neurosci Res."},{"key":"B23","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1037\/1040-3590.14.4.485","article-title":"The obsessive-compulsive inventory: development and validation of a short version","volume":"14","author":"Foa","year":"2002","journal-title":"Psychol Assessment."},{"key":"B24","first-page":"22","article-title":"The Mini-International Neuropsychiatric Interview (MINI): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10","volume":"59","author":"Sheehan","year":"1998","journal-title":"J Clin Psychiat."},{"key":"B25","doi-asserted-by":"publisher","first-page":"1006","DOI":"10.1001\/archpsyc.1989.01810110048007","article-title":"De yale-brown obsessive compulsive scale","volume":"46","author":"Goodman","year":"1999","journal-title":"Arch of Gen Psychiat."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/cercor\/bhz019","article-title":"Neurostimulation reveals context-dependent arbitration between model-based and model-free reinforcement learning","volume":"1","author":"Weissengruber","year":"2019","journal-title":"Cereb Cortex."},{"key":"B27","volume-title":"Manual for the State-Trait Anxiety Inventory (Form Y1 \u2013 Y2)","author":"Spielberger","year":"1983"},{"key":"B28","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1001\/archpsyc.1961.01710120031004","article-title":"An inventory for measuring depression","volume":"4","author":"Beck","year":"1961","journal-title":"Arch of Gen Psychiat."},{"key":"B29","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1002\/jclp.1051","article-title":"Psychometric properties of an Italian version of the Barratt Impulsiveness Scale-11 (BIS-11) in nonclinical subjects","volume":"57","author":"Fossati","year":"2001","journal-title":"J Clin Psychol."},{"key":"B30","article-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"1998","journal-title":"Cambridge: MIT Press."},{"key":"B31","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"Gl\u00e4scher","year":"2010","journal-title":"Neuron."},{"key":"B32","volume-title":"Individual Choice Behavior","author":"Luce","year":"1959"},{"key":"B33","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1137\/S1052623496303470","article-title":"Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions","volume":"9","author":"Lagarias","year":"1998","journal-title":"SIAM J Optimiz."},{"key":"B34","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1177\/2167702612472487","article-title":"Mild uncertainty promotes checking behavior in subclinical obsessive-compulsive disorder","volume":"1","author":"Toffolo","year":"2013","journal-title":"Clin Psychol Sci."},{"key":"B35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1009003","article-title":"Effects of subclinical depression on prefrontal striatal model-based and model-free learning","volume":"17","author":"Heo","year":"2021","journal-title":"PLoS Comput Biol."},{"key":"B36","doi-asserted-by":"publisher","first-page":"5738","DOI":"10.1038\/s41467-019-13632-1","article-title":"O Doherty JP, Lee SW. Task complexity interacts with state-space uncertainty in the arbitration between model-based and model-free learning","volume":"10","author":"Kim","year":"2019","journal-title":"Nat Commun."},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/tp.2015.165","article-title":"Motivation and value influences in the relative balance of goal-directed and habitual behaviours in obsessive-compulsive disorder","volume":"5","author":"Voon","year":"2015","journal-title":"Transl Psychiat."},{"key":"B38","doi-asserted-by":"publisher","first-page":"2337","DOI":"10.1007\/s00213-019-05325-w","article-title":"Computational modelling reveals contrasting effects on reinforcement learning and cognitive flexibility in stimulant use disorder and obsessive-compulsive disorder: remediating effects of dopaminergic D2\/3 receptor agents","volume":"236","author":"Kanen","year":"2019","journal-title":"Psychopharmacology."},{"key":"B39","doi-asserted-by":"publisher","first-page":"4583","DOI":"10.1038\/s41380-020-01007-8","article-title":"Toward a neurocircuit-based taxonomy to guide treatment of obsessive\u2013compulsive disorder","volume":"26","author":"Shephard","year":"2021","journal-title":"Mol Psychiatr."},{"key":"B40","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: an integrative theory of anterior cingulate cortex function","volume":"79","author":"Shenhav","year":"2013","journal-title":"Neuron."},{"key":"B41","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1523\/ENEURO.0240-18.2018","article-title":"Conflicted between goal-directed and habitual control, an fMRI investigation","volume":"5","author":"Watson","year":"2018","journal-title":"eNeuro."},{"key":"B42","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1159\/000325223","article-title":"Functional neuroimaging in obsessive-compulsive disorder","volume":"64","author":"Del Casale","year":"2011","journal-title":"Neuropsychobiology."},{"key":"B43","doi-asserted-by":"publisher","first-page":"5306","DOI":"10.1002\/hbm.22551","article-title":"Independent component analysis of resting state activity in pediatric obsessive-compulsive disorder","volume":"35","author":"Gruner","year":"2014","journal-title":"Hum Brain Mapp."},{"key":"B44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.celrep.2021.110185","article-title":"Prefrontal solution to the bias-variance tradeoff during reinforcement learning","volume":"37","author":"Kim","year":"2021","journal-title":"Cell Rep."},{"key":"B45","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.neuron.2019.01.046","article-title":"Review obsessive-compulsive disorder: puzzles and prospects","volume":"102","author":"Robbins","year":"2019","journal-title":"Neuron."},{"key":"B46","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1038\/s41562-020-0905-y","article-title":"Silva C, Hare T. Humans primarily use model-based inference in the two-stage task","volume":"4","author":"Feher","year":"2020","journal-title":"Nat Hum Behav."}],"container-title":["Frontiers in Psychiatry"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyt.2023.1162800\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,26]],"date-time":"2023-05-26T04:46:52Z","timestamp":1685076412000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyt.2023.1162800\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,26]]},"references-count":46,"alternative-id":["10.3389\/fpsyt.2023.1162800"],"URL":"https:\/\/doi.org\/10.3389\/fpsyt.2023.1162800","relation":{},"ISSN":["1664-0640"],"issn-type":[{"value":"1664-0640","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,26]]},"article-number":"1162800"}}