{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T08:22:51Z","timestamp":1768551771270,"version":"3.49.0"},"posted":{"date-parts":[[2020,9,8]]},"group-title":"Psychiatry and Clinical Psychology","reference-count":72,"publisher":"openRxiv","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2021,9,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>Explicit information obtained through instruction profoundly shapes human choice behaviour. However, this has been studied in computationally simple tasks, and it is unknown how model-based and model-free systems, respectively generating goal-directed and habitual actions, are affected by the absence or presence of instructions. We assessed behaviour in a novel variant of a computationally more complex decision-making task, before and after providing information about task structure, both in healthy volunteers and individuals suffering from obsessive-compulsive (OCD) or other disorders. Initial behaviour was model-free, with rewards directly reinforcing preceding actions. Model-based control, employing predictions of states resulting from each action, emerged with experience in a minority of subjects, and less in OCD. Providing task structure information strongly increased model-based control, similarly across all groups. Thus, explicit task structural knowledge determines human use of model-based reinforcement learning, and is most readily acquired from instruction rather than experience.<\/jats:p>","DOI":"10.1101\/2020.09.06.20189241","type":"posted-content","created":{"date-parts":[[2020,9,8]],"date-time":"2020-09-08T11:00:11Z","timestamp":1599562811000},"source":"Crossref","is-referenced-by-count":1,"title":["Explicit knowledge of task structure is the primary determinant of human model-based action"],"prefix":"10.64898","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3911-7164","authenticated-orcid":false,"given":"Pedro","family":"Castro-Rodrigues","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1810-0494","authenticated-orcid":false,"given":"Thomas","family":"Akam","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5979-3151","authenticated-orcid":false,"given":"Ivar","family":"Snorasson","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1490-5703","authenticated-orcid":false,"given":"M","family":"Marta Camacho","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3722-0747","authenticated-orcid":false,"given":"Vitor","family":"Paix\u00e3o","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8558-3126","authenticated-orcid":false,"given":"J. Bernardo","family":"Barahona-Corr\u00eaa","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3476-1839","authenticated-orcid":false,"given":"Peter","family":"Dayan","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1860-7874","authenticated-orcid":false,"given":"H. Blair","family":"Simpson","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0495-8374","authenticated-orcid":false,"given":"Rui M.","family":"Costa","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5071-3007","authenticated-orcid":false,"given":"Albino J.","family":"Oliveira-Maia","sequence":"additional","affiliation":[]}],"member":"54368","reference":[{"key":"2021092009150729000_2020.09.06.20189241v2.1","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.1985.0010"},{"key":"2021092009150729000_2020.09.06.20189241v2.2","doi-asserted-by":"publisher","DOI":"10.1037\/\/0033-2909.119.1.3"},{"key":"2021092009150729000_2020.09.06.20189241v2.3","first-page":"697","article-title":"A perspective on judgment and choice: Mapping bounded rationality","volume":"58","year":"2003","journal-title":"Behav. Sci"},{"key":"2021092009150729000_2020.09.06.20189241v2.4","doi-asserted-by":"publisher","DOI":"10.1038\/nn1560"},{"key":"2021092009150729000_2020.09.06.20189241v2.5","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2013.09.007"},{"key":"2021092009150729000_2020.09.06.20189241v2.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2017.09.060"},{"key":"2021092009150729000_2020.09.06.20189241v2.7","first-page":"109","article-title":"Instrumental responding following reinforcer devaluation","volume":"33","year":"1981","journal-title":"Q. J. Exp. Psychol. Sect. B Comp. Physiol. Psychol"},{"key":"2021092009150729000_2020.09.06.20189241v2.8","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1080\/14640748208400878","article-title":"Variations in the sensitivity of instrumental responding to reinforcer devaluation","volume":"34","year":"1982","journal-title":"Q. J. Exp. Psychol. Sect. B"},{"key":"2021092009150729000_2020.09.06.20189241v2.9","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.11.1.120"},{"key":"2021092009150729000_2020.09.06.20189241v2.10","unstructured":"Sutton, R. S. & Barto, A. G. Introduction to Reinforcement Learning. 4, (1998)."},{"key":"2021092009150729000_2020.09.06.20189241v2.11","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2011.02.027"},{"key":"2021092009150729000_2020.09.06.20189241v2.12","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2013.11.028"},{"key":"2021092009150729000_2020.09.06.20189241v2.13","doi-asserted-by":"publisher","DOI":"10.1126\/science.aac6076"},{"key":"2021092009150729000_2020.09.06.20189241v2.14","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2010.04.016"},{"key":"2021092009150729000_2020.09.06.20189241v2.15","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3068"},{"key":"2021092009150729000_2020.09.06.20189241v2.16","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1312011110"},{"key":"2021092009150729000_2020.09.06.20189241v2.17","doi-asserted-by":"publisher","DOI":"10.1177\/0956797612463080"},{"key":"2021092009150729000_2020.09.06.20189241v2.18","doi-asserted-by":"crossref","unstructured":"Skatova, A. , Chan, P. A. & Daw, N. D. Extraversion differentiates between model-based and model-free strategies in a reinforcement learning task. Front. Hum. Neurosci. 7, (2013).","DOI":"10.3389\/fnhum.2013.00525"},{"key":"2021092009150729000_2020.09.06.20189241v2.19","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2013.00253"},{"key":"2021092009150729000_2020.09.06.20189241v2.20","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2013.08.009"},{"key":"2021092009150729000_2020.09.06.20189241v2.21","doi-asserted-by":"crossref","unstructured":"Schad, D. J. et al. Processing speed enhances model-based over model-free reinforcement learning in the presence of high working memory functioning. Front. Psychol. 5, (2014).","DOI":"10.3389\/fpsyg.2014.01450"},{"key":"2021092009150729000_2020.09.06.20189241v2.22","doi-asserted-by":"publisher","DOI":"10.1016\/j.psyneuen.2014.12.017"},{"key":"2021092009150729000_2020.09.06.20189241v2.23","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1417219112"},{"key":"2021092009150729000_2020.09.06.20189241v2.24","doi-asserted-by":"crossref","unstructured":"Economides, M. , Kurth-Nelson, Z. , L\u00fcbbert, A. , Guitart-Masip, M. & Dolan, R. J. Model-Based Reasoning in Humans Becomes Automatic with Training. PLoS Comput. Biol. 11, (2015).","DOI":"10.1371\/journal.pcbi.1004463"},{"key":"2021092009150729000_2020.09.06.20189241v2.25","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1038\/mp.2015.46","article-title":"Valence-dependent influence of serotonin depletion on model-based choice strategy","volume":"21","year":"2016","journal-title":"Mol. Psychiatry"},{"key":"2021092009150729000_2020.09.06.20189241v2.26","doi-asserted-by":"crossref","unstructured":"Friedel, E. et al. Devaluation and sequential decisions: linking goal-directed and model-based behavior. Front. Hum. Neurosci. 8, (2014).","DOI":"10.3389\/fnhum.2014.00587"},{"key":"2021092009150729000_2020.09.06.20189241v2.27","doi-asserted-by":"publisher","DOI":"10.1159\/000362840"},{"key":"2021092009150729000_2020.09.06.20189241v2.28","doi-asserted-by":"publisher","DOI":"10.1038\/mp.2014.44"},{"key":"2021092009150729000_2020.09.06.20189241v2.29","doi-asserted-by":"crossref","first-page":"e670","DOI":"10.1038\/tp.2015.165","article-title":"Motivation and value influences in the relative balance of goal-directed and habitual behaviours in obsessive-compulsive disorder","volume":"5","year":"2015","journal-title":"Transl. Psychiatry"},{"key":"2021092009150729000_2020.09.06.20189241v2.30","doi-asserted-by":"crossref","unstructured":"Gillan, C. M. , Kosinski, M. , Whelan, R. , Phelps, E. A. & Daw, N. D. Characterizing a psychiatric symptom dimension related to deficits in goaldirected control. Elife 5, (2016).","DOI":"10.7554\/eLife.11305"},{"key":"2021092009150729000_2020.09.06.20189241v2.31","doi-asserted-by":"publisher","DOI":"10.1037\/abn0000164"},{"key":"2021092009150729000_2020.09.06.20189241v2.32","doi-asserted-by":"crossref","unstructured":"da Silva, C. F. & Hare, T. Humans primarily use model-based inference in the two-stage task. Nat. Hum. Behav. 1\u201314 (2020).","DOI":"10.1101\/682922"},{"key":"2021092009150729000_2020.09.06.20189241v2.33","first-page":"243","article-title":"Some Effects of Instructions on Human Operant Behavior","volume":"1","year":"1966","journal-title":"Psychon. Monogr. Suppl"},{"key":"2021092009150729000_2020.09.06.20189241v2.34","doi-asserted-by":"publisher","DOI":"10.1901\/jeab.1969.12-701"},{"key":"2021092009150729000_2020.09.06.20189241v2.35","unstructured":"Baron, A. & Galizio, M. Instructional control of human operant behavior. Psychol. Rec. (1983)."},{"key":"2021092009150729000_2020.09.06.20189241v2.36","doi-asserted-by":"publisher","DOI":"10.1037\/h0025540"},{"key":"2021092009150729000_2020.09.06.20189241v2.37","doi-asserted-by":"publisher","DOI":"10.7554\/elife.15192"},{"key":"2021092009150729000_2020.09.06.20189241v2.38","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainres.2009.07.007"},{"key":"2021092009150729000_2020.09.06.20189241v2.39","doi-asserted-by":"publisher","DOI":"10.1111\/j.1551-6709.2009.01010.x"},{"key":"2021092009150729000_2020.09.06.20189241v2.40","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1014938108"},{"key":"2021092009150729000_2020.09.06.20189241v2.41","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2009.09.004"},{"key":"2021092009150729000_2020.09.06.20189241v2.42","doi-asserted-by":"crossref","unstructured":"Akam, T. , Costa, R. & Dayan, P. Simple Plans or Sophisticated Habits? State, Transition and Learning Interactions in the Two-Step Task. PLoS Comput. Biol. 11, (2015).","DOI":"10.1101\/021428"},{"key":"2021092009150729000_2020.09.06.20189241v2.43","doi-asserted-by":"crossref","unstructured":"Kool, W. , Cushman, F. A. & Gershman, S. J. When Does Model-Based Control Pay Off? PLoS Comput. Biol. 12, (2016).","DOI":"10.1371\/journal.pcbi.1005090"},{"key":"2021092009150729000_2020.09.06.20189241v2.44","doi-asserted-by":"publisher","DOI":"10.1016\/S0028-3908(98)00033-1"},{"key":"2021092009150729000_2020.09.06.20189241v2.45","doi-asserted-by":"publisher","DOI":"10.1038\/s41583-018-0002-7"},{"key":"2021092009150729000_2020.09.06.20189241v2.46","first-page":"1","article-title":"Animal intelligence: An experimental study of the associative processes in animals","volume":"2","year":"1898","journal-title":"Psychol. Rev"},{"key":"2021092009150729000_2020.09.06.20189241v2.47","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1001089"},{"key":"2021092009150729000_2020.09.06.20189241v2.48","doi-asserted-by":"publisher","DOI":"10.1001\/jamapsychiatry.2019.2998"},{"key":"2021092009150729000_2020.09.06.20189241v2.49","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.2200"},{"key":"2021092009150729000_2020.09.06.20189241v2.50","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(00)00113-6"},{"key":"2021092009150729000_2020.09.06.20189241v2.51","doi-asserted-by":"publisher","DOI":"10.1176\/appi.ajp.2011.10071062"},{"key":"2021092009150729000_2020.09.06.20189241v2.52","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2018.12.079"},{"key":"2021092009150729000_2020.09.06.20189241v2.53","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1821647116"},{"key":"2021092009150729000_2020.09.06.20189241v2.54","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2007.01.004"},{"key":"2021092009150729000_2020.09.06.20189241v2.55","doi-asserted-by":"publisher","DOI":"10.1101\/126292"},{"key":"2021092009150729000_2020.09.06.20189241v2.56","doi-asserted-by":"crossref","unstructured":"Konovalov, Arkady ; Krajbich, I. Mouse tracking reveals structure knowledge in the absence of model-based choice. Nat. Commun. 11, (2020).","DOI":"10.1038\/s41467-020-15696-w"},{"key":"2021092009150729000_2020.09.06.20189241v2.57","doi-asserted-by":"publisher","DOI":"10.1038\/s41583-019-0220-7"},{"key":"2021092009150729000_2020.09.06.20189241v2.58","doi-asserted-by":"publisher","DOI":"10.1001\/archpsyc.1987.01800150017003"},{"key":"2021092009150729000_2020.09.06.20189241v2.59","doi-asserted-by":"publisher","DOI":"10.1016\/j.neubiorev.2007.09.005"},{"key":"2021092009150729000_2020.09.06.20189241v2.60","doi-asserted-by":"publisher","DOI":"10.1126\/science.1154433"},{"key":"2021092009150729000_2020.09.06.20189241v2.61","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2016.08.019"},{"key":"2021092009150729000_2020.09.06.20189241v2.62","doi-asserted-by":"publisher","DOI":"10.1016\/S0924-9338(97)83297-X"},{"key":"2021092009150729000_2020.09.06.20189241v2.63","unstructured":"First, M. B. , Spitzer, R. L. , Gibbon, M. & Williams, J. B. W. Structured Clinical Interview for DSM-IV Axis I Disorders. New York State Psychiatric Institute (2002)."},{"key":"2021092009150729000_2020.09.06.20189241v2.64","doi-asserted-by":"publisher","DOI":"10.1001\/archpsyc.1989.01810110048007"},{"key":"2021092009150729000_2020.09.06.20189241v2.65","doi-asserted-by":"publisher","DOI":"10.1037\/a0018492"},{"key":"2021092009150729000_2020.09.06.20189241v2.66","doi-asserted-by":"crossref","unstructured":"Spielberger, C. Manual for the State-Trait Anxiety Inventory (STAI). Consult. Psychol. Press 4\u201326 (1983).","DOI":"10.1037\/t06496-000"},{"key":"2021092009150729000_2020.09.06.20189241v2.67","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyt.2018.00397"},{"key":"2021092009150729000_2020.09.06.20189241v2.68","doi-asserted-by":"crossref","unstructured":"Beck, A. T. , Steer, R. A. & Brown, G. K. Manual for the Beck depression inventory-II. San Antonio, TX Psychol. Corp. 1\u201382 (1996).","DOI":"10.1037\/t00742-000"},{"key":"2021092009150729000_2020.09.06.20189241v2.69","doi-asserted-by":"publisher","DOI":"10.1006\/brcg.1998.1039"},{"key":"2021092009150729000_2020.09.06.20189241v2.70","doi-asserted-by":"publisher","DOI":"10.1016\/j.jneumeth.2013.10.024"},{"key":"2021092009150729000_2020.09.06.20189241v2.71","doi-asserted-by":"publisher","DOI":"10.1016\/0005-7967(94)00075-U"},{"key":"2021092009150729000_2020.09.06.20189241v2.72","doi-asserted-by":"crossref","unstructured":"Huys, Q. J. M. et al. Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput. Biol. 7, (2011).","DOI":"10.1371\/journal.pcbi.1002028"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1101\/2020.09.06.20189241","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T13:53:34Z","timestamp":1768485214000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.1101\/2020.09.06.20189241"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,8]]},"references-count":72,"URL":"https:\/\/doi.org\/10.1101\/2020.09.06.20189241","relation":{"is-preprint-of":[{"id-type":"doi","id":"10.1038\/s41562-022-01346-2","asserted-by":"subject"}]},"subject":[],"published":{"date-parts":[[2020,9,8]]},"subtype":"preprint"}}