{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T17:47:43Z","timestamp":1778089663504,"version":"3.51.4"},"reference-count":33,"publisher":"MIT Press","issue":"1","funder":[{"name":"Hefei Institutes of Physical Science (HFIPS) Director's Fund","award":["YZJJ202207-TS"],"award-info":[{"award-number":["YZJJ202207-TS"]}]},{"name":"Natural Science Fund of China","award":["82371931"],"award-info":[{"award-number":["82371931"]}]},{"DOI":"10.13039\/501100003995","name":"Anhui Provincial Natural Science Foundation","doi-asserted-by":"crossref","award":["2408085MC056"],"award-info":[{"award-number":["2408085MC056"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,1,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Humans use both model-free (or habitual) and model-based (or goal-directed) strategies in sequential decision-making. Working memory (WM) is essential for the model-based strategy; however, its exact role in these processes remains elusive. This study investigates the influence of WM processes on decision-making and the underlying cognitive computing mechanisms. Specifically, we used experimental data from two-stage decision tasks and found that delay and load, two WM-specific variables, impact goal-revisiting behaviors. Then, we proposed possible computational mechanisms by which WM participates in information processing and integrated them into the model-based system. The proposed Hybrid-WM model reproduced the observed experimental effects and fit human behavior better than the classic hybrid reinforcement learning model. These results were verified with independent data sets. Furthermore, differences in model parameters explain the age-related difference in sequential decision-making. Overall, this study suggests that WM guides action valuation in model-based strategies, highlighting the contribution of higher cognitive functions to sequential decision-making.<\/jats:p>","DOI":"10.1162\/jocn_a_02237","type":"journal-article","created":{"date-parts":[[2024,8,13]],"date-time":"2024-08-13T13:44:01Z","timestamp":1723556641000},"page":"86-96","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":5,"title":["Working Memory Guides Action Valuation in Model-based Decision-making Strategy"],"prefix":"10.1162","volume":"37","author":[{"given":"Zhaoyu","family":"Zuo","sequence":"first","affiliation":[{"name":"Anhui Province Key Laboratory of Medical Physics and Technology, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences"},{"name":"University of Science and Technology of China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li-Zhuang","family":"Yang","sequence":"additional","affiliation":[{"name":"Anhui Province Key Laboratory of Medical Physics and Technology, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences"},{"name":"University of Science and Technology of China"},{"name":"Hefei Cancer Hospital, Chinese Academy of Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongzhi","family":"Wang","sequence":"additional","affiliation":[{"name":"Anhui Province Key Laboratory of Medical Physics and Technology, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences"},{"name":"University of Science and Technology of China"},{"name":"Hefei Cancer Hospital, Chinese Academy of Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hai","family":"Li","sequence":"additional","affiliation":[{"name":"Anhui Province Key Laboratory of Medical Physics and Technology, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences"},{"name":"University of Science and Technology of China"},{"name":"Hefei Cancer Hospital, Chinese Academy of Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2025,1,2]]},"reference":[{"key":"2024121915544758500_bib1","doi-asserted-by":"publisher","first-page":"e49154","DOI":"10.7554\/eLife.49154","article-title":"Metacontrol of decision-making strategies in human aging","volume":"8","author":"Bolenz","year":"2019","journal-title":"eLife"},{"key":"2024121915544758500_bib2","doi-asserted-by":"publisher","first-page":"1126","DOI":"10.1038\/s41562-022-01346-2","article-title":"Explicit knowledge of task structure is a primary determinant of human model-based action","volume":"6","author":"Castro-Rodrigues","year":"2022","journal-title":"Nature Human Behaviour"},{"key":"2024121915544758500_bib3","doi-asserted-by":"publisher","first-page":"13747","DOI":"10.1523\/JNEUROSCI.0989-14.2014","article-title":"Working memory contributions to reinforcement learning impairments in schizophrenia","volume":"34","author":"Collins","year":"2014","journal-title":"Journal of Neuroscience"},{"key":"2024121915544758500_bib4","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1038\/s41583-020-0355-6","article-title":"Beyond dichotomies in reinforcement learning","volume":"21","author":"Collins","year":"2020","journal-title":"Nature Reviews Neuroscience"},{"key":"2024121915544758500_bib5","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1111\/j.1460-9568.2011.07980.x","article-title":"How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis","volume":"35","author":"Collins","year":"2012","journal-title":"European Journal of Neuroscience"},{"key":"2024121915544758500_bib6","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1037\/abn0000164","article-title":"Reduced model-based decision-making in schizophrenia","volume":"125","author":"Culbreth","year":"2016","journal-title":"Journal of Abnormal Psychology"},{"key":"2024121915544758500_bib7","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1038\/s41562-020-0905-y","article-title":"Humans primarily use model-based inference in the two-stage task","volume":"4","author":"da Silva","year":"2020","journal-title":"Nature Human Behaviour"},{"key":"2024121915544758500_bib8","doi-asserted-by":"publisher","first-page":"1497","DOI":"10.1038\/s41593-018-0258-2","article-title":"Are we of two minds?","volume":"21","author":"Daw","year":"2018","journal-title":"Nature Neuroscience"},{"key":"2024121915544758500_bib9","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans' choices and striatal prediction errors","volume":"69","author":"Daw","year":"2011","journal-title":"Neuron"},{"key":"2024121915544758500_bib10","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nature Neuroscience"},{"key":"2024121915544758500_bib11","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1146\/annurev-psych-010814-015031","article-title":"The cognitive neuroscience of working memory","volume":"66","author":"D'Esposito","year":"2015","journal-title":"Annual Review of Psychology"},{"key":"2024121915544758500_bib12","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.ics.2004.11.056","article-title":"An assessment of working memory using P300 wave in healthy subjects","volume":"1278","author":"Dolu","year":"2005","journal-title":"International Congress Series"},{"key":"2024121915544758500_bib13","doi-asserted-by":"publisher","first-page":"17456916231204811","DOI":"10.1177\/17456916231204811","article-title":"Diminished state space theory of human aging","author":"Eppinger","year":"2023","journal-title":"Perspectives on Psychological Science"},{"key":"2024121915544758500_bib14","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3389\/fnins.2013.00253","article-title":"Of goals and habits: Age-related and individual differences in goal-directed decision-making","author":"Eppinger","year":"2013","journal-title":"Frontiers in Neuroscience"},{"key":"2024121915544758500_bib15","doi-asserted-by":"publisher","first-page":"406","DOI":"10.3758\/s13415-016-0487-3","article-title":"Electrophysiological correlates reflect the integration of model-based and model-free decision information","volume":"17","author":"Eppinger","year":"2017","journal-title":"Cognitive, Affective, & Behavioral Neuroscience"},{"key":"2024121915544758500_bib16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jmp.2016.01.006","article-title":"Empirical priors for reinforcement learning models","volume":"71","author":"Gershman","year":"2016","journal-title":"Journal of Mathematical Psychology"},{"key":"2024121915544758500_bib17","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1146\/annurev-psych-122414-033625","article-title":"Reinforcement learning and episodic memory in humans and animals: An integrative framework","volume":"68","author":"Gershman","year":"2017","journal-title":"Annual Review of Psychology"},{"key":"2024121915544758500_bib18","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: Dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"Gl\u00e4scher","year":"2010","journal-title":"Neuron"},{"key":"2024121915544758500_bib19","doi-asserted-by":"publisher","first-page":"1893","DOI":"10.1038\/s41467-020-15696-w","article-title":"Mouse tracking reveals structure knowledge in the absence of model-based choice","volume":"11","author":"Konovalov","year":"2020","journal-title":"Nature Communications"},{"key":"2024121915544758500_bib20","doi-asserted-by":"publisher","first-page":"e1005090","DOI":"10.1371\/journal.pcbi.1005090","article-title":"When does model-based control pay off?","volume":"12","author":"Kool","year":"2016","journal-title":"PLoS Computational Biology"},{"key":"2024121915544758500_bib21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.conb.2017.10.006","article-title":"Model-based predictions for dopamine","volume":"49","author":"Langdon","year":"2018","journal-title":"Current Opinion in Neurobiology"},{"key":"2024121915544758500_bib22","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1038\/36846","article-title":"The capacity of visual working memory for features and conjunctions","volume":"390","author":"Luck","year":"1997","journal-title":"Nature"},{"key":"2024121915544758500_bib23","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1038\/nn.3655","article-title":"Changing concepts of working memory","volume":"17","author":"Ma","year":"2014","journal-title":"Nature Neuroscience"},{"key":"2024121915544758500_bib24","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1177\/0956797612463080","article-title":"The curse of planning: Dissecting multiple reinforcement-learning systems by taxing the central executive","volume":"24","author":"Otto","year":"2013","journal-title":"Psychological Science"},{"key":"2024121915544758500_bib25","doi-asserted-by":"publisher","first-page":"e85243","DOI":"10.7554\/eLife.85243","article-title":"Age-related differences in prefrontal glutamate are associated with increased working memory decay that gives the appearance of learning deficits","volume":"12","author":"Rmus","year":"2023","journal-title":"eLife"},{"key":"2024121915544758500_bib26","volume-title":"On-line Q-learning using connectionist systems","author":"Rummery","year":"1994"},{"key":"2024121915544758500_bib27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3564240","article-title":"Explainable goal-driven agents and robots\u2014A comprehensive review","volume":"55","author":"Sado","year":"2023","journal-title":"ACM Computing Surveys"},{"key":"2024121915544758500_bib28","doi-asserted-by":"publisher","first-page":"e1006803","DOI":"10.1371\/journal.pcbi.1006803","article-title":"Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling","volume":"15","year":"2019","journal-title":"PLoS Computational Biology"},{"key":"2024121915544758500_bib29","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1109\/THMS.2021.3064815","article-title":"Investigating the P300 response as a marker of working memory in virtual training environments","volume":"51","author":"Simpson","year":"2021","journal-title":"IEEE Transactions on Human-Machine Systems"},{"key":"2024121915544758500_bib30","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1038\/mp.2014.44","article-title":"Disorders of compulsivity: A common bias towards learning habits","volume":"20","author":"Voon","year":"2015","journal-title":"Molecular Psychiatry"},{"key":"2024121915544758500_bib31","doi-asserted-by":"publisher","first-page":"466","DOI":"10.1037\/a0033455","article-title":"Navigating complex decision spaces: Problems and paradigms in sequential choice","volume":"140","author":"Walsh","year":"2014","journal-title":"Psychological Bulletin"},{"key":"2024121915544758500_bib32","doi-asserted-by":"publisher","first-page":"e49547","DOI":"10.7554\/eLife.49547","article-title":"Ten simple rules for the computational modeling of behavioral data","volume":"8","author":"Wilson","year":"2019","journal-title":"eLife"},{"key":"2024121915544758500_bib33","doi-asserted-by":"publisher","first-page":"1070","DOI":"10.3758\/s13415-020-00820-6","article-title":"The influence of internal models on feedback-related brain activity","volume":"20","author":"Wurm","year":"2020","journal-title":"Cognitive, Affective, & Behavioral Neuroscience"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/37\/1\/86\/2481742\/jocn_a_02237.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/37\/1\/86\/2481742\/jocn_a_02237.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T15:55:02Z","timestamp":1734623702000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/37\/1\/86\/123973\/Working-Memory-Guides-Action-Valuation-in-Model"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025]]},"references-count":33,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1,2]]},"published-print":{"date-parts":[[2025,1,2]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_02237","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025]]},"published":{"date-parts":[[2025]]}}}