{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T23:50:59Z","timestamp":1792799459300,"version":"4.5.0"},"reference-count":194,"publisher":"MIT Press","issue":"4","funder":[{"name":"National Science Foundation","award":["NSF 2020844"],"award-info":[{"award-number":["NSF 2020844"]}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,3,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Reinforcement learning and working memory are two core processes of human cognition and are often considered cognitively, neuroscientifically, and algorithmically distinct. Here, we show that the brain networks that support them actually overlap significantly and that they are less distinct cognitive processes than often assumed. We review literature demonstrating the benefits of considering each process to explain properties of the other and highlight recent work investigating their more complex interactions. We discuss how future research in both computational and cognitive sciences can benefit from one another, suggesting that a key missing piece for artificial agents to learn to behave with more human-like efficiency is taking working memory's role in learning seriously. This review highlights the risks of neglecting the interplay between different processes when studying human behavior (in particular when considering individual differences). We emphasize the importance of investigating these dynamics to build a comprehensive understanding of human cognition.<\/jats:p>","DOI":"10.1162\/jocn_a_01808","type":"journal-article","created":{"date-parts":[[2021,12,23]],"date-time":"2021-12-23T17:55:50Z","timestamp":1640282150000},"page":"551-568","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":74,"title":["How Working Memory and Reinforcement Learning Are Intertwined: A Cognitive, Neural, and Computational Perspective"],"prefix":"10.1162","volume":"34","author":[{"given":"Aspen H.","family":"Yoo","sequence":"first","affiliation":[{"name":"University of California, Berkeley"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3751-3662","authenticated-orcid":true,"given":"Anne G. E.","family":"Collins","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2022,3,5]]},"reference":[{"key":"2022030421205791800_bib200","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1146\/annurev.ne.09.030186.002041","article-title":"Parallel organization of functionally segregated circuits linking basal ganglia and cortex","volume":"9","author":"Alexander","year":"1986","journal-title":"Annual Reviews of Neuroscience"},{"key":"2022030421205791800_bib1","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780195324259.001.0001","volume-title":"How can the human mind occur in the physical universe?","author":"Anderson","year":"2007"},{"key":"2022030421205791800_bib2","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/S0079-7421(08)60452-1","article-title":"Working memory","volume":"8","author":"Baddeley","year":"1974","journal-title":"Psychology of Learning and Motivation"},{"key":"2022030421205791800_bib3","doi-asserted-by":"publisher","first-page":"560080","DOI":"10.3389\/fpsyg.2020.560080","article-title":"The missing link between memory and reinforcement learning","volume":"11","author":"Balkenius","year":"2020","journal-title":"Frontiers in Psychology"},{"key":"2022030421205791800_bib4","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.tics.2011.11.015","article-title":"Cognition in schizophrenia: Core psychological and neural mechanisms","volume":"16","author":"Barch","year":"2012","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib5","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1016\/j.neuron.2005.05.020","article-title":"Midbrain dopamine neurons encode a quantitative reward prediction error signal","volume":"47","author":"Bayer","year":"2005","journal-title":"Neuron"},{"key":"2022030421205791800_bib6","doi-asserted-by":"publisher","first-page":"812","DOI":"10.1080\/13825585.2020.1828804","article-title":"Dopamine effects on memory load and distraction during visuospatial working memory in cognitively normal Parkinson's disease","volume":"28","author":"Bayram","year":"2021","journal-title":"Aging, Neuropsychology, and Cognition"},{"key":"2022030421205791800_bib7","doi-asserted-by":"publisher","first-page":"3632","DOI":"10.1523\/JNEUROSCI.3204-13.2014","article-title":"Noise in neural populations accounts for errors in working memory","volume":"34","author":"Bays","year":"2014","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib8","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1126\/science.1158023","article-title":"Dynamic shifts of limited working memory resources in human vision","volume":"321","author":"Bays","year":"2008","journal-title":"Science"},{"key":"2022030421205791800_bib9","doi-asserted-by":"publisher","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"Behrens","year":"2007","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib10","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1038\/s41586-020-2939-8","article-title":"Autonomous navigation of stratospheric balloons using reinforcement learning","volume":"588","author":"Bellemare","year":"2020","journal-title":"Nature"},{"key":"2022030421205791800_bib11","doi-asserted-by":"publisher","first-page":"4648","DOI":"10.1038\/s41467-021-24907-x","article-title":"Memory and decision making interact to shape the value of unchosen options","volume":"12","author":"Biderman","year":"2021","journal-title":"Nature Communications"},{"key":"2022030421205791800_bib12","doi-asserted-by":"publisher","first-page":"997","DOI":"10.1038\/nn.4573","article-title":"Reinstated episodic context guides sampling-based decisions for reward","volume":"20","author":"Bornstein","year":"2017","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib13","doi-asserted-by":"publisher","first-page":"408","DOI":"10.1016\/j.tics.2019.02.006","article-title":"Reinforcement learning, fast and slow","volume":"23","author":"Botvinick","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib14","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1177\/0956797610397956","article-title":"Hierarchical encoding in visual working memory: Ensemble statistics bias memory for individual items","volume":"22","author":"Brady","year":"2011","journal-title":"Psychological Science"},{"key":"2022030421205791800_bib15","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1037\/a0016797","article-title":"Compression in visual working memory: Using statistical regularities to form more efficient memory representations","volume":"138","author":"Brady","year":"2009","journal-title":"Journal of Experimental Psychology: General"},{"key":"2022030421205791800_bib16","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1037\/a0030779","article-title":"A probabilistic model of visual working memory: Incorporating higher order regularities into working memory capacity estimates","volume":"120","author":"Brady","year":"2013","journal-title":"Psychological Review"},{"key":"2022030421205791800_bib17","doi-asserted-by":"publisher","first-page":"551","DOI":"10.7551\/mitpress\/1481.003.0044","article-title":"On the control of control: The role of dopamine in regulating prefrontal function and working","volume-title":"Making working memory work","author":"Braver","year":"2000"},{"key":"2022030421205791800_bib18","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1016\/s0896-6273(03)00466-5","article-title":"Neural mechanisms of transient and sustained cognitive control during task switching","volume":"39","author":"Braver","year":"2003","journal-title":"Neuron"},{"key":"2022030421205791800_bib19","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1080\/17470215808416249","article-title":"Some tests of the decay theory of immediate memory","volume":"10","author":"Brown","year":"1958","journal-title":"Quarterly Journal of Experimental Psychology"},{"key":"2022030421205791800_bib20","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/0166-2236(90)90058-i","article-title":"Cognitive function in Parkinson's disease: From description to theory","volume":"13","author":"Brown","year":"1990","journal-title":"Trends in Neurosciences"},{"key":"2022030421205791800_bib21","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1016\/s0959-4388(97)80003-7","article-title":"Sensory and cognitive functions of the basal ganglia","volume":"7","author":"Brown","year":"1997","journal-title":"Current Opinion in Neurobiology"},{"key":"2022030421205791800_bib22","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1080\/87565640801982312","article-title":"Short-term memory, working memory, and executive functioning in preschoolers: Longitudinal predictors of mathematical achievement at age 7 years","volume":"33","author":"Bull","year":"2008","journal-title":"Developmental Neuropsychology"},{"key":"2022030421205791800_bib23","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.cobeha.2014.08.001","article-title":"Multiple gates on working memory","volume":"1","author":"Chatham","year":"2015","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"2022030421205791800_bib24","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1016\/j.neuron.2014.01.002","article-title":"Corticostriatal output gating during selection from working memory","volume":"81","author":"Chatham","year":"2014","journal-title":"Neuron"},{"key":"2022030421205791800_bib25","doi-asserted-by":"publisher","first-page":"12983","DOI":"10.1523\/JNEUROSCI.0184-12.2012","article-title":"Decoding the contents of visual short-term memory from human visual and parietal cortex","volume":"32","author":"Christophel","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib26","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1016\/j.tics.2016.12.007","article-title":"The distributed nature of working memory","volume":"21","author":"Christophel","year":"2017","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib27","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1146\/annurev.psych.093008.100427","article-title":"A taxonomy of external and internal attention","volume":"62","author":"Chun","year":"2011","journal-title":"Annual Review of Psychology"},{"key":"2022030421205791800_bib28","doi-asserted-by":"publisher","first-page":"9121","DOI":"10.1038\/s41598-018-27378-1","article-title":"The relationship between reinforcement and explicit control during visuomotor adaptation","volume":"8","author":"Codol","year":"2018","journal-title":"Scientific Reports"},{"key":"2022030421205791800_bib29","doi-asserted-by":"publisher","first-page":"1422","DOI":"10.1162\/jocn_a_01238","article-title":"The tortoise and the hare: Interactions between reinforcement learning and working memory","volume":"30","author":"Collins","year":"2018","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib30","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1016\/j.biopsych.2017.05.017","article-title":"Interactions among working memory, reinforcement learning, and effort in value-based choice: A new paradigm and selective deficits in schizophrenia","volume":"82","author":"Collins","year":"2017","journal-title":"Biological Psychiatry"},{"key":"2022030421205791800_bib31","doi-asserted-by":"publisher","first-page":"13747","DOI":"10.1523\/JNEUROSCI.0989-14.2014","article-title":"Working memory contributions to reinforcement learning impairments in schizophrenia","volume":"34","author":"Collins","year":"2014","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib32","doi-asserted-by":"publisher","first-page":"4332","DOI":"10.1523\/JNEUROSCI.2700-16.2017","article-title":"Working memory load strengthens reward prediction errors","volume":"37","author":"Collins","year":"2017","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib33","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1038\/s41583-020-0355-6","article-title":"Beyond simple dichotomies in reinforcement learning","volume":"21","author":"Collins","year":"2020","journal-title":"Nature Reviews Neuroscience"},{"key":"2022030421205791800_bib34","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1111\/j.1460-9568.2011.07980.x","article-title":"How much of reinforcement learning is working memory, not reinforcement learning? A behavioral, computational, and neurogenetic analysis: Working memory in reinforcement learning","volume":"35","author":"Collins","year":"2012","journal-title":"European Journal of Neuroscience"},{"key":"2022030421205791800_bib35","doi-asserted-by":"publisher","first-page":"2502","DOI":"10.1073\/pnas.1720963115","article-title":"Within- and across-trial dynamics of human EEG reveal cooperative interplay between reinforcement learning and working memory","volume":"115","author":"Collins","year":"2018","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib36","doi-asserted-by":"publisher","first-page":"910","DOI":"10.1093\/cercor\/10.9.910","article-title":"Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model","volume":"10","author":"Compte","year":"2000","journal-title":"Cerebral Cortex"},{"key":"2022030421205791800_bib37","doi-asserted-by":"publisher","first-page":"7020","DOI":"10.1523\/JNEUROSCI.2486-17.2018","article-title":"Persistent spiking activity underlies working memory","volume":"38","author":"Constantinidis","year":"2018","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib38","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1016\/j.tics.2003.10.005","article-title":"Working memory capacity and its relation to general intelligence","volume":"7","author":"Conway","year":"2003","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib39","doi-asserted-by":"publisher","first-page":"e113","DOI":"10.1016\/j.biopsych.2011.03.028","article-title":"Inverted-U\u2013Shaped dopamine actions on human working memory and cognitive control","volume":"69","author":"Cools","year":"2011","journal-title":"Biological Psychiatry"},{"key":"2022030421205791800_bib40","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1016\/j.tics.2006.05.004","article-title":"Bayesian theories of conditioning in a changing world","volume":"10","author":"Courville","year":"2006","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib41","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.3758\/s13423-016-1191-6","article-title":"The many faces of working memory and short-term storage","volume":"24","author":"Cowan","year":"2017","journal-title":"Psychonomic Bulletin & Review"},{"key":"2022030421205791800_bib42","doi-asserted-by":"publisher","DOI":"10.1101\/107698","article-title":"Can model-free reinforcement learning operate over information stored in working-memory?","author":"da Silva","year":"2017","journal-title":"BioRxiv"},{"key":"2022030421205791800_bib43","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1016\/S0022-5371(80)90312-6","article-title":"Individual differences in working memory and reading","volume":"19","author":"Daneman","year":"1980","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"2022030421205791800_bib44","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans' choices and striatal prediction errors","volume":"69","author":"Daw","year":"2011","journal-title":"Neuron"},{"key":"2022030421205791800_bib45","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1016\/B978-0-12-416008-8.00021-8","article-title":"Multiple systems for value learning","volume-title":"Neuroeconomics","author":"Daw","year":"2014","edition":"2nd ed."},{"key":"2022030421205791800_bib46","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib47","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1016\/B978-0-12-416008-8.00015-2","article-title":"Value learning through reinforcement: The basics of dopamine and reinforcement learning","volume-title":"Neuroeconomics","author":"Daw","year":"2014","edition":"2nd ed."},{"key":"2022030421205791800_bib48","article-title":"IQ and working memory predict plan-based sequential action learning","author":"de Kleijn","year":"2018","journal-title":"Proceedings of the Annual Conference of the Cognitive Science Society"},{"key":"2022030421205791800_bib49","doi-asserted-by":"publisher","first-page":"172","DOI":"10.3389\/fpsyt.2013.00172","article-title":"Reinforcement learning and dopamine in schizophrenia: Dimensions of symptoms or specific features of a disease group?","volume":"4","author":"Deserno","year":"2013","journal-title":"Frontiers in Psychiatry"},{"key":"2022030421205791800_bib50","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/ktrsj","article-title":"Influences of both prior knowledge and recent history on visual working memory","volume-title":"Proceedings of the Annual Conference of the Cognitive Science Society","author":"Destefano","year":"2020"},{"key":"2022030421205791800_bib51","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1167\/17.11.4","article-title":"Monkeys and humans take local uncertainty into account when localizing a change","volume":"17","author":"Devkar","year":"2017","journal-title":"Journal of Vision"},{"key":"2022030421205791800_bib52","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.neuron.2013.09.007","article-title":"Goals and habits in the brain","volume":"80","author":"Dolan","year":"2013","journal-title":"Neuron"},{"key":"2022030421205791800_bib53","doi-asserted-by":"publisher","first-page":"1211","DOI":"10.1523\/JNEUROSCI.1901-15.2016","article-title":"Variability in dopamine genes dissociates model-based and model-free reinforcement learning","volume":"36","author":"Doll","year":"2016","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib54","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/s0926-6410(99)00029-4","article-title":"Prefrontal cortex activation in task switching: An event-related fMRI study","volume":"9","author":"Dove","year":"2000","journal-title":"Cognitive Brain Research"},{"key":"2022030421205791800_bib55","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1111\/1467-9280.00290","article-title":"Interactions between visual working memory and selective attention","volume":"11","author":"Downing","year":"2000","journal-title":"Psychological Science"},{"key":"2022030421205791800_bib56","article-title":"One-shot imitation learning","author":"Duan","year":"2017","journal-title":"ArXiv"},{"key":"2022030421205791800_bib57","article-title":"RL2: Fast reinforcement learning via slow reinforcement learning","author":"Duan","year":"2016","journal-title":"ArXiv"},{"key":"2022030421205791800_bib58","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/j.cobeha.2021.06.004","article-title":"What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience","volume":"41","author":"Eckstein","year":"2021","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"2022030421205791800_bib59","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1037\/xhp0000398","article-title":"Attention mediates the flexible allocation of visual working memory resources","volume":"43","author":"Emrich","year":"2017","journal-title":"Journal of Experimental Psychology. Human Perception and Performance"},{"key":"2022030421205791800_bib60","doi-asserted-by":"publisher","first-page":"2812","DOI":"10.1162\/jocn_a_00676","article-title":"Reward acts on the pFC to enhance distractor resistance of working memory representations","volume":"26","author":"Fallon","year":"2014","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib61","doi-asserted-by":"publisher","first-page":"5727","DOI":"10.1093\/cercor\/bhx242","article-title":"Fractionating the neurocognitive mechanisms underlying working memory: Independent effects of dopamine and Parkinson's disease","volume":"27","author":"Fallon","year":"2017","journal-title":"Cerebral Cortex"},{"key":"2022030421205791800_bib62","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.neuropsychologia.2015.07.031","article-title":"Differential optimal dopamine levels for set-shifting and working memory in Parkinson's disease","volume":"77","author":"Fallon","year":"2015","journal-title":"Neuropsychologia"},{"key":"2022030421205791800_bib63","doi-asserted-by":"publisher","first-page":"728","DOI":"10.1162\/jocn_a_01073","article-title":"Dopamine alters the fidelity of working memory representations according to attentional demands","volume":"29","author":"Fallon","year":"2017","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib64","doi-asserted-by":"publisher","first-page":"e0214146","DOI":"10.1371\/journal.pone.0214146","article-title":"More than an \u201cinverted-U\u201d? An exploratory study of the association between the catechol-o-methyltransferase gene polymorphism and executive functions in Parkinson's disease","volume":"14","author":"Fang","year":"2019","journal-title":"PLoS One"},{"key":"2022030421205791800_bib65","doi-asserted-by":"publisher","first-page":"1768","DOI":"10.1038\/s41467-017-01874-w","article-title":"Feature-based learning improves adaptability without compromising precision","volume":"8","author":"Farashahi","year":"2017","journal-title":"Nature Communications"},{"key":"2022030421205791800_bib66","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1038\/nature04587","article-title":"Reverse replay of behavioural sequences in hippocampal place cells during the awake state","volume":"440","author":"Foster","year":"2006","journal-title":"Nature"},{"key":"2022030421205791800_bib67","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1038\/ncomms2237","article-title":"Variability in the quality of visual working memory","volume":"3","author":"Fougnie","year":"2012","journal-title":"Nature Communications"},{"key":"2022030421205791800_bib68","doi-asserted-by":"publisher","first-page":"137","DOI":"10.3758\/CABN.1.2.137","article-title":"Interactions between frontal cortex and basal ganglia in working memory: A computational model","volume":"1","author":"Frank","year":"2001","journal-title":"Cognitive, Affective, & Behavioral Neuroscience"},{"key":"2022030421205791800_bib69","doi-asserted-by":"publisher","first-page":"16311","DOI":"10.1073\/pnas.0706111104","article-title":"Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning","volume":"104","author":"Frank","year":"2007","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib70","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1152\/jn.1989.61.2.331","article-title":"Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex","volume":"61","author":"Funahashi","year":"1989","journal-title":"Journal of Neurophysiology"},{"key":"2022030421205791800_bib71","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1162\/jocn_a_01518","article-title":"Effects of dopaminergic drugs on cognitive control processes vary by genotype","volume":"32","author":"Furman","year":"2020","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib72","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1126\/science.173.3997.652","article-title":"Neuron activity related to short-term memory","volume":"173","author":"Fuster","year":"1971","journal-title":"Science"},{"key":"2022030421205791800_bib73","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1146\/annurev-psych-122414-033625","article-title":"Reinforcement learning and episodic memory in humans and animals: An integrative framework","volume":"68","author":"Gershman","year":"2017","journal-title":"Annual Review of Psychology"},{"key":"2022030421205791800_bib74","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1093\/schbul\/sbn068","article-title":"Reward processing in schizophrenia: A deficit in the representation of value","volume":"34","author":"Gold","year":"2008","journal-title":"Schizophrenia Bulletin"},{"key":"2022030421205791800_bib75","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1037\/0021-843X.112.1.61","article-title":"Working memory for visual features and conjunctions in schizophrenia","volume":"112","author":"Gold","year":"2003","journal-title":"Journal of Abnormal Psychology"},{"key":"2022030421205791800_bib76","article-title":"Neural turing machines","author":"Graves","year":"2014","journal-title":"ArXiv"},{"key":"2022030421205791800_bib77","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1515\/ppb-2016-0013","article-title":"The role of executive processes in working memory deficits in Parkinson's disease","volume":"47","author":"Gruszka","year":"2016","journal-title":"Polish Psychological Bulletin"},{"key":"2022030421205791800_bib78","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9780262016438.003.0002","article-title":"Neural circuits of reward and decision making: Integrative networks across corticobasal ganglia loops","volume-title":"Neural basis of motivational and cognitive control","author":"Haber","year":"2011"},{"key":"2022030421205791800_bib79","volume-title":"Training a single AI model can emit as much carbon as five cars in their lifetimes","author":"Hao","year":"2019"},{"key":"2022030421205791800_bib80","doi-asserted-by":"publisher","first-page":"389","DOI":"10.3758\/s13421-014-0473-3","article-title":"Why is working memory capacity related to matrix reasoning tasks?","volume":"43","author":"Harrison","year":"2015","journal-title":"Memory & Cognition"},{"key":"2022030421205791800_bib81","doi-asserted-by":"publisher","first-page":"632","DOI":"10.1038\/nature07832","article-title":"Decoding reveals the contents of visual working memory in early visual areas","volume":"458","author":"Harrison","year":"2009","journal-title":"Nature"},{"key":"2022030421205791800_bib82","doi-asserted-by":"publisher","first-page":"1601","DOI":"10.1098\/rstb.2007.2055","article-title":"Towards an executive without a homunculus: Computational models of the prefrontal cortex\/basal ganglia system","volume":"362","author":"Hazy","year":"2007","journal-title":"Philosophical Transactions of the Royal Society of London, Serires B: Biological Sciences"},{"key":"2022030421205791800_bib83","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2022030421205791800_bib84","doi-asserted-by":"publisher","first-page":"2241","DOI":"10.1152\/jn.00901.2017","article-title":"Contribution of explicit processes to reinforcement-based motor learning","volume":"119","author":"Holland","year":"2018","journal-title":"Journal of Neurophysiology"},{"key":"2022030421205791800_bib85","doi-asserted-by":"publisher","first-page":"8391","DOI":"10.1073\/pnas.1918143117","article-title":"Humans incorporate trial-to-trial working memory uncertainty into rewarded decisions","volume":"117","author":"Honig","year":"2020","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib86","doi-asserted-by":"publisher","first-page":"338","DOI":"10.3758\/s13415-018-00674-z","article-title":"Refresh my memory: Episodic memory reinstatements intrude on working memory maintenance","volume":"19","author":"Hoskin","year":"2019","journal-title":"Cognitive, Affective, & Behavioral Neuroscience"},{"key":"2022030421205791800_bib87","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/4708.003.0004","article-title":"Information processing in modular circuits linking basal ganglia and cerebral cortex","volume-title":"Models of information processing in the basal ganglia","author":"Houk","year":"1995"},{"key":"2022030421205791800_bib88","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1016\/j.neuron.2013.09.009","article-title":"Hierarchical prediction errors in midbrain and basal forebrain during sensory learning","volume":"80","author":"Iglesias","year":"2013","journal-title":"Neuron"},{"key":"2022030421205791800_bib89","doi-asserted-by":"publisher","DOI":"10.1101\/571380","article-title":"Event segmentation reveals working memory forgetting rate","author":"Jafarpour","year":"2019","journal-title":"BioRxiv"},{"key":"2022030421205791800_bib90","doi-asserted-by":"publisher","first-page":"2670","DOI":"10.1162\/jocn_a_00677","article-title":"Adolescents adapt more slowly than adults to varying reward contingencies","volume":"26","author":"Javadi","year":"2014","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib91","doi-asserted-by":"publisher","first-page":"17382","DOI":"10.1523\/JNEUROSCI.3810-12.2012","article-title":"Prioritized maps of space in human frontoparietal cortex","volume":"32","author":"Jerde","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib92","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1016\/S0893-6080(02)00047-3","article-title":"Actor\u2013critic models of the basal ganglia: New anatomical and computational perspectives","volume":"15","author":"Joel","year":"2002","journal-title":"Neural Networks"},{"key":"2022030421205791800_bib93","doi-asserted-by":"publisher","first-page":"e40216","DOI":"10.1371\/journal.pone.0040216","article-title":"Probabilistic computation in human perception under variability in encoding precision","volume":"7","author":"Keshvari","year":"2012","journal-title":"PLoS One"},{"key":"2022030421205791800_bib94","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1016\/j.brainres.2007.08.049","article-title":"Impaired strategic decision making in schizophrenia","volume":"1180","author":"Kim","year":"2007","journal-title":"Brain Research"},{"key":"2022030421205791800_bib95","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.visres.2014.09.002","article-title":"Attentional priority determines working memory precision","volume":"105","author":"Klyszejko","year":"2014","journal-title":"Vision Research"},{"key":"2022030421205791800_bib96","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1167\/jov.20.8.3","article-title":"Working memory is corrupted by strategic changes in search templates","volume":"20","author":"Kong","year":"2020","journal-title":"Journal of Vision"},{"key":"2022030421205791800_bib97","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/neco_a_01339","article-title":"Flexible working memory through selective gating and attentional tagging","volume":"33","author":"Kruijne","year":"2021","journal-title":"Neural Computation"},{"key":"2022030421205791800_bib98","doi-asserted-by":"publisher","first-page":"13592","DOI":"10.1073\/pnas.0805423105","article-title":"Neural predictors of moment-to-moment fluctuations in cognitive flexibility","volume":"105","author":"Leber","year":"2008","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib99","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1016\/j.neuron.2016.12.040","article-title":"Dynamic interaction between reinforcement learning and attention in multidimensional environments","volume":"93","author":"Leong","year":"2017","journal-title":"Neuron"},{"key":"2022030421205791800_bib100","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1016\/j.conb.2020.10.012","article-title":"Dopamine, updated: Reward prediction error and beyond","volume":"67","author":"Lerner","year":"2021","journal-title":"Current Opinion in Neurobiology"},{"key":"2022030421205791800_bib101","doi-asserted-by":"publisher","first-page":"3699","DOI":"10.1016\/j.neuron.2021.08.022","article-title":"Joint representation of working memory and uncertainty in human cortex","volume":"109","author":"Li","year":"2021","journal-title":"Neuron"},{"key":"2022030421205791800_bib102","unstructured":"Lin, L.-J.\n           (1993). Reinforcement learning for robots using neural networks. Technical Report, DTIC Document."},{"key":"2022030421205791800_bib103","doi-asserted-by":"publisher","first-page":"eabf1357","DOI":"10.1126\/science.abf1357","article-title":"Experience replay is associated with efficient nonlocal learning","volume":"372","author":"Liu","year":"2021","journal-title":"Science"},{"key":"2022030421205791800_bib104","doi-asserted-by":"publisher","first-page":"87","DOI":"10.3389\/fncom.2012.00087","article-title":"Learning to use working memory: A reinforcement learning gating model of rule acquisition in rats","volume":"6","author":"Lloyd","year":"2012","journal-title":"Frontiers in Computational Neuroscience"},{"key":"2022030421205791800_bib105","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1162\/jocn.1997.9.3.330","article-title":"Dopaminergic modulation of working memory for spatial but not object cues in normal humans","volume":"9","author":"Luciana","year":"1997","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib106","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1162\/jocn.1992.4.1.58","article-title":"Facilitation of working memory in humans by a D2 dopamine receptor agonist","volume":"4","author":"Luciana","year":"1992","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib107","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1038\/36846","article-title":"The capacity of visual working memory for features and conjunctions","volume":"390","author":"Luck","year":"1997","journal-title":"Nature"},{"key":"2022030421205791800_bib108","doi-asserted-by":"publisher","first-page":"7013","DOI":"10.1523\/JNEUROSCI.2485-17.2018","article-title":"Working memory: Delay activity, yes! Persistent activity? Maybe not","volume":"38","author":"Lundqvist","year":"2018","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib109","doi-asserted-by":"publisher","first-page":"176","DOI":"10.1038\/s41593-019-0574-1","article-title":"Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors","volume":"23","author":"Maes","year":"2020","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib110","doi-asserted-by":"publisher","first-page":"1159","DOI":"10.1038\/s41593-019-0414-3","article-title":"Circuit mechanisms for the maintenance and manipulation of information in working memory","volume":"22","author":"Masse","year":"2019","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib111","doi-asserted-by":"publisher","first-page":"39","DOI":"10.3389\/fnhum.2011.00039","article-title":"A Bayesian foundation for individual learning under uncertainty","volume":"5","author":"Mathys","year":"2011","journal-title":"Frontiers in Human Neuroscience"},{"key":"2022030421205791800_bib112","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1093\/cercor\/bhab205","article-title":"Executive function assigns value to novel goal-congruent outcomes","volume":"32","author":"McDougle","year":"2021","journal-title":"Cerebral Cortex"},{"key":"2022030421205791800_bib113","doi-asserted-by":"publisher","first-page":"20","DOI":"10.3758\/s13423-020-01774-z","article-title":"Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning","volume":"28","author":"McDougle","year":"2020","journal-title":"Psychonomic Bulletin & Review"},{"key":"2022030421205791800_bib114","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1006\/brcg.1999.1099","article-title":"Basal ganglia output and cognition: Evidence from anatomical, behavioral, and clinical studies","volume":"42","author":"Middleton","year":"2000","journal-title":"Brain and Cognition"},{"key":"2022030421205791800_bib115","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"2022030421205791800_bib116","doi-asserted-by":"publisher","first-page":"107657","DOI":"10.1016\/j.neuropsychologia.2020.107657","article-title":"Rational use of episodic and working memory: A normative account of prospective memory","volume":"158","author":"Momennejad","year":"2021","journal-title":"Neuropsychologia"},{"key":"2022030421205791800_bib117","doi-asserted-by":"publisher","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive Hebbian learning","volume":"16","author":"Montague","year":"1996","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib118","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1523\/JNEUROSCI.18-01-00399.1998","article-title":"A model that accounts for activity in primate frontal cortex during a delayed matching-to-sample task","volume":"18","author":"Moody","year":"1998","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib119","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1073\/pnas.1619449114","article-title":"Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex","volume":"114","author":"Murray","year":"2017","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib120","doi-asserted-by":"publisher","first-page":"757","DOI":"10.1016\/j.tics.2021.06.001","article-title":"Rationalizing constraints on the capacity for cognitive control","volume":"25","author":"Musslick","year":"2021","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib121","article-title":"Learning to adapt in dynamic, real-world environments through meta-reinforcement learning","author":"Nagabandi","year":"2019","journal-title":"ArXiv"},{"key":"2022030421205791800_bib122","doi-asserted-by":"publisher","first-page":"486","DOI":"10.1037\/rev0000101","article-title":"Chunking as a rational strategy for lossy data compression in visual working memory","volume":"125","author":"Nassar","year":"2018","journal-title":"Psychological Review"},{"key":"2022030421205791800_bib123","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.jmp.2008.12.005","article-title":"Reinforcement learning in the brain","volume":"53","author":"Niv","year":"2009","journal-title":"Journal of Mathematical Psychology"},{"key":"2022030421205791800_bib124","doi-asserted-by":"publisher","first-page":"1544","DOI":"10.1038\/s41593-019-0470-8","article-title":"Learning task-state representations","volume":"22","author":"Niv","year":"2019","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib125","doi-asserted-by":"publisher","first-page":"8145","DOI":"10.1523\/JNEUROSCI.2978-14.2015","article-title":"Reinforcement learning in multidimensional environments relies on attention mechanisms","volume":"35","author":"Niv","year":"2015","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib126","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1162\/089976606775093909","article-title":"Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia","volume":"18","author":"O'Reilly","year":"2006","journal-title":"Neural Computation"},{"key":"2022030421205791800_bib127","doi-asserted-by":"publisher","first-page":"885","DOI":"10.1037\/bul0000153","article-title":"Benchmarks for models of short-term and working memory","volume":"144","author":"Oberauer","year":"2018","journal-title":"Psychological Bulletin"},{"key":"2022030421205791800_bib128","doi-asserted-by":"publisher","first-page":"1243","DOI":"10.1037\/0096-1523.32.5.1243","article-title":"Feature-based memory-driven attentional capture: Visual working memory content affects visual attention","volume":"32","author":"Olivers","year":"2006","journal-title":"Journal of Experimental Psychology. Human Perception and Performance"},{"key":"2022030421205791800_bib129","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1038\/sj.npp.1300108","article-title":"Schizophrenia subjects show intact success-related neural activation but impaired uncertainty processing during decision-making","volume":"28","author":"Paulus","year":"2003","journal-title":"Neuropsychopharmacology"},{"key":"2022030421205791800_bib130","article-title":"Learning policies with external memory","author":"Peshkin","year":"2001","journal-title":"ArXiv"},{"key":"2022030421205791800_bib131","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1037\/h0049234","article-title":"Short-term retention of individual verbal items","volume":"58","author":"Peterson","year":"1959","journal-title":"Journal of Experimental Psychology"},{"key":"2022030421205791800_bib132","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3389\/fpsyg.2013.00092","article-title":"The mixed instrumental controller: Using value of information to combine habitual choice and mental simulation","volume":"4","author":"Pezzulo","year":"2013","journal-title":"Frontiers in Psychology"},{"key":"2022030421205791800_bib133","doi-asserted-by":"publisher","first-page":"e1007963","DOI":"10.1371\/journal.pcbi.1007963","article-title":"A simple model for learning in volatile environments","volume":"16","author":"Piray","year":"2020","journal-title":"PLoS Computational Biology"},{"key":"2022030421205791800_bib134","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/j.schres.2007.07.015","article-title":"The Wisconsin card sorting impairment in schizophrenia is evident in the first four trials","volume":"106","author":"Prentice","year":"2008","journal-title":"Schizophrenia Research"},{"key":"2022030421205791800_bib135","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.nlm.2019.01.010","article-title":"Stress-induced impairment in goal-directed instrumental behaviour is moderated by baseline working memory","volume":"158","author":"Quaedflieg","year":"2019","journal-title":"Neurobiology of Learning and Memory"},{"key":"2022030421205791800_bib136","doi-asserted-by":"publisher","first-page":"e1008971","DOI":"10.1371\/journal.pcbi.1008971","article-title":"Analogous computations in working memory input, output and motor gating: Electrophysiological and computational modeling evidence","volume":"17","author":"Rac-Lubashevsky","year":"2021","journal-title":"PLoS Computational Biology"},{"key":"2022030421205791800_bib137","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1167\/12.13.21","article-title":"Introspective judgments predict the precision and likelihood of successful maintenance of visual working memory","volume":"12","author":"Rademaker","year":"2012","journal-title":"Journal of Vision"},{"key":"2022030421205791800_bib138","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1016\/j.tics.2019.01.010","article-title":"Holistic reinforcement learning: The role of structure and attention","volume":"23","author":"Radulescu","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib139","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1162\/jocn_a_01196","article-title":"Population dynamics of early visual cortex during working memory","volume":"30","author":"Rahmati","year":"2018","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib140","doi-asserted-by":"publisher","first-page":"994","DOI":"10.1162\/0898929054475118","article-title":"Working memory maintenance contributes to long-term memory formation: Neural and behavioral evidence","volume":"17","author":"Ranganath","year":"2005","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022030421205791800_bib141","article-title":"A theory of Pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement","volume-title":"Classical conditioning: Current research and theory","author":"Rescorla","year":"1972"},{"key":"2022030421205791800_bib142","doi-asserted-by":"publisher","first-page":"12990","DOI":"10.1523\/JNEUROSCI.1892-12.2012","article-title":"The relationship between working memory storage and elevated activity as measured with functional magnetic resonance imaging","volume":"32","author":"Riggall","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib143","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/j.cobeha.2020.10.003","article-title":"The role of executive function in shaping reinforcement learning","volume":"38","author":"Rmus","year":"2021","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"2022030421205791800_bib201","doi-asserted-by":"publisher","first-page":"e1004060","DOI":"10.1371\/journal.pcbi.1004060","article-title":"How attention can create synaptic tags for the learning of working memories in sequential tasks","volume":"11","author":"Rombouts","year":"2015","journal-title":"PLoS Computational Biology"},{"key":"2022030421205791800_bib202","first-page":"1871","article-title":"Neurally plausible reinforcement learning of working memory tasks","volume":"25","author":"Rombouts","year":"2012","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2022030421205791800_bib144","doi-asserted-by":"publisher","first-page":"20172035","DOI":"10.1098\/rspb.2017.2035","article-title":"Correlated individual differences suggest a common mechanism underlying metacognition in visual perception and visual short-term memory","volume":"284","author":"Samaha","year":"2017","journal-title":"Proceedings of the Royal Society of London, Series B: Biological Sciences"},{"key":"2022030421205791800_bib145","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1126\/science.1115270","article-title":"Representation of action-specific reward values in the striatum","volume":"310","author":"Samejima","year":"2005","journal-title":"Science"},{"key":"2022030421205791800_bib146","doi-asserted-by":"publisher","first-page":"9913","DOI":"10.1523\/JNEUROSCI.23-30-09913.2003","article-title":"Correlated coding of motivation and outcome of decision by dopamine neurons","volume":"23","author":"Satoh","year":"2003","journal-title":"Journal of Neuroscience"},{"key":"2022030421205791800_bib147","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.neuroimage.2013.11.034","article-title":"Striatal dysfunction during reversal learning in unmedicated schizophrenia patients","volume":"89","author":"Schlagenhauf","year":"2014","journal-title":"Neuroimage"},{"key":"2022030421205791800_bib148","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S0896-6273(02)00967-4","article-title":"Getting formal with dopamine and reward","volume":"36","author":"Schultz","year":"2002","journal-title":"Neuron"},{"key":"2022030421205791800_bib149","doi-asserted-by":"publisher","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"Schultz","year":"1997","journal-title":"Science"},{"key":"2022030421205791800_bib150","first-page":"233","article-title":"Reward-related signals carried by dopamine neurons","volume-title":"Models of information processing in the basal ganglia","author":"Schultz","year":"1995"},{"key":"2022030421205791800_bib151","doi-asserted-by":"publisher","first-page":"394","DOI":"10.3389\/fpsyg.2018.00394","article-title":"Working memory and reinforcement schedule jointly determine reinforcement learning in children: Potential implications for behavioral parent training","volume":"9","author":"Segers","year":"2018","journal-title":"Frontiers in Psychology"},{"key":"2022030421205791800_bib152","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1111\/j.1467-9280.2009.02276.x","article-title":"Stimulus-specific delay activity in human primary visual cortex","volume":"20","author":"Serences","year":"2009","journal-title":"Psychological Science"},{"key":"2022030421205791800_bib153","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1037\/0278-7393.5.2.179","article-title":"Contextual interference effects on the acquisition, retention, and transfer of a motor skill","volume":"5","author":"Shea","year":"1979","journal-title":"Journal of Experimental Psychology: Human Learning and Memory"},{"key":"2022030421205791800_bib154","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1016\/j.tics.2010.08.002","article-title":"Dopamine and adaptive memory","volume":"14","author":"Shohamy","year":"2010","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib155","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.schres.2004.03.020","article-title":"Schizophrenia patients demonstrate a distinctive pattern of decision-making impairment on the Iowa Gambling Task","volume":"72","author":"Shurman","year":"2005","journal-title":"Schizophrenia Research"},{"key":"2022030421205791800_bib156","doi-asserted-by":"publisher","first-page":"3275","DOI":"10.1152\/jn.00442.2018","article-title":"Somatosensory working memory in human reinforcement-based motor learning","volume":"120","author":"Sidarta","year":"2018","journal-title":"Journal of Neurophysiology"},{"key":"2022030421205791800_bib157","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1038\/nn.2450","article-title":"Synaptic computation underlying probabilistic inference","volume":"13","author":"Soltani","year":"2010","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib158","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1037\/0096-1523.31.2.248","article-title":"Early, involuntary top\u2013down guidance of attention from working memory","volume":"31","author":"Soto","year":"2005","journal-title":"Journal of Experimental Psychology: Human Perception and Performance"},{"key":"2022030421205791800_bib159","doi-asserted-by":"publisher","first-page":"2281","DOI":"10.3758\/s13423-018-1471-4","article-title":"The precision of spatial selection into the focus of attention in working memory","volume":"25","author":"Souza","year":"2018","journal-title":"Psychonomic Bulletin & Review"},{"key":"2022030421205791800_bib160","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1126\/science.153.3736.652","article-title":"High-speed scanning in human memory","volume":"153","author":"Sternberg","year":"1966","journal-title":"Science"},{"key":"2022030421205791800_bib161","article-title":"An integrated computational framework for attention, reinforcement learning, and working memory","volume":"6","author":"Stocco","year":"2017","journal-title":"AAAI 2017 Fall Symposium"},{"key":"2022030421205791800_bib162","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1016\/j.tics.2015.05.004","article-title":"\u2018Activity-silent\u2019 working memory in prefrontal cortex: A dynamic coding framework","volume":"19","author":"Stokes","year":"2015","journal-title":"Trends in Cognitive Sciences"},{"key":"2022030421205791800_bib163","article-title":"Energy and policy considerations for deep learning in NLP","author":"Strubell","year":"2019","journal-title":"ArXiv"},{"key":"2022030421205791800_bib164","doi-asserted-by":"publisher","first-page":"660","DOI":"10.1037\/xlm0000320","article-title":"Looking inward and back: Real-time monitoring of visual working memories","volume":"43","author":"Suchow","year":"2017","journal-title":"Journal of Experimental Psychology: Learning, Memory, and Cognition"},{"key":"2022030421205791800_bib165","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/S0306-4522(00)00554-6","article-title":"Modeling functions of striatal dopamine modulation in learning and planning","volume":"103","author":"Suri","year":"2001","journal-title":"Neuroscience"},{"key":"2022030421205791800_bib166","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/S0160-2896(01)00100-3","article-title":"Working-memory capacity explains reasoning ability\u2014And a little bit more","volume":"30","author":"S\u00fc\u00df","year":"2002","journal-title":"Intelligence"},{"key":"2022030421205791800_bib167","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"1998"},{"key":"2022030421205791800_bib168","doi-asserted-by":"publisher","first-page":"2136","DOI":"10.3758\/s13414-014-0633-3","article-title":"The binding pool: A model of shared neural resources for distinct items in visual working memory","volume":"76","author":"Swan","year":"2014","journal-title":"Attention, Perception & Psychophysics"},{"key":"2022030421205791800_bib169","doi-asserted-by":"publisher","first-page":"1689","DOI":"10.1371\/journal.pone.0075455","article-title":"Learning to use working memory in partially observable environments through dopaminergic reinforcement","volume":"21","author":"Todd","year":"2009","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2022030421205791800_bib170","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1080\/13825585.2019.1664389","article-title":"Relevance of working memory for reinforcement learning in older adults varies with timescale of learning","volume":"27","author":"van de Vijver","year":"2019","journal-title":"Aging, Neuropsychology, and Cognition"},{"key":"2022030421205791800_bib171","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1080\/13825585.2015.1020917","article-title":"Age-related changes in deterministic learning from positive versus negative performance feedback","volume":"22","author":"van de Vijver","year":"2015","journal-title":"Aging, Neuropsychology, and Cognition"},{"key":"2022030421205791800_bib172","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1037\/a0035234","article-title":"Factorial comparison of working memory models","volume":"121","author":"van den Berg","year":"2014","journal-title":"Psychological Review"},{"key":"2022030421205791800_bib173","doi-asserted-by":"publisher","first-page":"e34963","DOI":"10.7554\/eLife.34963","article-title":"A resource-rational theory of set size effects in human visual working memory","volume":"7","author":"van den Berg","year":"2018","journal-title":"eLife"},{"key":"2022030421205791800_bib174","doi-asserted-by":"publisher","first-page":"8780","DOI":"10.1073\/pnas.1117465109","article-title":"Variability in encoding precision accounts for visual short-term memory limitations","volume":"109","author":"van den Berg","year":"2012","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022030421205791800_bib175","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1177\/0956797613516146","article-title":"Accurate metacognition for visual sensory memory representations","volume":"25","author":"Vandenbroucke","year":"2014","journal-title":"Psychological Science"},{"key":"2022030421205791800_bib176","doi-asserted-by":"publisher","DOI":"10.3389\/fnbeh.2015.00225","article-title":"Modeling choice and reaction time during arbitrary visuomotor learning through the coordination of adaptive working memory and reinforcement learning","volume":"9","author":"Viejo","year":"2015","journal-title":"Frontiers in Behavioral Neuroscience"},{"key":"2022030421205791800_bib177","doi-asserted-by":"publisher","first-page":"756","DOI":"10.1016\/j.biopsych.2006.09.042","article-title":"Selective reinforcement learning deficits in schizophrenia support predictions from computational models of striatal-cortical dysfunction","volume":"62","author":"Waltz","year":"2007","journal-title":"Biological Psychiatry"},{"key":"2022030421205791800_bib178","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1037\/a0020882","article-title":"Altered probabilistic learning and response biases in schizophrenia: Behavioral evidence and neurocomputational modeling","volume":"25","author":"Waltz","year":"2011","journal-title":"Neuropsychology"},{"key":"2022030421205791800_bib179","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.schres.2007.03.010","article-title":"Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction","volume":"93","author":"Waltz","year":"2007","journal-title":"Schizophrenia Research"},{"key":"2022030421205791800_bib180","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1038\/s41593-018-0147-8","article-title":"Prefrontal cortex as a meta-reinforcement learning system","volume":"21","author":"Wang","year":"2018","journal-title":"Nature Neuroscience"},{"key":"2022030421205791800_bib181","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.bbr.2008.10.030","article-title":"Synaptic plasticity in the basal ganglia","volume":"199","author":"Wickens","year":"2009","journal-title":"Behavioural Brain Research"},{"key":"2022030421205791800_bib182","doi-asserted-by":"publisher","first-page":"1324","DOI":"10.1609\/aaai.v34i02.5488","article-title":"Transfer reinforcement learning using output-gated working memory","volume":"34","author":"Williams","year":"2020","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"2022030421205791800_bib183","doi-asserted-by":"publisher","first-page":"189","DOI":"10.3389\/fnhum.2011.00189","article-title":"Inferring relevance in a changing world","volume":"5","author":"Wilson","year":"2012","journal-title":"Frontiers in Human Neuroscience"},{"key":"2022030421205791800_bib184","doi-asserted-by":"publisher","DOI":"10.1101\/2020.03.19.997098","article-title":"Reward learning and working memory: Effects of massed versus spaced training and post-learning delay period","author":"Wimmer","year":"2020","journal-title":"BioRxiv"},{"key":"2022030421205791800_bib185","doi-asserted-by":"publisher","DOI":"10.1101\/2020.09.27.315432","article-title":"Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning","author":"Womelsdorf","year":"2020","journal-title":"BioRxiv"},{"key":"2022030421205791800_bib186","doi-asserted-by":"publisher","first-page":"5899","DOI":"10.1038\/s41598-021-85322-2","article-title":"Individual differences in experienced and observational decision-making illuminate interactions between reinforcement learning and declarative memory","volume":"11","author":"Yifrah","year":"2021","journal-title":"Scientific Reports"},{"key":"2022030421205791800_bib187","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1167\/jov.21.8.13","article-title":"Uncertainty is maintained and used in working memory","volume":"21","author":"Yoo","year":"2021","journal-title":"Journal of Vision"},{"key":"2022030421205791800_bib188","doi-asserted-by":"publisher","first-page":"16162","DOI":"10.1038\/s41598-018-34282-1","article-title":"Strategic allocation of working memory resource","volume":"8","author":"Yoo","year":"2018","journal-title":"Scientific Reports"},{"key":"2022030421205791800_bib189","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1038\/nature06860","article-title":"Discrete fixed-resolution representations in visual working memory","volume":"453","author":"Zhang","year":"2008","journal-title":"Nature"},{"key":"2022030421205791800_bib190","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1007\/s12559-017-9511-3","article-title":"A brain-inspired decision making model based on top\u2013down biasing of prefrontal cortex to basal ganglia and its application in autonomous UAV explorations","volume":"10","author":"Zhao","year":"2018","journal-title":"Cognitive Computation"},{"key":"2022030421205791800_bib191","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1162\/neco.1991.3.2.179","article-title":"Recurrent network model of the neural mechanism of short-term active memory","volume":"3","author":"Zipser","year":"1991","journal-title":"Neural Computation"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/34\/4\/551\/1996648\/jocn_a_01808.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/34\/4\/551\/1996648\/jocn_a_01808.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T16:21:24Z","timestamp":1646410884000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/34\/4\/551\/108895\/How-Working-Memory-and-Reinforcement-Learning-Are"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,5]]},"references-count":194,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,3,5]]},"published-print":{"date-parts":[[2022,3,5]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_01808","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,4]]},"published":{"date-parts":[[2022,3,5]]}}}