{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T16:45:05Z","timestamp":1791909905221,"version":"4.3.4"},"reference-count":49,"publisher":"MIT Press","issue":"11","funder":[{"name":"National Institute of General Medical Sciences Centers of Biomedical Research Excellence","award":["5P20GM103645-10"],"award-info":[{"award-number":["5P20GM103645-10"]}],"id":[{"id":"https:\/\/ror.org\/04q48ey07","id-type":"ROR","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014-23-1-2792"],"award-info":[{"award-number":["N00014-23-1-2792"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Defense Advanced Research Projects Agency","award":["D24AP00261"],"award-info":[{"award-number":["D24AP00261"]}],"id":[{"id":"https:\/\/ror.org\/02caytj08","id-type":"ROR","asserted-by":"crossref"}]},{"name":"Office of Naval Research","award":["N00014-23-1-2792"],"award-info":[{"award-number":["N00014-23-1-2792"]}]},{"name":"National Institutes of Health","award":["S10OD025181"],"award-info":[{"award-number":["S10OD025181"]}]},{"name":"Google DeepMind","id":[{"id":"https:\/\/ror.org\/00971b260","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Working memory (WM) is thought to rely on sophisticated frontostriatal mechanisms for selective gating, supporting updating and readout of information to and from distinct memory \u201caddresses\u201d (neural populations). According to computational models, capacity limitations in WM arise due to challenges in \u201crole addressability\u201d: learning to correctly bind items in memory to their respective roles and assigning credit to the corresponding gating operations. However, these conclusions are based on particular assumptions about biological neural networks, and it is unclear whether the principles generalize to other architectures. To address this question, we examine whether similar gating mechanisms emerge in Transformer neural network architectures, which have demonstrated success on tasks requiring executive function\u2014the ability to represent, coordinate, and manage multiple subtasks\u2014yet lack intentionally built-in gating mechanisms. We analyze the mechanisms that emerge within Transformers trained on human WM tasks explicitly designed to place demands on gating. We find that the Transformer's attention mechanism develops role-addressable input and output gating, but only when trained on task distributions that benefit from frontostriatal-like gating mechanisms. Moreover, these gating strategies support enhanced generalization and variable binding and increase the models' effective capacity to store and access multiple items in memory, resembling the constraints found in frontostriatal models. These results suggest that gating mechanisms serve a fundamental computational role in managing role addressability and binding and highlight opportunities for future research on computational similarities between modern artificial intelligence architectures and models of the human brain.<\/jats:p>","DOI":"10.1162\/jocn.a.2679","type":"journal-article","created":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T15:28:49Z","timestamp":1785338929000},"page":"2128-2140","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Transformer Mechanisms Mimic Frontostriatal Gating Operations When Trained on Human Working Memory Tasks"],"prefix":"10.1162","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6410-8414","authenticated-orcid":true,"given":"Aneri","family":"Soni","sequence":"first","affiliation":[{"name":"Brown University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron","family":"Traylor","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jack","family":"Merullo","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael J.","family":"Frank","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ellie","family":"Pavlick","sequence":"additional","affiliation":[{"name":"Brown University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2026,11,1]]},"reference":[{"key":"2026101311462215100_bib1","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1093\/cercor\/bhr117","article-title":"Mechanisms of hierarchical reinforcement learning in cortico-striatal circuits 2: Evidence from fMRI","volume":"22","author":"Badre","year":"2012","journal-title":"Cerebral Cortex"},{"key":"2026101311462215100_bib2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.0473","article-title":"Neural machine translation by jointly learning to align and translate","author":"Bahdanau","year":"2014","journal-title":"arXiv"},{"key":"2026101311462215100_bib3","doi-asserted-by":"publisher","first-page":"9788","DOI":"10.1523\/JNEUROSCI.1513-10.2010","article-title":"Keeping memory clear and stable\u2014The contribution of 18 human basal ganglia and prefrontal cortex to working memory","volume":"30","author":"Baier","year":"2010","journal-title":"Journal of Neuroscience"},{"key":"2026101311462215100_bib4","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.cognition.2017.12.001","article-title":"Learning and transfer of working memory gating policies","volume":"172","author":"Bhandari","year":"2018","journal-title":"Cognition"},{"key":"2026101311462215100_bib5","doi-asserted-by":"publisher","first-page":"1877","DOI":"10.48550\/arXiv.2005.14165","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026101311462215100_bib6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.11527","article-title":"Memory transformer","author":"Burtsev","year":"2020","journal-title":"arXiv"},{"key":"2026101311462215100_bib7","doi-asserted-by":"publisher","first-page":"e1009854","DOI":"10.1371\/journal.pcbi.1009854","article-title":"Thunderstruck: The acdc model of flexible sequences and rhythms in recurrent neural circuits","volume":"18","author":"Calderon","year":"2022","journal-title":"PLoS Computational Biology"},{"key":"2026101311462215100_bib8","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1016\/j.neuron.2014.01.002","article-title":"Corticostriatal output gating during selection from working memory","volume":"81","author":"Chatham","year":"2014","journal-title":"Neuron"},{"key":"2026101311462215100_bib9","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1037\/a0030852","article-title":"Cognitive control over learning: Creating, clustering, and generalizing task-set structure","volume":"120","author":"Collins","year":"2013","journal-title":"Psychological Review"},{"key":"2026101311462215100_bib10","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1016\/S0079-6123(07)00020-9","article-title":"What are the differences between long-term, short-term, and working memory?","volume":"169","author":"Cowan","year":"2008","journal-title":"Progress in Brain Research"},{"key":"2026101311462215100_bib11","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1285","article-title":"Transformer-xl: Attentive language models beyond a fixed-length context","author":"Dai","year":"2019","journal-title":"arXiv"},{"key":"2026101311462215100_bib12","doi-asserted-by":"publisher","first-page":"1068","DOI":"10.1016\/j.conb.2012.05.011","article-title":"How to set the switches on this thing","volume":"22","author":"Dayan","year":"2012","journal-title":"Current Opinion in Neurobiology"},{"key":"2026101311462215100_bib13","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1093\/cercor\/bhr114","article-title":"Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: Computational analysis","volume":"22","author":"Frank","year":"2012","journal-title":"Cerebral Cortex"},{"key":"2026101311462215100_bib14","doi-asserted-by":"publisher","first-page":"1694","DOI":"10.1016\/j.neuron.2025.02.029","article-title":"Key-value memory in the brain","volume":"113","author":"Gershman","year":"2025","journal-title":"Neuron"},{"key":"2026101311462215100_bib15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.05969","article-title":"Localizing model behavior with path patching","author":"Goldowsky-Dill","year":"2023","journal-title":"arXiv"},{"key":"2026101311462215100_bib16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1410.5401","article-title":"Neural turing machines","author":"Graves","year":"2014","journal-title":"arXiv"},{"key":"2026101311462215100_bib17","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1038\/nature20101","article-title":"Hybrid computing using a neural network with dynamic external memory","volume":"538","author":"Graves","year":"2016","journal-title":"Nature"},{"key":"2026101311462215100_bib18","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Computation"},{"key":"2026101311462215100_bib19","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1037\/0278-7393.26.2.336","article-title":"Working-memory capacity, proactive interference, and divided attention: Limits on long-term memory retrieval","volume":"26","author":"Kane","year":"2000","journal-title":"Journal of Experimental Psychology: Learning, Memory, and Cognition"},{"key":"2026101311462215100_bib20","doi-asserted-by":"publisher","first-page":"16390","DOI":"10.1073\/pnas.1303547110","article-title":"Indirection and symbol-like processing in the prefrontal cortex and basal ganglia","volume":"110","author":"Kriete","year":"2013","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2026101311462215100_bib21","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2026101311462215100_bib22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2209.11143","article-title":"Towards understanding grokking: An effective theory of representation learning","volume-title":"NeurIPs","author":"Liu","year":"2022"},{"key":"2026101311462215100_bib23","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1016\/j.tics.2013.06.006","article-title":"Visual working memory capacity: From psychophysics and neurobiology to individual differences","volume":"17","author":"Luck","year":"2013","journal-title":"Trends in Cognitive Sciences"},{"key":"2026101311462215100_bib24","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1038\/nn.3655","article-title":"Changing concepts of working memory","volume":"17","author":"Ma","year":"2014","journal-title":"Nature Neuroscince"},{"key":"2026101311462215100_bib25","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1177\/09637214241268098","article-title":"How can deep neural networks inform theory in psychological science?","volume":"33","author":"McGrath","year":"2024","journal-title":"Current Directions in Psychological Science"},{"key":"2026101311462215100_bib26","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1038\/nn2024","article-title":"Prefrontal cortex and basal ganglia control access to working memory","volume":"11","author":"McNab","year":"2008","journal-title":"Nature Neuroscience"},{"key":"2026101311462215100_bib27","volume-title":"Transformer lens","author":"Nanda","year":"2022"},{"key":"2026101311462215100_bib28","doi-asserted-by":"publisher","first-page":"486","DOI":"10.1037\/rev0000101","article-title":"Chunking as a rational strategy for lossy data compression in visual working memory","volume":"125","author":"Nassar","year":"2018","journal-title":"Psychological Review"},{"key":"2026101311462215100_bib29","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1162\/089976606775093909","article-title":"Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia","volume":"18","author":"O'Reilly","year":"2006","journal-title":"Neural Computation"},{"key":"2026101311462215100_bib30","doi-asserted-by":"publisher","first-page":"673","DOI":"10.3389\/fnhum.2013.00673","article-title":"The focus of attention in working memory\u2014From metaphors to mechanisms","volume":"7","author":"Oberauer","year":"2013","journal-title":"Frontiers in Human Neuroscience"},{"key":"2026101311462215100_bib31","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1037\/rev0000044","article-title":"An interference model of visual working memory","volume":"124","author":"Oberauer","year":"2017","journal-title":"Psychological Review"},{"key":"2026101311462215100_bib32","article-title":"Mechanistic interpretability, variables, and the importance of interpretable bases","author":"Olah","year":"2022"},{"key":"2026101311462215100_bib33","first-page":"411","article-title":"Direct and indirect effects","volume-title":"Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence","author":"Pearl","year":"2001"},{"key":"2026101311462215100_bib34","doi-asserted-by":"publisher","first-page":"e1008971","DOI":"10.1371\/journal.pcbi.1008971","article-title":"Analogous computations in working memory input, output and motor gating: Electrophysiological and computational modeling evidence","volume":"17","author":"Rac-Lubashevsky","year":"2021","journal-title":"PLoS Computational Biology"},{"key":"2026101311462215100_bib35","doi-asserted-by":"publisher","first-page":"951","DOI":"10.1037\/xlm0000223","article-title":"Dissociating working memory updating and automatic updating: The reference-back paradigm","volume":"42","author":"Rac-Lubashevsky","year":"2016","journal-title":"Journal of Experimental Psychology: Learning, Memory, and Cognition"},{"key":"2026101311462215100_bib36","doi-asserted-by":"publisher","first-page":"1136","DOI":"10.1126\/science.aah7011","article-title":"Reactivation of latent working memories with transcranial magnetic stimulation","volume":"354","author":"Rose","year":"2016","journal-title":"Science"},{"key":"2026101311462215100_bib37","doi-asserted-by":"publisher","first-page":"e2510270122","DOI":"10.1073\/pnas.2510270122","article-title":"Parallel trade-offs in human cognition and neural networks: The dynamic interplay between in-context and in-weight learning","volume":"122","author":"Russin","year":"2025","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2026101311462215100_bib38","doi-asserted-by":"publisher","first-page":"RP97894","DOI":"10.7554\/eLife.97894","article-title":"Adaptive chunking improves effective working memory capacity in a prefrontal cortex and basal ganglia circuit","volume":"13","author":"Soni","year":"2024","journal-title":"eLife"},{"key":"2026101311462215100_bib39","doi-asserted-by":"publisher","first-page":"2136","DOI":"10.3758\/s13414-014-0633-3","article-title":"The binding pool: A model of shared neural resources for distinct items in visual working memory","volume":"76","author":"Swan","year":"2014","journal-title":"Attention, Perception, & Psychophysics"},{"key":"2026101311462215100_bib40","article-title":"Learning to use working memory in partially observable environments through dopaminergic reinforcement","volume-title":"Advances in neural information processing systems 21\u2014Proceedings of the 2008 Conference","author":"Todd","year":"2009"},{"key":"2026101311462215100_bib41","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.13971","article-title":"Llama: Open and efficient foundation language models","author":"Touvron","year":"2023","journal-title":"arXiv"},{"key":"2026101311462215100_bib42","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1037\/a0035234","article-title":"Factorial comparison of working memory models","volume":"121","author":"van den Berg","year":"2014","journal-title":"Psychological Review"},{"key":"2026101311462215100_bib43","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1706.03762","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2026101311462215100_bib44","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/nature04171","article-title":"Neural measures reveal individual differences in controlling access to working memory","volume":"438","author":"Vogel","year":"2005","journal-title":"Nature"},{"key":"2026101311462215100_bib45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1909.06639","article-title":"Tree transformer: Integrating tree structures into self-attention","author":"Wang","year":"2019","journal-title":"arXiv"},{"key":"2026101311462215100_bib46","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2211.00593","article-title":"Interpretability in the wild: A circuit for indirect object identification in gpt-2 small","author":"Wang","year":"2022","journal-title":"arXiv"},{"key":"2026101311462215100_bib47","doi-asserted-by":"publisher","first-page":"11228","DOI":"10.1523\/JNEUROSCI.0735-12.2012","article-title":"From distributed resources to limited slots in multiple-item working memory: A spiking network model with normalization","volume":"32","author":"Wei","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"2026101311462215100_bib48","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2112.04035","article-title":"Relating transformers to models and neural representations of the hippocampal formation","author":"Whittington","year":"2021","journal-title":"arXiv"},{"key":"2026101311462215100_bib49","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1038\/nature06860","article-title":"Discrete fixed-resolution representations in visual working memory","volume":"453","author":"Zhang","year":"2008","journal-title":"Nature"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/38\/11\/2128\/2617974\/jocn.a.2679.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/38\/11\/2128\/2617974\/jocn.a.2679.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T15:46:30Z","timestamp":1791906390000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/38\/11\/2128\/137953\/Transformer-Mechanisms-Mimic-Frontostriatal-Gating"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":49,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2026,11,1]]},"published-print":{"date-parts":[[2026,11,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn.a.2679","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]}}}