{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T21:22:42Z","timestamp":1785619362196,"version":"3.56.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1013696","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T00:00:00Z","timestamp":1763942400000}}],"reference-count":71,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T00:00:00Z","timestamp":1763337600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["R01MH136875"],"award-info":[{"award-number":["R01MH136875"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["R01MH135587"],"award-info":[{"award-number":["R01MH135587"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>\n                    Decisions in humans and other organisms depend, in part, on learning and using models that capture the statistical structure of the world, including the long-run expected outcomes of our actions. One prominent approach to forecasting such long-run outcomes is the successor representation (SR), which predicts future states aggregated over multiple timesteps. Although much behavioral and neural evidence suggests that people and animals use such a representation, it remains unknown how they acquire it. It has frequently been assumed to be learned by temporal difference bootstrapping (SR-TD(0)), but this assumption has largely not been empirically tested or compared to alternatives including eligibility traces (SR-TD(\n                    <jats:inline-formula id=\"pcbi.1013696.e001\">\n                      <jats:alternatives>\n                        <jats:graphic xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" id=\"pcbi.1013696.e001g\" mimetype=\"image\" position=\"anchor\" xlink:href=\"info:doi\/10.1371\/journal.pcbi.1013696.e001\" xlink:type=\"simple\"\/>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" display=\"inline\" id=\"M1\">\n                          <mml:mrow>\n                            <mml:mi>\u03bb<\/mml:mi>\n                            <mml:mo>&gt;<\/mml:mo>\n                            <mml:mn>0<\/mml:mn>\n                          <\/mml:mrow>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    )). Here we address this gap by leveraging trial-by-trial reaction times in graph sequence learning tasks, which are favorable for studying learning dynamics because the long horizons in these studies differentiate the transient update dynamics of different learning rules. We examined the behavior of SR-TD(\n                    <jats:italic>\u03bb<\/jats:italic>\n                    ) on a probabilistic graph learning task alongside a number of alternatives, and found that behavior was best explained by a hybrid model which learned via SR-TD(\n                    <jats:italic>\u03bb<\/jats:italic>\n                    ) alongside an additional predictive model of recency. The relatively large\n                    <jats:italic>\u03bb<\/jats:italic>\n                    we estimate indicates a predominant role of eligibility trace mechanisms over the bootstrap-based chaining typically assumed. Our results provide insight into how humans learn predictive representations, and demonstrate that people simultaneously learn the SR alongside lower-order predictions.\n                  <\/jats:p>","DOI":"10.1371\/journal.pcbi.1013696","type":"journal-article","created":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T18:39:17Z","timestamp":1763404757000},"page":"e1013696","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":5,"title":["Trial-by-trial learning of successor representations in human behavior"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2127-0507","authenticated-orcid":true,"given":"Ari E.","family":"Kahn","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dani S.","family":"Bassett","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5029-1430","authenticated-orcid":true,"given":"Nathaniel D.","family":"Daw","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,11,17]]},"reference":[{"issue":"1","key":"pcbi.1013696.ref001","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1002\/hipo.22523","article-title":"Statistical learning of temporal community structure in the hippocampus","volume":"26","author":"AC Schapiro","year":"2016","journal-title":"Hippocampus."},{"key":"pcbi.1013696.ref002","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.17086","article-title":"A map of abstract relational knowledge in the human hippocampal-entorhinal cortex","volume":"6","author":"MM Garvert","year":"2017","journal-title":"Elife."},{"issue":"11","key":"pcbi.1013696.ref003","doi-asserted-by":"crossref","first-page":"1643","DOI":"10.1038\/nn.4650","article-title":"The hippocampus as a predictive map","volume":"20","author":"KL Stachenfeld","year":"2017","journal-title":"Nat Neurosci."},{"issue":"9","key":"pcbi.1013696.ref004","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1038\/s41562-017-0180-8","article-title":"The successor representation in human reinforcement learning","volume":"1","author":"I Momennejad","year":"2017","journal-title":"Nat Hum Behav."},{"key":"pcbi.1013696.ref005","doi-asserted-by":"crossref","unstructured":"Russek EM, Momennejad I, Botvinick MM, Gershman SJ, Daw ND. Neural evidence for the successor representation in choice evaluation. 2021.","DOI":"10.1101\/2021.08.29.458114"},{"key":"pcbi.1013696.ref006","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.78904","article-title":"Successor-like representation guides the prediction of future events in human visual cortex and hippocampus","volume":"12","author":"M Ekman","year":"2023","journal-title":"Elife."},{"key":"pcbi.1013696.ref007","article-title":"Network structure influences the strength of learned neural representations","author":"AE Kahn","year":"2023","journal-title":"bioRxiv."},{"issue":"9","key":"pcbi.1013696.ref008","doi-asserted-by":"crossref","first-page":"1726","DOI":"10.1038\/s41562-024-01930-8","article-title":"Humans adaptively deploy forward and backward prediction","volume":"8","author":"PB Sharp","year":"2024","journal-title":"Nat Hum Behav."},{"issue":"4","key":"pcbi.1013696.ref009","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1162\/neco.1993.5.4.613","article-title":"Improving generalization for temporal difference learning: the successor representation","volume":"5","author":"P Dayan","year":"1993","journal-title":"Neural Computation."},{"key":"pcbi.1013696.ref010","doi-asserted-by":"crossref","unstructured":"Kahn AE, Daw ND. Humans rationally balance detailed and temporally abstract world models. Cold Spring Harbor Laboratory; 2023. https:\/\/doi.org\/10.1101\/2023.11.28.569070","DOI":"10.1101\/2023.11.28.569070"},{"issue":"1","key":"pcbi.1013696.ref011","doi-asserted-by":"crossref","first-page":"12733","DOI":"10.1038\/s41598-017-12876-5","article-title":"Process reveals structure: how a network is traversed mediates expectations about its architecture","volume":"7","author":"EA Karuza","year":"2017","journal-title":"Sci Rep."},{"issue":"12","key":"pcbi.1013696.ref012","doi-asserted-by":"crossref","first-page":"936","DOI":"10.1038\/s41562-018-0463-8","article-title":"Network constraints on learnability of probabilistic motor sequences","volume":"2","author":"AE Kahn","year":"2018","journal-title":"Nat Hum Behav."},{"issue":"1","key":"pcbi.1013696.ref013","doi-asserted-by":"crossref","first-page":"2313","DOI":"10.1038\/s41467-020-15146-7","article-title":"Abstract representations of events arise from mental errors in learning and memory","volume":"11","author":"CW Lynn","year":"2020","journal-title":"Nat Commun."},{"issue":"10","key":"pcbi.1013696.ref014","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1162\/jocn_a_01864","article-title":"Representations of temporal community structure in hippocampus and precuneus predict inductive reasoning decisions","volume":"34","author":"A Pudhiyidath","year":"2022","journal-title":"J Cogn Neurosci."},{"issue":"9","key":"pcbi.1013696.ref015","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1005768","article-title":"Predictive representations can link model-based reinforcement learning to model-free mechanisms","volume":"13","author":"EM Russek","year":"2017","journal-title":"PLoS Comput Biol."},{"issue":"6","key":"pcbi.1013696.ref016","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1162\/NECO_a_00282","article-title":"The successor representation and temporal context","volume":"24","author":"SJ Gershman","year":"2012","journal-title":"Neural Comput."},{"issue":"1891","key":"pcbi.1013696.ref017","first-page":"20181645","article-title":"Rethinking dopamine as generalized prediction error","volume":"285","author":"MPH Gardner","year":"2018","journal-title":"Proc Biol Sci."},{"issue":"7084","key":"pcbi.1013696.ref018","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1038\/nature04587","article-title":"Reverse replay of behavioural sequences in hippocampal place cells during the awake state","volume":"440","author":"DJ Foster","year":"2006","journal-title":"Nature."},{"issue":"21","key":"pcbi.1013696.ref019","doi-asserted-by":"crossref","DOI":"10.1016\/j.neuron.2023.07.017","article-title":"Dual credit assignment processes underlie dopamine signals in a complex spatial environment","volume":"111","author":"TA Krausz","year":"2023","journal-title":"Neuron."},{"key":"pcbi.1013696.ref020","doi-asserted-by":"crossref","unstructured":"Sagiv Y, Akam T, Witten IB, Daw ND. Prioritizing replay when future goals are unknown. 2024.","DOI":"10.1101\/2024.02.29.582822"},{"issue":"9","key":"pcbi.1013696.ref021","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1038\/s41567-020-0924-7","article-title":"Human information processing in complex networks","volume":"16","author":"CW Lynn","year":"2020","journal-title":"Nat Phys."},{"key":"pcbi.1013696.ref022","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.conb.2017.08.015","article-title":"The many worlds hypothesis of dopamine prediction error: implications of a parallel circuit architecture in the basal ganglia","volume":"46","author":"B Lau","year":"2017","journal-title":"Curr Opin Neurobiol."},{"issue":"8","key":"pcbi.1013696.ref023","doi-asserted-by":"crossref","first-page":"1574","DOI":"10.1038\/s41593-024-01689-1","article-title":"A feature-specific prediction error model explains dopaminergic heterogeneity","volume":"27","author":"RS Lee","year":"2024","journal-title":"Nat Neurosci."},{"issue":"8037","key":"pcbi.1013696.ref024","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1038\/s41586-024-07973-1","article-title":"Human hippocampal and entorhinal neurons encode the temporal structure of experience","volume":"635","author":"P Tacikowski","year":"2024","journal-title":"Nature."},{"issue":"3","key":"pcbi.1013696.ref025","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1037\/0096-3445.120.3.235","article-title":"Learning the structure of event sequences","volume":"120","author":"A Cleeremans","year":"1991","journal-title":"J Exp Psychol Gen."},{"issue":"7","key":"pcbi.1013696.ref026","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1111\/j.1460-9568.2011.07920.x","article-title":"Dissociating hippocampal and striatal contributions to sequential prediction learning","volume":"35","author":"AM Bornstein","year":"2012","journal-title":"Eur J Neurosci."},{"issue":"5","key":"pcbi.1013696.ref027","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1101\/lm.053306.120","article-title":"Visual object recognition is facilitated by temporal community structure","volume":"28","author":"E Kakaei","year":"2021","journal-title":"Learn Mem."},{"key":"pcbi.1013696.ref028","doi-asserted-by":"crossref","first-page":"119459","DOI":"10.1016\/j.neuroimage.2022.119459","article-title":"Reduced functional connectivity supports statistical learning of temporally distributed regularities","volume":"260","author":"J Park","year":"2022","journal-title":"Neuroimage."},{"issue":"3","key":"pcbi.1013696.ref029","doi-asserted-by":"crossref","first-page":"458","DOI":"10.1037\/xlm0001257","article-title":"Structure shapes the representation of a novel category","volume":"50","author":"SH Solomon","year":"2024","journal-title":"J Exp Psychol Learn Mem Cogn."},{"issue":"1","key":"pcbi.1013696.ref030","doi-asserted-by":"crossref","first-page":"4269","DOI":"10.1038\/s41467-024-48547-z","article-title":"Heuristics in risky decision-making relate to preferential representation of information","volume":"15","author":"EM Russek","year":"2024","journal-title":"Nat Commun."},{"issue":"2","key":"pcbi.1013696.ref031","doi-asserted-by":"crossref","DOI":"10.1523\/ENEURO.0361-21.2022","article-title":"Neurophysiological evidence for cognitive map formation during sequence learning","volume":"9","author":"J Stiso","year":"2022","journal-title":"eNeuro."},{"key":"pcbi.1013696.ref032","doi-asserted-by":"crossref","first-page":"102373","DOI":"10.1016\/j.pneurobio.2022.102373","article-title":"Dynamic emergence of relational structure network in human brains","volume":"219","author":"X Ren","year":"2022","journal-title":"Prog Neurobiol."},{"key":"pcbi.1013696.ref033","unstructured":"Wittkuhn L, Krippner LM, Schuck NW. Statistical learning of successor representations is related to on-task replay. 2022."},{"issue":"2","key":"pcbi.1013696.ref034","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1011312","article-title":"The successor representation subserves hierarchical abstraction for goal-directed behavior","volume":"20","author":"S Wientjes","year":"2024","journal-title":"PLoS Comput Biol."},{"key":"pcbi.1013696.ref035","unstructured":"Stachenfeld KL, Botvinick M, Gershman SJ. Design principles of the hippocampal cognitive map. Advances in Neural Information Processing Systems. Curran Associates, Inc.; 2014."},{"key":"pcbi.1013696.ref036","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.cobeha.2021.02.017","article-title":"Cognitive maps and novel inferences: a flexibility hierarchy","volume":"38","author":"ED Boorman","year":"2021","journal-title":"Current Opinion in Behavioral Sciences."},{"issue":"10","key":"pcbi.1013696.ref037","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1038\/s41593-022-01153-y","article-title":"How to build a cognitive map","volume":"25","author":"JCR Whittington","year":"2022","journal-title":"Nat Neurosci."},{"issue":"4","key":"pcbi.1013696.ref038","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1038\/s41593-023-01283-x","article-title":"Hippocampal spatio-predictive cognitive maps adaptively guide reward generalization","volume":"26","author":"MM Garvert","year":"2023","journal-title":"Nat Neurosci."},{"issue":"4","key":"pcbi.1013696.ref039","first-page":"537","article-title":"Cognitive graphs: representational substrates for planning","volume":"11","author":"J Yoo","year":"2024","journal-title":"Decision (Wash D C )."},{"key":"pcbi.1013696.ref040","doi-asserted-by":"crossref","unstructured":"Zhou CY, Talmi D, Daw N, Mattar MG. Episodic retrieval for model-based evaluation in sequential decision tasks. 2023.","DOI":"10.31234\/osf.io\/3sqjh"},{"key":"pcbi.1013696.ref041","doi-asserted-by":"crossref","unstructured":"Sutton RS. TD models: modeling the world at a mixture of time scales. In: Proceedings of the Twelfth International Conference on International Conference on Machine Learning. San Francisco, CA, USA, 1995. p. 531\u20139.","DOI":"10.1016\/B978-1-55860-377-6.50072-4"},{"key":"pcbi.1013696.ref042","doi-asserted-by":"crossref","unstructured":"Momennejad I, Howard MW. Predicting the future with multi-scale successor representations. Cold Spring Harbor Laboratory; 2018. https:\/\/doi.org\/10.1101\/449470","DOI":"10.1101\/449470"},{"key":"pcbi.1013696.ref043","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1146\/annurev.neuro.28.061604.135722","article-title":"Multiple dopamine functions at different time courses","volume":"30","author":"W Schultz","year":"2007","journal-title":"Annu Rev Neurosci."},{"issue":"3","key":"pcbi.1013696.ref044","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1016\/j.neuron.2010.06.031","article-title":"Multiple timescales of memory in lateral habenula and dopamine neurons","volume":"67","author":"ES Bromberg-Martin","year":"2010","journal-title":"Neuron."},{"issue":"4","key":"pcbi.1013696.ref045","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1038\/s41593-023-01566-3","article-title":"Dopamine transients follow a striatal gradient of reward time horizons","volume":"27","author":"A Mohebi","year":"2024","journal-title":"Nat Neurosci."},{"issue":"1","key":"pcbi.1013696.ref046","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1111\/j.1551-6709.2009.01076.x","article-title":"Direct associations or internal transformations? Exploring the mechanisms underlying sequential learning behavior","volume":"34","author":"TM Gureckis","year":"2010","journal-title":"Cogn Sci."},{"issue":"2","key":"pcbi.1013696.ref047","doi-asserted-by":"crossref","DOI":"10.1523\/ENEURO.0085-21.2021","article-title":"The planning horizon for movement sequences","volume":"8","author":"G Ariani","year":"2021","journal-title":"eNeuro."},{"issue":"3","key":"pcbi.1013696.ref048","doi-asserted-by":"crossref","DOI":"10.1523\/JNEUROSCI.1300-24.2024","article-title":"Cortical areas for planning sequences before and during movement","volume":"45","author":"G Ariani","year":"2025","journal-title":"J Neurosci."},{"key":"pcbi.1013696.ref049","unstructured":"Sutton RS. Temporal credit assignment in reinforcement learning. University of Massachusetts Amherst; 1984."},{"issue":"2","key":"pcbi.1013696.ref050","doi-asserted-by":"crossref","first-page":"131","DOI":"10.3758\/s13415-011-0027-0","article-title":"Learning from delayed feedback: neural responses in temporal credit assignment","volume":"11","author":"MM Walsh","year":"2011","journal-title":"Cogn Affect Behav Neurosci."},{"issue":"2","key":"pcbi.1013696.ref051","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0088725","article-title":"Neural correlates of temporal credit assignment in the parietal lobe","volume":"9","author":"TM Gersch","year":"2014","journal-title":"PLoS One."},{"key":"pcbi.1013696.ref052","doi-asserted-by":"crossref","first-page":"27056","DOI":"10.1038\/srep27056","article-title":"Neuronal activity in dorsomedial and dorsolateral striatum under the requirement for temporal credit assignment","volume":"6","author":"ES Her","year":"2016","journal-title":"Sci Rep."},{"issue":"10","key":"pcbi.1013696.ref053","doi-asserted-by":"crossref","DOI":"10.1016\/j.cell.2021.03.046","article-title":"Wave-like dopamine dynamics as a mechanism for spatiotemporal credit assignment","volume":"184","author":"AA Hamid","year":"2021","journal-title":"Cell."},{"issue":"37","key":"pcbi.1013696.ref054","doi-asserted-by":"crossref","first-page":"15462","DOI":"10.1073\/pnas.1014457108","article-title":"Dopamine neurons learn to encode the long-term value of multiple future rewards","volume":"108","author":"K Enomoto","year":"2011","journal-title":"Proc Natl Acad Sci U S A."},{"issue":"2","key":"pcbi.1013696.ref055","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1038\/s41593-019-0574-1","article-title":"Causal evidence supporting the proposal that dopamine transients function as temporal difference prediction errors","volume":"23","author":"EJP Maes","year":"2020","journal-title":"Nat Neurosci."},{"issue":"8","key":"pcbi.1013696.ref056","doi-asserted-by":"crossref","first-page":"1082","DOI":"10.1038\/s41593-022-01109-2","article-title":"A gradual temporal shift of dopamine responses mirrors the progression of temporal difference error in machine learning","volume":"25","author":"R Amo","year":"2022","journal-title":"Nat Neurosci."},{"issue":"1","key":"pcbi.1013696.ref057","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1023\/A:1022633531479","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"RS Sutton","year":"1988","journal-title":"Mach Learn."},{"issue":"1","key":"pcbi.1013696.ref058","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF01245020","article-title":"Striatal dopamine in motor activation and reward-mediated learning: steps towards a unifying model","volume":"80","author":"J Wickens","year":"1990","journal-title":"J Neural Transm Gen Sect."},{"key":"pcbi.1013696.ref059","doi-asserted-by":"crossref","unstructured":"Wickens J, K\u00f6tter R. Cellular models of reinforcement. Models of Information Processing in the Basal Ganglia. Cambridge, MA, US: The MIT Press; 1995. p. 187\u2013214.","DOI":"10.7551\/mitpress\/4708.003.0017"},{"key":"pcbi.1013696.ref060","doi-asserted-by":"crossref","unstructured":"Houk JC, Adams JL, Barto AG. A model of how the basal ganglia generate and use neural signals that predict reinforcement. Models of information processing in the basal ganglia. Cambridge, MA, US: The MIT Press; 1995. p. 249\u201370.","DOI":"10.7551\/mitpress\/4708.003.0020"},{"issue":"48","key":"pcbi.1013696.ref061","doi-asserted-by":"crossref","DOI":"10.1126\/sciadv.adj4897","article-title":"Exponential history integration with diverse temporal scales in retrosplenial cortex supports hyperbolic behavior","volume":"9","author":"BP Danskin","year":"2023","journal-title":"Sci Adv."},{"issue":"3","key":"pcbi.1013696.ref062","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1006\/jmps.2001.1388","article-title":"a distributed representation of temporal context","volume":"46","author":"MW Howard","year":"2002","journal-title":"Journal of Mathematical Psychology."},{"issue":"6","key":"pcbi.1013696.ref063","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1162\/NECO_a_00282","article-title":"The successor representation and temporal context","volume":"24","author":"SJ Gershman","year":"2012","journal-title":"Neural Comput."},{"key":"pcbi.1013696.ref064","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.80680","article-title":"Neural learning rules for generating flexible predictions and computing the successor representation","volume":"12","author":"C Fang","year":"2023","journal-title":"Elife."},{"key":"pcbi.1013696.ref065","unstructured":"Sutton RS, Pinette B. The learning of world models by connectionist networks. In: Proceedings of the Annual Meeting of the Cognitive Science Society; 1985."},{"issue":"12","key":"pcbi.1013696.ref066","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1003387","article-title":"Cortical and hippocampal correlates of deliberation during model-based decisions for rewards in humans","volume":"9","author":"AM Bornstein","year":"2013","journal-title":"PLoS Comput Biol."},{"key":"pcbi.1013696.ref067","unstructured":"van Seijen H, Mahmood AR, Pilarski PM, Machado MC, Sutton RS. True online temporal-difference learning. 2016."},{"issue":"4","key":"pcbi.1013696.ref068","article-title":"Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding","volume":"7","author":"QJM Huys","year":"2011","journal-title":"PLoS Comput Biol."},{"issue":"4","key":"pcbi.1013696.ref069","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1016\/j.neuroimage.2009.03.025","article-title":"Bayesian model selection for group studies","volume":"46","author":"KE Stephan","year":"2009","journal-title":"Neuroimage."},{"key":"pcbi.1013696.ref070","unstructured":"Bates D, Alday P, Kleinschmidt D, Jos\u00e9 Bayo\u00e1n Santiago Calder\u00f3n P, Zhan L, Noack A, et al. JuliaStats\/MixedModels.jl: v4.22.1. 2023."},{"issue":"1","key":"pcbi.1013696.ref071","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1137\/141000671","article-title":"Julia: a fresh approach to numerical computing","volume":"59","author":"J Bezanson","year":"2017","journal-title":"SIAM Rev."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1013696","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T00:00:00Z","timestamp":1763942400000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013696","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,24]],"date-time":"2025-11-24T18:40:15Z","timestamp":1764009615000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1013696"}},"subtitle":[],"editor":[{"given":"Wolfgang","family":"Einh\u00e4user","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,11,17]]},"references-count":71,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,11,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1013696","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,17]]}}}