{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T06:18:31Z","timestamp":1772173111693,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009897","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,4,6]],"date-time":"2022-04-06T00:00:00Z","timestamp":1649203200000}}],"reference-count":41,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T00:00:00Z","timestamp":1648166400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/publicdomain\/zero\/1.0\/"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA042065"],"award-info":[{"award-number":["R01DA042065"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA050647"],"award-info":[{"award-number":["R01DA050647"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>There is no single way to represent a task. Indeed, despite experiencing the same task events and contingencies, different subjects may form distinct task representations. As experimenters, we often assume that subjects represent the task as we envision it. However, such a representation cannot be taken for granted, especially in animal experiments where we cannot deliver explicit instruction regarding the structure of the task. Here, we tested how rats represent an odor-guided choice task in which two odor cues indicated which of two responses would lead to reward, whereas a third odor indicated free choice among the two responses. A parsimonious task representation would allow animals to learn from the forced trials what is the better option to choose in the free-choice trials. However, animals may not necessarily generalize across odors in this way. We fit reinforcement-learning models that use different task representations to trial-by-trial choice behavior of individual rats performing this task, and quantified the degree to which each animal used the more parsimonious representation, generalizing across trial types. Model comparison revealed that most rats did not acquire this representation despite extensive experience. Our results demonstrate the importance of formally testing possible task representations that can afford the observed behavior, rather than assuming that animals\u2019 task representations abide by the generative task structure that governs the experimental design.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009897","type":"journal-article","created":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T13:49:42Z","timestamp":1648216182000},"page":"e1009897","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Minimal cross-trial generalization in learning the representation of an odor-guided choice task"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6833-8329","authenticated-orcid":true,"given":"Mingyu","family":"Song","sequence":"first","affiliation":[]},{"given":"Yuji K.","family":"Takahashi","sequence":"additional","affiliation":[]},{"given":"Amanda C.","family":"Burton","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2854-6593","authenticated-orcid":true,"given":"Matthew R.","family":"Roesch","sequence":"additional","affiliation":[]},{"given":"Geoffrey","family":"Schoenbaum","sequence":"additional","affiliation":[]},{"given":"Yael","family":"Niv","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4742-6976","authenticated-orcid":true,"given":"Angela J.","family":"Langdon","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,3,25]]},"reference":[{"key":"pcbi.1009897.ref001","doi-asserted-by":"crossref","first-page":"103891","DOI":"10.1016\/j.beproc.2019.103891","article-title":"Uncovering the \u201cstate\u201d: Tracing the hidden state representations that structure learning and decision-making","volume":"167","author":"AJ Langdon","year":"2019","journal-title":"Behavioural Processes"},{"issue":"8","key":"pcbi.1009897.ref002","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation Learning: A Review and New Perspectives","volume":"35","author":"Y Bengio","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"10","key":"pcbi.1009897.ref003","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1038\/s41593-019-0470-8","article-title":"Learning task-state representations","volume":"22","author":"Y Niv","year":"2019","journal-title":"Nature Neuroscience"},{"key":"pcbi.1009897.ref004","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"1998"},{"issue":"5","key":"pcbi.1009897.ref005","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1016\/j.tics.2019.02.006","article-title":"Reinforcement Learning, Fast and Slow","volume":"23","author":"M Botvinick","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"pcbi.1009897.ref006","unstructured":"Wang JX, Kurth-Nelson Z, Tirumala D, Soyer H, Leibo JZ, Munos R, et al. Learning to reinforcement learn; 2017."},{"issue":"6","key":"pcbi.1009897.ref007","doi-asserted-by":"crossref","first-page":"860","DOI":"10.1038\/s41593-018-0147-8","article-title":"Prefrontal cortex as a meta-reinforcement learning system","volume":"21","author":"JX Wang","year":"2018","journal-title":"Nature Neuroscience"},{"issue":"2","key":"pcbi.1009897.ref008","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1038\/s41593-018-0310-2","article-title":"Task representations in neural networks trained to perform many cognitive tasks","volume":"22","author":"GR Yang","year":"2019","journal-title":"Nature Neuroscience"},{"issue":"4","key":"pcbi.1009897.ref009","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.neuron.2006.06.027","article-title":"Encoding of Time-Discounted Rewards in Orbitofrontal Cortex Is Independent of Value Representation","volume":"51","author":"MR Roesch","year":"2006","journal-title":"Neuron"},{"issue":"1","key":"pcbi.1009897.ref010","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.neuron.2016.05.015","article-title":"Temporal Specificity of Reward Prediction Errors Signaled by Putative Dopamine Neurons in Rat VTA Depends on Ventral Striatum","volume":"91","author":"YK Takahashi","year":"2016","journal-title":"Neuron"},{"issue":"42","key":"pcbi.1009897.ref011","doi-asserted-by":"crossref","first-page":"13365","DOI":"10.1523\/JNEUROSCI.2572-09.2009","article-title":"Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards","volume":"29","author":"MR Roesch","year":"2009","journal-title":"Journal of Neuroscience"},{"issue":"12","key":"pcbi.1009897.ref012","doi-asserted-by":"crossref","first-page":"2350","DOI":"10.1038\/s41386-018-0058-0","article-title":"Previous cocaine self-administration disrupts reward expectancy encoding in ventral striatum","volume":"43","author":"AC Burton","year":"2018","journal-title":"Neuropsychopharmacology"},{"issue":"1","key":"pcbi.1009897.ref013","doi-asserted-by":"crossref","DOI":"10.18637\/jss.v076.i01","article-title":"Stan: A probabilistic programming language","volume":"76","author":"B Carpenter","year":"2017","journal-title":"Journal of statistical software"},{"key":"pcbi.1009897.ref014","doi-asserted-by":"crossref","DOI":"10.1201\/b16018","volume-title":"Bayesian data analysis","author":"A Gelman","year":"2013"},{"key":"pcbi.1009897.ref015","first-page":"3571","article-title":"Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory","volume":"11","author":"S Watanabe","year":"2010","journal-title":"J Mach Learn Res"},{"issue":"5","key":"pcbi.1009897.ref016","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1007\/s11222-016-9696-4","article-title":"Practical Bayesian Model Evaluation Using Leave-One-out Cross-Validation and WAIC","volume":"27","author":"A Vehtari","year":"2017","journal-title":"Statistics and Computing"},{"issue":"6","key":"pcbi.1009897.ref017","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1016\/j.cub.2019.01.048","article-title":"Rat Orbitofrontal Ensemble Activity Contains Multiplexed but Dissociable Representations of Value and Task Structure in an Odor Sequence Task","volume":"29","author":"J Zhou","year":"2019","journal-title":"Current Biology"},{"issue":"9","key":"pcbi.1009897.ref018","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1038\/nn.4613","article-title":"Dorsal hippocampus contributes to model-based planning","volume":"20","author":"KJ Miller","year":"2017","journal-title":"Nature Neuroscience"},{"issue":"6398","key":"pcbi.1009897.ref019","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1126\/science.aar8644","article-title":"Sensitivity to \u201csunk costs\u201d in mice, rats, and humans","volume":"361","author":"BM Sweis","year":"2018","journal-title":"Science"},{"issue":"2","key":"pcbi.1009897.ref020","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1111\/tops.12142","article-title":"Rational Use of Cognitive Resources: Levels of Analysis Between the Computational and the Algorithmic","volume":"7","author":"TL Griffiths","year":"2015","journal-title":"Topics in Cognitive Science"},{"key":"pcbi.1009897.ref021","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1017\/S0140525X1900061X","article-title":"Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources","volume":"43","author":"F Lieder","year":"2020","journal-title":"Behavioral and Brain Sciences"},{"key":"pcbi.1009897.ref022","doi-asserted-by":"crossref","unstructured":"Honey RC, Hall G. Acquired equivalence and distinctiveness of cues.; 1989.","DOI":"10.1037\/0097-7403.15.4.338"},{"issue":"18","key":"pcbi.1009897.ref023","doi-asserted-by":"crossref","first-page":"4819","DOI":"10.1523\/JNEUROSCI.5443-06.2007","article-title":"Orbitofrontal Cortex Mediates Outcome Encoding in Pavlovian But Not Instrumental Conditioning","volume":"27","author":"SB Ostlund","year":"2007","journal-title":"Journal of Neuroscience"},{"issue":"7","key":"pcbi.1009897.ref024","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1101\/lm.2229311","article-title":"Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling","volume":"18","author":"KM Wassum","year":"2011","journal-title":"Learning & Memory"},{"issue":"3","key":"pcbi.1009897.ref025","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"RJ Williams","year":"1992","journal-title":"Machine learning"},{"key":"pcbi.1009897.ref026","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.cobeha.2021.04.020","article-title":"Value-free reinforcement learning: policy optimization as a minimal model of operant behavior","volume":"41","author":"D Bennett","year":"2021","journal-title":"Current Opinion in Behavioral Sciences"},{"issue":"27","key":"pcbi.1009897.ref027","doi-asserted-by":"crossref","first-page":"11235","DOI":"10.1073\/pnas.1103317108","article-title":"Detection and avoidance of a carnivore odor by prey","volume":"108","author":"DM Ferrero","year":"2011","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"4","key":"pcbi.1009897.ref028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pbio.0060082","article-title":"Rapid Encoding and Perception of Novel Odors in the Rat","volume":"6","author":"DW Wesson","year":"2008","journal-title":"PLOS Biology"},{"issue":"11","key":"pcbi.1009897.ref029","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1038\/nn1142","article-title":"Speed and accuracy of olfactory discrimination in the rat","volume":"6","author":"N Uchida","year":"2003","journal-title":"Nature Neuroscience"},{"issue":"6","key":"pcbi.1009897.ref030","doi-asserted-by":"crossref","first-page":"1395","DOI":"10.1016\/j.neuron.2017.08.025","article-title":"Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected Rewards","volume":"95","author":"YK Takahashi","year":"2017","journal-title":"Neuron"},{"issue":"7847","key":"pcbi.1009897.ref031","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1038\/s41586-020-03061-2","article-title":"Evolving schema representations in orbitofrontal ensembles during learning","volume":"590","author":"J Zhou","year":"2021","journal-title":"Nature"},{"issue":"1","key":"pcbi.1009897.ref032","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1037\/a0030852","article-title":"Cognitive control over learning: creating, clustering, and generalizing task-set structure","volume":"120","author":"AG Collins","year":"2013","journal-title":"Psychological review"},{"issue":"7","key":"pcbi.1009897.ref033","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.tics.2006.05.004","article-title":"Bayesian theories of conditioning in a changing world","volume":"10","author":"AC Courville","year":"2006","journal-title":"Trends in Cognitive Sciences"},{"key":"pcbi.1009897.ref034","doi-asserted-by":"crossref","unstructured":"Redish AD, Jensen S, Johnson A, Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addiction, relapse, and problem gambling.; 2007.","DOI":"10.1037\/0033-295X.114.3.784"},{"key":"pcbi.1009897.ref035","doi-asserted-by":"crossref","first-page":"164","DOI":"10.3389\/fnbeh.2013.00164","article-title":"Gradual extinction prevents the return of fear: implications for the discovery of state","volume":"7","author":"S Gershman","year":"2013","journal-title":"Frontiers in Behavioral Neuroscience"},{"key":"pcbi.1009897.ref036","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.neuropharm.2013.05.040","article-title":"On the motivational properties of reward cues: Individual differences","volume":"76","author":"TE Robinson","year":"2014","journal-title":"Neuropharmacology"},{"issue":"1-4","key":"pcbi.1009897.ref037","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1163\/156853987X00099","article-title":"Individual Differences in Behavioural Reaction To a Changing Environment in Mice and Rats","volume":"100","author":"JM Koolhaas","year":"1987","journal-title":"Behaviour"},{"issue":"2","key":"pcbi.1009897.ref038","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1523\/JNEUROSCI.5498-10.2012","article-title":"Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain","volume":"32","author":"Y Niv","year":"2012","journal-title":"Journal of Neuroscience"},{"issue":"10","key":"pcbi.1009897.ref039","doi-asserted-by":"crossref","first-page":"1544","DOI":"10.1038\/s41593-019-0470-8","article-title":"Learning task-state representations","volume":"22","author":"Y Niv","year":"2019","journal-title":"Nature neuroscience"},{"key":"pcbi.1009897.ref040","first-page":"64","article-title":"A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement","author":"RA Rescorla","year":"1972","journal-title":"Current research and theory"},{"key":"pcbi.1009897.ref041","unstructured":"Team SD. PyStan: the Python interface to Stan; 2018. http:\/\/mc-stan.org."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009897","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,4,6]],"date-time":"2022-04-06T00:00:00Z","timestamp":1649203200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009897","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,6]],"date-time":"2022-04-06T13:57:05Z","timestamp":1649253425000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009897"}},"subtitle":[],"editor":[{"given":"Daniele","family":"Marinazzo","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,3,25]]},"references-count":41,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3,25]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009897","relation":{"has-preprint":[{"id-type":"doi","id":"10.31234\/osf.io\/rgcak","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,25]]}}}