{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T17:46:59Z","timestamp":1778089619485,"version":"3.51.4"},"reference-count":42,"publisher":"Society for Neuroscience","issue":"7","license":[{"start":{"date-parts":[[2011,8,16]],"date-time":"2011-08-16T00:00:00Z","timestamp":1313452800000},"content-version":"vor","delay-in-days":181,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Neurosci."],"published-print":{"date-parts":[[2011,2,16]]},"abstract":"<jats:p>In many cases, learning is thought to be driven by differences between the value of rewards we expect and rewards we actually receive. Yet learning can also occur when the identity of the reward we receive is not as expected, even if its value remains unchanged. Learning from changes in reward identity implies access to an internal model of the environment, from which information about the identity of the expected reward can be derived. As a result, such learning is not easily accounted for by model-free reinforcement learning theories such as temporal difference reinforcement learning (TDRL), which predicate learning on changes in reward value, but not identity. Here, we used unblocking procedures to assess learning driven by value- versus identity-based prediction errors. Rats were trained to associate distinct visual cues with different food quantities and identities. These cues were subsequently presented in compound with novel auditory cues and the reward quantity or identity was selectively changed. Unblocking was assessed by presenting the auditory cues alone in a probe test. Consistent with neural implementations of TDRL models, we found that the ventral striatum was necessary for learning in response to changes in reward value. However, this area, along with orbitofrontal cortex, was also required for learning driven by changes in reward identity. This observation requires that existing models of TDRL in the ventral striatum be modified to include information about the specific features of expected outcomes derived from model-based representations, and that the role of orbitofrontal cortex in these models be clearly delineated.<\/jats:p>","DOI":"10.1523\/jneurosci.5499-10.2011","type":"journal-article","created":{"date-parts":[[2011,2,16]],"date-time":"2011-02-16T20:01:33Z","timestamp":1297886493000},"page":"2700-2705","source":"Crossref","is-referenced-by-count":184,"title":["Ventral Striatum and Orbitofrontal Cortex Are Both Required for Model-Based, But Not Model-Free, Reinforcement Learning"],"prefix":"10.1523","volume":"31","author":[{"given":"Michael A.","family":"McDannald","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Federica","family":"Lucantonio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kathryn A.","family":"Burke","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yael","family":"Niv","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geoffrey","family":"Schoenbaum","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"393","published-online":{"date-parts":[[2011,2,16]]},"reference":[{"key":"2023041303485833000_31.7.2700.1","doi-asserted-by":"publisher","DOI":"10.1038\/nn1817"},{"key":"2023041303485833000_31.7.2700.2","doi-asserted-by":"crossref","unstructured":"Barto AG (1994) in Models of information processing in the basal ganglia, Adaptive critics and the basal ganglia, eds Houk JC Davis JL (MIT Press, Cambridge, MA), pp 215\u2013232.","DOI":"10.7551\/mitpress\/4708.003.0018"},{"key":"2023041303485833000_31.7.2700.3","doi-asserted-by":"publisher","DOI":"10.1038\/nature06993"},{"key":"2023041303485833000_31.7.2700.4","doi-asserted-by":"publisher","DOI":"10.1016\/S0149-7634(02)00007-6"},{"key":"2023041303485833000_31.7.2700.5","doi-asserted-by":"crossref","first-page":"3251","DOI":"10.1523\/JNEUROSCI.21-09-03251.2001","article-title":"The role of the nucleus accumbens in instrumental conditioning: evidence of a functional dissociation between accumbens core and shell","volume":"21","author":"Corbit","year":"2001","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.6","doi-asserted-by":"publisher","DOI":"10.1038\/nn1560"},{"key":"2023041303485833000_31.7.2700.7","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2006.03.002"},{"key":"2023041303485833000_31.7.2700.8","doi-asserted-by":"crossref","first-page":"6610","DOI":"10.1523\/JNEUROSCI.19-15-06610.1999","article-title":"Orbitofrontal cortex and representation of incentive value in associative learning","volume":"19","author":"Gallagher","year":"1999","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.9","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2460"},{"key":"2023041303485833000_31.7.2700.10","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2010.04.016"},{"key":"2023041303485833000_31.7.2700.11","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.10.4.476"},{"key":"2023041303485833000_31.7.2700.12","doi-asserted-by":"publisher","DOI":"10.1038\/1124"},{"key":"2023041303485833000_31.7.2700.13","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.1921-04.2004"},{"key":"2023041303485833000_31.7.2700.14","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00047-3"},{"key":"2023041303485833000_31.7.2700.15","unstructured":"Kamin LJ (1969) in Punishment and aversive behavior, Predictability, suprise, attention, and conditioning, eds Campbell BA Church RM (Appleton-Century-Crofts, New York), pp 242\u2013259."},{"key":"2023041303485833000_31.7.2700.16","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.1054-08.2008"},{"key":"2023041303485833000_31.7.2700.17","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pbio.1000173"},{"key":"2023041303485833000_31.7.2700.18","doi-asserted-by":"publisher","DOI":"10.1016\/j.nlm.2009.11.002"},{"key":"2023041303485833000_31.7.2700.19","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5301-04.2005"},{"key":"2023041303485833000_31.7.2700.20","doi-asserted-by":"crossref","first-page":"8655","DOI":"10.1523\/JNEUROSCI.21-21-08655.2001","article-title":"The circuitry mediating cocaine-induced reinstatement of drug-seeking behavior","volume":"21","author":"McFarland","year":"2001","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.21","doi-asserted-by":"publisher","DOI":"10.1016\/0301-0082(80)90018-0"},{"key":"2023041303485833000_31.7.2700.22","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive hebbian learning","volume":"16","author":"Montague","year":"1996","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.23","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2008.03.006"},{"key":"2023041303485833000_31.7.2700.24","doi-asserted-by":"publisher","DOI":"10.1126\/science.1094285"},{"key":"2023041303485833000_31.7.2700.25","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5443-06.2007"},{"key":"2023041303485833000_31.7.2700.26","doi-asserted-by":"publisher","DOI":"10.1038\/nature04676"},{"key":"2023041303485833000_31.7.2700.27","doi-asserted-by":"crossref","first-page":"2401","DOI":"10.1523\/JNEUROSCI.19-06-02401.1999","article-title":"Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine","volume":"19","author":"Parkinson","year":"1999","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.28","unstructured":"Paxinos G Watson C (1998) The rat brain in stereotaxic coordinates (Academic Press, San Diego), Ed 4."},{"key":"2023041303485833000_31.7.2700.29","doi-asserted-by":"crossref","first-page":"11078","DOI":"10.1523\/JNEUROSCI.23-35-11078.2003","article-title":"Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task","volume":"23","author":"Pickens","year":"2003","journal-title":"J Neurosci"},{"key":"2023041303485833000_31.7.2700.30","doi-asserted-by":"publisher","DOI":"10.3758\/BF03199671"},{"key":"2023041303485833000_31.7.2700.31","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1038\/nrn2753","article-title":"A new perspective on the role of the orbitofrontal cortex in adaptive behaviour","volume":"10","author":"Schoenbaum","year":"2009","journal-title":"Nat Rev Neurosci"},{"key":"2023041303485833000_31.7.2700.32","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"2023041303485833000_31.7.2700.33","doi-asserted-by":"publisher","DOI":"10.1037\/0735-7044.116.2.267"},{"key":"2023041303485833000_31.7.2700.34","doi-asserted-by":"crossref","first-page":"126","DOI":"10.3389\/fnint.2010.00126","article-title":"Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on Pavlovian conditioned responding","volume":"4","author":"Singh","year":"2010","journal-title":"Front Integr Neurosci"},{"key":"2023041303485833000_31.7.2700.35","unstructured":"Sutton RS Barto AG (1990) in Learning and computational neuroscience: foundations of adaptive networks, Time-derivative models of Pavlovian reinforcement, eds Gabriel M Moore J (MIT, Boston), pp 497\u2013537."},{"key":"2023041303485833000_31.7.2700.36","doi-asserted-by":"publisher","DOI":"10.1038\/19525"},{"key":"2023041303485833000_31.7.2700.37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3389\/neuro.07.001.2009","article-title":"Covert expectation-of-reward in rat ventral striatum at decision points","volume":"3","author":"van der Meer","year":"2009","journal-title":"Front Integr Neurosci"},{"key":"2023041303485833000_31.7.2700.38","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2010.06.023"},{"key":"2023041303485833000_31.7.2700.39","doi-asserted-by":"publisher","DOI":"10.1016\/j.tins.2004.06.006"},{"key":"2023041303485833000_31.7.2700.40","doi-asserted-by":"publisher","DOI":"10.1038\/35083500"},{"key":"2023041303485833000_31.7.2700.41","doi-asserted-by":"publisher","DOI":"10.1046\/j.1460-9568.2003.02922.x"},{"key":"2023041303485833000_31.7.2700.42","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2010.02.027"}],"container-title":["The Journal of Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1523\/JNEUROSCI.5499-10.2011","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,4]],"date-time":"2024-04-04T05:36:51Z","timestamp":1712209011000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jneurosci.org\/lookup\/doi\/10.1523\/JNEUROSCI.5499-10.2011"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,2,16]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2011,2,16]]},"published-print":{"date-parts":[[2011,2,16]]}},"alternative-id":["10.1523\/JNEUROSCI.5499-10.2011"],"URL":"https:\/\/doi.org\/10.1523\/jneurosci.5499-10.2011","relation":{},"ISSN":["0270-6474","1529-2401"],"issn-type":[{"value":"0270-6474","type":"print"},{"value":"1529-2401","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,2,16]]}}}