{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:27:31Z","timestamp":1772119651189,"version":"3.50.1"},"reference-count":75,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2009,10,22]],"date-time":"2009-10-22T00:00:00Z","timestamp":1256169600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Comput Neurosci"],"published-print":{"date-parts":[[2010,2]]},"DOI":"10.1007\/s10827-009-0191-1","type":"journal-article","created":{"date-parts":[[2009,10,21]],"date-time":"2009-10-21T12:22:08Z","timestamp":1256127728000},"page":"107-130","source":"Crossref","is-referenced-by-count":13,"title":["Alternative time representation in dopamine models"],"prefix":"10.1007","volume":"28","author":[{"given":"Fran\u00e7ois","family":"Rivest","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John F.","family":"Kalaska","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yoshua","family":"Bengio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2009,10,22]]},"reference":[{"key":"191_CR1","first-page":"1475","volume-title":"Neural information processing systems","author":"B Bakker","year":"2002","unstructured":"Bakker, B. (2002). Reinforcement learning with long short-term memory. In T. G. Dietterich, S. Becker & Z. Ghahramani (Eds.), Neural information processing systems (pp. 1475\u20131482). Cambridge: MIT."},{"key":"191_CR2","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1006\/lmot.2001.1104","volume":"33","author":"PD Balsam","year":"2002","unstructured":"Balsam, P. D., Drew, M. R., & Yang, C. (2002). Timing at the start of associative learning. Learning and Motivation, 33, 141\u2013155.","journal-title":"Learning and Motivation"},{"key":"191_CR3","unstructured":"Barto, A. G. (1995). Adaptive critics and the basal ganglia. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 215\u2013232). The MIT."},{"key":"191_CR4","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1016\/j.neunet.2007.04.028","volume":"20","author":"M Bertin","year":"2007","unstructured":"Bertin, M., Schweighofer, N., & Doya, K. (2007). Multiple model-based reinforcement learning explains dopamine neuronal activity. Neural Networks, 20, 668\u2013675.","journal-title":"Neural Networks"},{"key":"191_CR5","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1006\/nlme.2001.4039","volume":"76","author":"AV Beylin","year":"2001","unstructured":"Beylin, A. V., Gandhi, C. C., Wood, G. E., Talk, A. C., Matzel, L. D., & Shors, T. J. (2001). The role of the hippocampus in trace conditioning: temporal discontinuity or task difficulty? Neurobiology of Learning and Memory, 76, 447\u2013461.","journal-title":"Neurobiology of Learning and Memory"},{"key":"191_CR6","first-page":"713","volume-title":"Control of cognitive processes: Attention and performance XVIII","author":"TS Braver","year":"2000","unstructured":"Braver, T. S., & Cohen, J. D. (2000). On the control of control: The role of dopamine in regulating prefrontal function and working memory. In S. Monsell & J. Driver (Eds.), Control of cognitive processes: Attention and performance XVIII (pp. 713\u2013737). Cambridge: MIT."},{"key":"191_CR7","doi-asserted-by":"crossref","first-page":"1196","DOI":"10.1093\/cercor\/bhg100","volume":"13","author":"CD Brody","year":"2003","unstructured":"Brody, C. D., Hernandez, A., Zainos, A., & Romo, R. (2003). Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cerebral Cortex, 13, 1196\u20131207.","journal-title":"Cerebral Cortex"},{"key":"191_CR8","doi-asserted-by":"crossref","first-page":"10502","DOI":"10.1523\/JNEUROSCI.19-23-10502.1999","volume":"19","author":"J Brown","year":"1999","unstructured":"Brown, J., Bullock, D., & Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. Journal of Neuroscience, 19, 10502\u201310511.","journal-title":"Journal of Neuroscience"},{"key":"191_CR9","first-page":"305","volume":"26","author":"CV Buhusi","year":"2000","unstructured":"Buhusi, C. V., & Meck, W. H. (2000). Timing for the absence of a stimulus: the gap paradigm reversed. Journal of Experimental Psychology: Animal Behavior Processes, 26, 305\u2013322.","journal-title":"Journal of Experimental Psychology: Animal Behavior Processes"},{"key":"191_CR10","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1038\/nrn1764","volume":"6","author":"CV Buhusi","year":"2005","unstructured":"Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nature Reviews. Neuroscience, 6, 755\u2013765.","journal-title":"Nature Reviews. Neuroscience"},{"key":"191_CR11","first-page":"3","volume-title":"Functional and neural mechanisms of interval timing","author":"RM Church","year":"2003","unstructured":"Church, R. M. (2003). A concise introduction to scalar timing theory. In W. H. Meck (Ed.), Functional and neural mechanisms of interval timing (pp. 3\u201322). Boca Raton: CRC."},{"key":"191_CR12","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1126\/science.280.5360.77","volume":"280","author":"RE Clark","year":"1998","unstructured":"Clark, R. E., & Squire, L. R. (1998). Classical conditioning and brain systems: the role of awareness. Science, 280, 77\u201381.","journal-title":"Science"},{"key":"191_CR13","doi-asserted-by":"crossref","unstructured":"Daw, N. D., & Doya, K. (2006). The computational neurobiology of learning and reward. Curr. Opin. Neurobiol.","DOI":"10.1016\/j.conb.2006.03.006"},{"key":"191_CR14","first-page":"83","volume-title":"Neural information processing systems","author":"ND Daw","year":"2003","unstructured":"Daw, N. D., Courville, A. C., & Touretzky, D. S. (2003). Timing and partial observability in the dopamine system. In S. Becker, S. Thrun & K. Obermayer (Eds.), Neural information processing systems (pp. 83\u201390). Cambridge: MIT."},{"key":"191_CR15","doi-asserted-by":"crossref","first-page":"1637","DOI":"10.1162\/neco.2006.18.7.1637","volume":"18","author":"ND Daw","year":"2006","unstructured":"Daw, N. D., Courville, A. C., & Touretzky, D. S. (2006). Representation and timing in theories of the dopamine system. Neural Computation, 18, 1637\u20131677.","journal-title":"Neural Computation"},{"key":"191_CR16","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1007\/s002210050474","volume":"121","author":"JF Dormont","year":"1998","unstructured":"Dormont, J. F., Conde, H., & Farin, D. (1998). The role of the pedunculopontine tegmental nucleus in relation to conditioned motor performance in the cat. I. Context-dependent and reinforcement-related single unit activity. Experimental Brain Research, 121, 401\u2013410.","journal-title":"Experimental Brain Research"},{"key":"191_CR17","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1016\/S0893-6080(99)00046-5","volume":"12","author":"K Doya","year":"1999","unstructured":"Doya, K. (1999). What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Networks, 12, 961\u2013974.","journal-title":"Neural Networks"},{"key":"191_CR18","doi-asserted-by":"crossref","first-page":"732","DOI":"10.1016\/S0959-4388(00)00153-7","volume":"10","author":"K Doya","year":"2000","unstructured":"Doya, K. (2000). Complementary roles of basal ganglia and cerebellum in learning and motor control. Current Opinion in Neurobiology, 10, 732\u2013739.","journal-title":"Current Opinion in Neurobiology"},{"key":"191_CR19","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1037\/0033-295X.110.1.126","volume":"110","author":"V Dragoi","year":"2003","unstructured":"Dragoi, V., Staddon, J. E., Palmer, R. G., & Buhusi, C. V. (2003). Interval timing as an emergent learning property. Psychological Review, 110, 126\u2013144.","journal-title":"Psychological Review"},{"key":"191_CR20","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1097\/00001756-200404090-00001","volume":"15","author":"D Durstewitz","year":"2004","unstructured":"Durstewitz, D. (2004). Neural representation of interval time. NeuroReport, 15, 745\u2013749.","journal-title":"NeuroReport"},{"issue":"2415","key":"191_CR21","first-page":"284","volume":"2002","author":"D Eck","year":"2002","unstructured":"Eck, D., & Schmidhuber, A. (2002). Learning the long-term structure of the blues. Artificial Neural Networks\u2014Icann, 2002(2415), 284\u2013289.","journal-title":"Artificial Neural Networks\u2014Icann"},{"key":"191_CR22","doi-asserted-by":"crossref","first-page":"1898","DOI":"10.1126\/science.1077349","volume":"299","author":"CD Fiorillo","year":"2003","unstructured":"Fiorillo, C. D., Tobler, P. N., & Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science, 299, 1898\u20131902.","journal-title":"Science"},{"key":"191_CR23","doi-asserted-by":"crossref","first-page":"966","DOI":"10.1038\/nn.2159","volume":"11","author":"CD Fiorillo","year":"2008","unstructured":"Fiorillo, C. D., Newsome, W. T., & Schultz, W. (2008). The temporal precision of reward prediction in dopamine neurons. Nature Neuroscience, 11, 966\u2013973.","journal-title":"Nature Neuroscience"},{"key":"191_CR24","doi-asserted-by":"crossref","first-page":"1468","DOI":"10.1162\/neco.2007.19.6.1468","volume":"19","author":"RV Florian","year":"2007","unstructured":"Florian, R. V. (2007). Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Computation, 19, 1468\u20131502.","journal-title":"Neural Computation"},{"key":"191_CR25","doi-asserted-by":"crossref","first-page":"879","DOI":"10.1016\/0361-9230(86)90102-4","volume":"17","author":"M Fukuda","year":"1986","unstructured":"Fukuda, M., Ono, T., Nishino, H., & Nakamura, K. (1986). Neuronal responses in monkey lateral hypothalamus during operant feeding behavior. Brain Research Bulletin, 17, 879\u2013883.","journal-title":"Brain Research Bulletin"},{"key":"191_CR26","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/0361-9230(90)90260-7","volume":"25","author":"M Fukuda","year":"1990","unstructured":"Fukuda, M., Ono, T., Nakamura, K., & Tamura, R. (1990). Dopamine and ACh involvement in plastic learning by hypothalamic neurons in rats. Brain Research Bulletin, 25, 109\u2013114.","journal-title":"Brain Research Bulletin"},{"key":"191_CR27","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1152\/jn.1989.61.2.331","volume":"61","author":"S Funahashi","year":"1989","unstructured":"Funahashi, S., Bruce, C. J., & Goldman-Rakic, P. S. (1989). Mnemonic coding of visual space in the monkey\u2019s dorsolateral prefrontal cortex. Journal of Neurophysiology, 61, 331\u2013349.","journal-title":"Journal of Neurophysiology"},{"key":"191_CR28","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1037\/0033-295X.107.2.289","volume":"107","author":"CR Gallistel","year":"2000","unstructured":"Gallistel, C. R., & Gibbon, J. (2000). Time, rate, and conditioning. Psychological Review, 107, 289\u2013344.","journal-title":"Psychological Review"},{"key":"191_CR29","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1162\/089976600300015015","volume":"12","author":"FA Gers","year":"2000","unstructured":"Gers, F. A., Schmidhuber, J., & Cummins, F. (2000). Learning to forget: continual prediction with LSTM. Neural Computation, 12, 2451\u20132471.","journal-title":"Neural Computation"},{"key":"191_CR30","first-page":"115","volume":"3","author":"FA Gers","year":"2002","unstructured":"Gers, F. A., Schraudolph, N. N., & Schmidhuber, J. (2002). Learning precise timing with LSTM recurrent networks. Journal of Machine Learning Research, 3, 115\u2013143.","journal-title":"Journal of Machine Learning Research"},{"key":"191_CR31","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9, 1735\u20131780.","journal-title":"Neural Computation"},{"key":"191_CR32","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1038\/1124","volume":"1","author":"JR Hollerman","year":"1998","unstructured":"Hollerman, J. R., & Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1, 304\u2013309.","journal-title":"Nature Neuroscience"},{"key":"191_CR33","first-page":"23","volume-title":"Functional and neural mechanisms of interval timing","author":"JW Hopson","year":"2003","unstructured":"Hopson, J. W. (2003). General learning models: Timing without a clock. In W. H. Meck (Ed.), Functional and neural mechanisms of interval timing (pp. 23\u201360). Boca Raton: CRC."},{"key":"191_CR34","unstructured":"Houk, J. C., Adams, J. L., & Barto, A. G. (1995). A model of how the basal ganglia generate and use neural signals that predict reinforcement. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 249\u2013270). The MIT."},{"key":"191_CR35","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/j.tics.2008.04.002","volume":"12","author":"RB Ivry","year":"2008","unstructured":"Ivry, R. B., & Schlerf, J. E. (2008). Dedicated and intrinsic models of time perception. Trends in Cognitive Sciences, 12, 273\u2013280.","journal-title":"Trends in Cognitive Sciences"},{"key":"191_CR36","doi-asserted-by":"crossref","first-page":"2443","DOI":"10.1093\/cercor\/bhl152","volume":"17","author":"EM Izhikevich","year":"2007","unstructured":"Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cerebral Cortex, 17, 2443\u20132452.","journal-title":"Cerebral Cortex"},{"key":"191_CR37","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/S0306-4522(99)00575-8","volume":"96","author":"D Joel","year":"2000","unstructured":"Joel, D., & Weiner, I. (2000). The connections of the dopaminergic system with the striatum in rats and primates: an analysis with respect to the functional and compartmental organization of the striatum. Neuroscience, 96, 451\u2013474.","journal-title":"Neuroscience"},{"key":"191_CR38","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.neuron.2007.01.006","volume":"53","author":"UR Karmarkar","year":"2007","unstructured":"Karmarkar, U. R., & Buonomano, D. V. (2007). Timing in the absence of clocks: encoding time in neural network states. Neuron, 53, 427\u2013438.","journal-title":"Neuron"},{"key":"191_CR39","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.1162\/neco.2008.04-08-750","volume":"21","author":"C Kolodziejski","year":"2009","unstructured":"Kolodziejski, C., Porr, B., & Worgotter, F. (2009). On the asymptotic equivalence between differential hebbian and temporal difference learning. Neural Computation, 21, 1173\u20131202.","journal-title":"Neural Computation"},{"key":"191_CR40","doi-asserted-by":"crossref","first-page":"546","DOI":"10.1038\/35087595","volume":"412","author":"Y Komura","year":"2001","unstructured":"Komura, Y., Tamura, R., Uwano, T., Nishijo, H., Kaga, K., & Ono, T. (2001). Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature, 412, 546\u2013549.","journal-title":"Nature"},{"key":"191_CR41","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1016\/j.tins.2005.07.008","volume":"28","author":"M Laubach","year":"2005","unstructured":"Laubach, M. (2005). Who\u2019s on first? What\u2019s on second? The time course of learning in corticostriatal systems. Trends in Neurosciences, 28, 508\u2013511.","journal-title":"Trends in Neurosciences"},{"key":"191_CR42","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1152\/jn.00734.2007","volume":"99","author":"MA Lebedev","year":"2008","unstructured":"Lebedev, M. A., O\u2019Doherty, J. E., & Nicolelis, M. A. (2008). Decoding of temporal intervals from cortical ensemble activity. Journal of Neurophysiology, 99, 166\u2013186.","journal-title":"Journal of Neurophysiology"},{"key":"191_CR43","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/S0896-6273(03)00185-5","volume":"38","author":"MI Leon","year":"2003","unstructured":"Leon, M. I., & Shadlen, M. N. (2003). Representation of time by neurons in the posterior parietal cortex of the macaque. Neuron, 38, 317\u2013327.","journal-title":"Neuron"},{"key":"191_CR44","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/S1364-6613(02)01906-X","volume":"6","author":"PA Lewis","year":"2002","unstructured":"Lewis, P. A. (2002). Finding the timer. Trends in Cognitive Sciences, 6, 195\u2013196.","journal-title":"Trends in Cognitive Sciences"},{"key":"191_CR45","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1152\/jn.1992.67.1.145","volume":"67","author":"T Ljungberg","year":"1992","unstructured":"Ljungberg, T., Apicella, P., & Schultz, W. (1992). Responses of monkey dopamine neurons during learning of behavioral reactions. Journal of Neurophysiology, 67, 145\u2013163.","journal-title":"Journal of Neurophysiology"},{"key":"191_CR46","doi-asserted-by":"crossref","first-page":"254","DOI":"10.1007\/s002210100818","volume":"141","author":"C Lucchetti","year":"2001","unstructured":"Lucchetti, C., & Bon, L. (2001). Time-modulated neuronal activity in the premotor cortex of macaque monkeys. Experimental Brain Research, 141, 254\u2013260.","journal-title":"Experimental Brain Research"},{"key":"191_CR47","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s00421-005-1360-1","volume":"95","author":"C Lucchetti","year":"2005","unstructured":"Lucchetti, C., Ulrici, A., & Bon, L. (2005). Dorsal premotor areas of nonhuman primate: functional flexibility in time domain. European Journal of Applied Physiology, 95, 121\u2013130.","journal-title":"European Journal of Applied Physiology"},{"key":"191_CR48","doi-asserted-by":"crossref","first-page":"3034","DOI":"10.1162\/neco.2008.11-07-654","volume":"20","author":"EA Ludvig","year":"2008","unstructured":"Ludvig, E. A., Sutton, R. S., & Kehoe, E. J. (2008). Stimulus representation and the timing of reward-prediction errors in models of the dopamine system. Neural Computation, 20, 3034\u20133054.","journal-title":"Neural Computation"},{"key":"191_CR49","unstructured":"Ludvig, E. A., Sutton, R. S., Verbeek, E., & Kehoe, E. J. (2009). A computational model of hippocampal function in trace conditioning. In D. Koller, D. Schuurmans, Y. Bengio, & L. Bottou (Eds.), Neural information processing systems (pp. 993\u20131000)."},{"key":"191_CR50","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1152\/jn.1994.72.2.1024","volume":"72","author":"J Mirenowicz","year":"1994","unstructured":"Mirenowicz, J., & Schultz, W. (1994). Importance of unpredictability for reward responses in primate dopamine neurons. Journal of Neurophysiology, 72, 1024\u20131027.","journal-title":"Journal of Neurophysiology"},{"key":"191_CR51","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","volume":"16","author":"PR Montague","year":"1996","unstructured":"Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive hebbian learning. Journal of Neuroscience, 16, 1936\u20131947.","journal-title":"Journal of Neuroscience"},{"key":"191_CR52","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1038\/nature03015","volume":"431","author":"PR Montague","year":"2004","unstructured":"Montague, P. R., Hyman, S. E., & Cohen, J. D. (2004). Computational roles for dopamine in behavioural control. Nature, 431, 760\u2013767.","journal-title":"Nature"},{"key":"191_CR53","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.neuron.2004.06.012","volume":"43","author":"G Morris","year":"2004","unstructured":"Morris, G., Arkadir, D., Nevet, A., Vaadia, E., & Bergman, H. (2004). Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron, 43, 133\u2013143.","journal-title":"Neuron"},{"key":"191_CR54","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1038\/nn1743","volume":"9","author":"G Morris","year":"2006","unstructured":"Morris, G., Nevet, A., Arkadir, D., Vaadia, E., & Bergman, H. (2006). Midbrain dopamine neurons encode decisions for future action. Nature Neuroscience, 9, 1057\u20131063.","journal-title":"Nature Neuroscience"},{"key":"191_CR55","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1162\/089976606775093909","volume":"18","author":"RC O\u2019Reilly","year":"2006","unstructured":"O\u2019Reilly, R. C., & Frank, M. J. (2006). Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Computation, 18, 283\u2013328.","journal-title":"Neural Computation"},{"key":"191_CR56","doi-asserted-by":"crossref","first-page":"1251","DOI":"10.1093\/cercor\/bhg092","volume":"13","author":"S Otani","year":"2003","unstructured":"Otani, S., Daniel, H., Roisin, M. P., & Crepel, F. (2003). Dopaminergic modulation of long-term synaptic plasticity in rat prefrontal neurons. Cerebral Cortex, 13, 1251\u20131256.","journal-title":"Cerebral Cortex"},{"key":"191_CR57","doi-asserted-by":"crossref","first-page":"6235","DOI":"10.1523\/JNEUROSCI.1478-05.2005","volume":"25","author":"WX Pan","year":"2005","unstructured":"Pan, W. X., Schmidt, R., Wickens, J. R., & Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. Journal of Neuroscience, 25, 6235\u20136242.","journal-title":"Journal of Neuroscience"},{"key":"191_CR58","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1162\/neco.2008.08-07-593","volume":"21","author":"W Potjans","year":"2009","unstructured":"Potjans, W., Morrison, A., & Diesmann, M. (2009). A spiking neural network model of an actor-critic learning agent. Neural Computation, 21, 301\u2013339.","journal-title":"Neural Computation"},{"key":"191_CR59","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1016\/S0893-6080(02)00045-X","volume":"15","author":"JN Reynolds","year":"2002","unstructured":"Reynolds, J. N., & Wickens, J. R. (2002). Dopamine-dependent plasticity of corticostriatal synapses. Neural Networks, 15, 507\u2013521.","journal-title":"Neural Networks"},{"key":"191_CR60","first-page":"1129","volume-title":"Neural information processing systems","author":"F Rivest","year":"2005","unstructured":"Rivest, F., Bengio, Y., & Kalaska, J. F. (2005). Brain inspired reinforcement learning. In L. K. Saul, Y. Weiss & L. Bottou (Eds.), Neural information processing systems (pp. 1129\u20131136). Cambridge: The MIT."},{"key":"191_CR61","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1007\/s00422-008-0265-6","volume":"99","author":"PD Roberts","year":"2008","unstructured":"Roberts, P. D., Santiago, R. A., & Lafferriere, G. (2008). An implementation of reinforcement learning based on spike timing dependent plasticity. Biological Cybernetics, 99, 517\u2013523.","journal-title":"Biological Cybernetics"},{"key":"191_CR62","doi-asserted-by":"crossref","first-page":"1615","DOI":"10.1038\/nn2013","volume":"10","author":"MR Roesch","year":"2007","unstructured":"Roesch, M. R., Calu, D. J., & Schoenbaum, G. (2007). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nature Neuroscience, 10, 1615\u20131624.","journal-title":"Nature Neuroscience"},{"key":"191_CR63","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1038\/20939","volume":"399","author":"R Romo","year":"1999","unstructured":"Romo, R., Brody, C. D., Hernandez, A., & Lemus, L. (1999). Neuronal correlates of parametric working memory in the prefrontal cortex. Nature, 399, 470\u2013473.","journal-title":"Nature"},{"key":"191_CR64","doi-asserted-by":"crossref","first-page":"7338","DOI":"10.1073\/pnas.0502455102","volume":"102","author":"NP Rougier","year":"2005","unstructured":"Rougier, N. P., Noelle, D. C., Braver, T. S., Cohen, J. D., & O\u2019Reilly, R. C. (2005). Prefrontal cortex and flexible cognitive control: rules without symbols. Proceedings of the National Academy of Sciences USA, 102, 7338\u20137343.","journal-title":"Proceedings of the National Academy of Sciences USA"},{"key":"191_CR65","doi-asserted-by":"crossref","first-page":"1337","DOI":"10.1126\/science.1115270","volume":"310","author":"K Samejima","year":"2005","unstructured":"Samejima, K., Ueda, Y., Doya, K., & Kimura, M. (2005). Representation of action-specific reward values in the striatum. Science, 310, 1337\u20131340.","journal-title":"Science"},{"key":"191_CR66","doi-asserted-by":"crossref","first-page":"4595","DOI":"10.1523\/JNEUROSCI.12-12-04595.1992","volume":"12","author":"W Schultz","year":"1992","unstructured":"Schultz, W., Apicella, P., Scarnati, E., & Ljungberg, T. (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. Journal of Neuroscience, 12, 4595\u20134610.","journal-title":"Journal of Neuroscience"},{"key":"191_CR67","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1523\/JNEUROSCI.13-03-00900.1993","volume":"13","author":"W Schultz","year":"1993","unstructured":"Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13, 900\u2013913.","journal-title":"Journal of Neuroscience"},{"key":"191_CR68","unstructured":"Schultz, W., Apicella, P., Romo, R., & Scarnati, E. (1995). Context-dependent activity in primate striatum reflecting past and future behavioral events. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 11\u201327). The MIT."},{"key":"191_CR69","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","volume":"275","author":"W Schultz","year":"1997","unstructured":"Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593\u20131599.","journal-title":"Science"},{"key":"191_CR70","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1007\/s002210050467","volume":"121","author":"RE Suri","year":"1998","unstructured":"Suri, R. E., & Schultz, W. (1998). Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Experimental Brain Research, 121, 350\u2013354.","journal-title":"Experimental Brain Research"},{"key":"191_CR71","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1016\/S0306-4522(98)00697-6","volume":"91","author":"RE Suri","year":"1999","unstructured":"Suri, R. E., & Schultz, W. (1999). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91, 871\u2013890.","journal-title":"Neuroscience"},{"key":"191_CR72","volume-title":"Reinforcement learning: An introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge: The MIT."},{"key":"191_CR73","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.bbr.2007.07.004","volume":"185","author":"G Thibaudeau","year":"2007","unstructured":"Thibaudeau, G., Potvin, O., Allen, K., Dore, F. Y., & Goulet, S. (2007). Dorsal, ventral, and complete excitotoxic lesions of the hippocampus in rats failed to impair appetitive trace conditioning. Behavioural Brain Research, 185, 9\u201320.","journal-title":"Behavioural Brain Research"},{"key":"191_CR74","first-page":"1689","volume-title":"Neural information processing systems","author":"MT Todd","year":"2009","unstructured":"Todd, M. T., Niv, Y., & Cohen, J. D. (2009). Learning to use working memory in partially observable environments through dopaminergic reinforcement. In D. Koller, D. Schuurmans, Y. Bengio & L. Bottou (Eds.), Neural information processing systems (pp. 1689\u20131696). Cambridge: The MIT."},{"key":"191_CR75","first-page":"187","volume-title":"Models of information processing in the basal ganglia","author":"J Wickens","year":"1995","unstructured":"Wickens, J., & Kotter, R. (1995). Cellular models of reinforcement. In J. C. Houk, J. L. Davis & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 187\u2013214). Cambridge: The MIT."}],"container-title":["Journal of Computational Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10827-009-0191-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10827-009-0191-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10827-009-0191-1","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,5,30]],"date-time":"2019-05-30T21:37:15Z","timestamp":1559252235000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10827-009-0191-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,10,22]]},"references-count":75,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2010,2]]}},"alternative-id":["191"],"URL":"https:\/\/doi.org\/10.1007\/s10827-009-0191-1","relation":{"has-review":[{"id-type":"doi","id":"10.3410\/f.1723956.1238054","asserted-by":"object"}]},"ISSN":["0929-5313","1573-6873"],"issn-type":[{"value":"0929-5313","type":"print"},{"value":"1573-6873","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,10,22]]}}}