{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T03:23:17Z","timestamp":1776309797803,"version":"3.50.1"},"reference-count":71,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2018,6,26]],"date-time":"2018-06-26T00:00:00Z","timestamp":1529971200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2018,6,26]],"date-time":"2018-06-26T00:00:00Z","timestamp":1529971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nat Commun"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.<\/jats:p>","DOI":"10.1038\/s41467-018-04840-2","type":"journal-article","created":{"date-parts":[[2018,6,20]],"date-time":"2018-06-20T08:09:12Z","timestamp":1529482152000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":114,"title":["An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals"],"prefix":"10.1038","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4748-8432","authenticated-orcid":false,"given":"Kiyohito","family":"Iigaya","sequence":"first","affiliation":[]},{"given":"Madalena S.","family":"Fonseca","sequence":"additional","affiliation":[]},{"given":"Masayoshi","family":"Murakami","sequence":"additional","affiliation":[]},{"given":"Zachary F.","family":"Mainen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3476-1839","authenticated-orcid":false,"given":"Peter","family":"Dayan","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2018,6,26]]},"reference":[{"key":"4840_CR1","first-page":"9","volume":"3","author":"RS Sutton","year":"1988","unstructured":"Sutton, R. S. Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9\u201344 (1988).","journal-title":"Mach. Learn."},{"key":"4840_CR2","doi-asserted-by":"publisher","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","volume":"16","author":"PR Montague","year":"1996","unstructured":"Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive hebbian learning. J. Neurosci. 16, 1936\u20131947 (1996).","journal-title":"J. Neurosci."},{"key":"4840_CR3","doi-asserted-by":"publisher","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","volume":"275","author":"W Schultz","year":"1997","unstructured":"Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593\u20131599 (1997).","journal-title":"Science"},{"key":"4840_CR4","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1126\/science.6294833","volume":"219","author":"RD Hawkins","year":"1983","unstructured":"Hawkins, R. D., Abrams, T. W., Carew, T. J. & Kandel, E. R. A cellular mechanism of classical conditioning in aplysia: activity-dependent amplification of presynaptic facilitation. Science 219, 400\u2013405 (1983).","journal-title":"Science"},{"key":"4840_CR5","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1111\/j.1460-9568.2010.07488.x","volume":"33","author":"JFM Vetencourt","year":"2011","unstructured":"Vetencourt, J. F. M., Tiraboschi, E., Spolidoro, M., Castr\u00e9n, E. & Maffei, L. Serotonin triggers a transient epigenetic mechanism that reinstates adult visual cortex plasticity in rats. Eur. J. Neurosci. 33, 49\u201357 (2011).","journal-title":"Eur. J. Neurosci."},{"key":"4840_CR6","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1523\/JNEUROSCI.3690-04.2005","volume":"25","author":"H Clarke","year":"2005","unstructured":"Clarke, H. et al. Prefrontal serotonin depletion affects reversal learning but not attentional set shifting. J. Neurosci. 25, 532\u2013538 (2005).","journal-title":"J. Neurosci."},{"key":"4840_CR7","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1093\/cercor\/bhn136","volume":"19","author":"S Walker","year":"2008","unstructured":"Walker, S., Robbins, T. & Roberts, A. Differential contributions of dopamine and serotonin to orbitofrontal cortex function in the marmoset. Cereb. Cortex 19, 889\u2013898 (2008).","journal-title":"Cereb. Cortex"},{"key":"4840_CR8","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1177\/026988119100500414","volume":"5","author":"JW Deakin","year":"1991","unstructured":"Deakin, J. W. & Graeff, F. G. 5-HT and mechanisms of defence. J. Psychopharmacol. 5, 305\u2013315 (1991).","journal-title":"J. Psychopharmacol."},{"key":"4840_CR9","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1017\/S0140525X00022871","volume":"9","author":"P Soubrie","year":"1986","unstructured":"Soubrie, P. Reconciling the role of central serotonin neurons in human and animal behavior. Behav. Brain Sci. 9, 319\u2013335 (1986).","journal-title":"Behav. Brain Sci."},{"key":"4840_CR10","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1016\/S0893-6080(02)00052-7","volume":"15","author":"ND Daw","year":"2002","unstructured":"Daw, N. D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603\u2013616 (2002).","journal-title":"Neural Netw."},{"key":"4840_CR11","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1038\/npp.2010.151","volume":"36","author":"YL Boureau","year":"2011","unstructured":"Boureau, Y.-L. & Dayan, P. Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36, 74 (2011).","journal-title":"Neuropsychopharmacology"},{"key":"4840_CR12","doi-asserted-by":"publisher","first-page":"e4","DOI":"10.1371\/journal.pcbi.0040004","volume":"4","author":"P Dayan","year":"2008","unstructured":"Dayan, P. & Huys, Q. J. Serotonin, inhibition, and negative mood. PLoS Comput. Biol. 4, e4 (2008).","journal-title":"PLoS Comput. Biol."},{"key":"4840_CR13","doi-asserted-by":"publisher","first-page":"11993","DOI":"10.1523\/JNEUROSCI.2513-09.2009","volume":"29","author":"MJ Crockett","year":"2009","unstructured":"Crockett, M. J., Clark, L. & Robbins, T. W. Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J. Neurosci. 29, 11993\u201311999 (2009).","journal-title":"J. Neurosci."},{"key":"4840_CR14","doi-asserted-by":"publisher","first-page":"5833","DOI":"10.1523\/JNEUROSCI.0053-12.2012","volume":"32","author":"B Seymour","year":"2012","unstructured":"Seymour, B., Daw, N. D., Roiser, J. P., Dayan, P. & Dolan, R. Serotonin selectively modulates reward value in human decision-making. J. Neurosci. 32, 5833\u20135842 (2012).","journal-title":"J. Neurosci."},{"key":"4840_CR15","doi-asserted-by":"publisher","first-page":"1360","DOI":"10.1016\/j.neuron.2014.02.010","volume":"81","author":"Z Liu","year":"2014","unstructured":"Liu, Z. et al. Dorsal raphe neurons signal reward through 5-ht and glutamate. Neuron 81, 1360\u20131374 (2014).","journal-title":"Neuron"},{"key":"4840_CR16","doi-asserted-by":"publisher","first-page":"2033","DOI":"10.1016\/j.cub.2014.07.041","volume":"24","author":"KW Miyazaki","year":"2014","unstructured":"Miyazaki, K. W. et al. Optogenetic activation of dorsal raphe serotonin neurons enhances patience for future rewards. Curr. Biol. 24, 2033\u20132040 (2014).","journal-title":"Curr. Biol."},{"key":"4840_CR17","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1016\/j.cub.2014.12.002","volume":"25","author":"MS Fonseca","year":"2015","unstructured":"Fonseca, M. S., Murakami, M. & Mainen, Z. F. Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing. Curr. Biol. 25, 306\u2013315 (2015).","journal-title":"Curr. Biol."},{"key":"4840_CR18","doi-asserted-by":"publisher","first-page":"e20975","DOI":"10.7554\/eLife.20975","volume":"6","author":"PA Correia","year":"2017","unstructured":"Correia, P. A. et al. Transient inhibition and long-term facilitation of locomotion by phasic optogenetic activation of serotonin neurons. eLife 6, e20975 (2017).","journal-title":"eLife"},{"key":"4840_CR19","doi-asserted-by":"publisher","first-page":"3111","DOI":"10.1016\/j.cub.2017.09.008","volume":"27","author":"S Xu","year":"2017","unstructured":"Xu, S., Das, G., Hueske, E. & Tonegawa, S. Dorsal raphe serotonergic neurons control intertemporal choice under trade-off. Curr. Biol. 27, 3111\u20133119 (2017).","journal-title":"Curr. Biol."},{"key":"4840_CR20","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1016\/S0896-6273(03)00761-X","volume":"40","author":"HS Seung","year":"2003","unstructured":"Seung, H. S. Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron 40, 1063\u20131073 (2003).","journal-title":"Neuron"},{"key":"4840_CR21","doi-asserted-by":"publisher","first-page":"3731","DOI":"10.1523\/JNEUROSCI.5159-05.2006","volume":"26","author":"A Soltani","year":"2006","unstructured":"Soltani, A. & Wang, X.-J. A biophysically based neural model of matching law behavior: melioration by stochastic synapses. J. Neurosci. 26, 3731\u20133744 (2006).","journal-title":"J. Neurosci."},{"key":"4840_CR22","doi-asserted-by":"publisher","first-page":"15224","DOI":"10.1073\/pnas.0505220103","volume":"103","author":"Y Loewenstein","year":"2006","unstructured":"Loewenstein, Y. & Seung, H. S. Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc. Natl Acad. Sci. USA 103, 15224\u201315229 (2006).","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"4840_CR23","doi-asserted-by":"publisher","first-page":"e18073","DOI":"10.7554\/eLife.18073","volume":"5","author":"K Iigaya","year":"2016","unstructured":"Iigaya, K. Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system. eLife 5, e18073 (2016).","journal-title":"eLife"},{"key":"4840_CR24","doi-asserted-by":"publisher","first-page":"291","DOI":"10.1007\/s002130051170","volume":"147","author":"PJ Fletcher","year":"1999","unstructured":"Fletcher, P. J., Korth, K. M. & Chambers, J. W. Selective destruction of brain serotonin neurons by 5, 7-dihydroxytryptamine increases responding for a conditioned reward. Psychopharmacology 147, 291\u2013299 (1999).","journal-title":"Psychopharmacology"},{"key":"4840_CR25","doi-asserted-by":"publisher","first-page":"1218","DOI":"10.1038\/81504","volume":"3","author":"P Dayan","year":"2000","unstructured":"Dayan, P., Kakade, S. & Montague, P. R. Learning and selective attention. Nat. Neurosci. 3, 1218 (2000).","journal-title":"Nat. Neurosci."},{"key":"4840_CR26","doi-asserted-by":"publisher","first-page":"1214","DOI":"10.1038\/nn1954","volume":"10","author":"TE Behrens","year":"2007","unstructured":"Behrens, T. E., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. Learning the value of information in an uncertain world. Nat. Neurosci. 10, 1214\u20131221 (2007).","journal-title":"Nat. Neurosci."},{"key":"4840_CR27","doi-asserted-by":"publisher","first-page":"12366","DOI":"10.1523\/JNEUROSCI.0822-10.2010","volume":"30","author":"MR Nassar","year":"2010","unstructured":"Nassar, M. R., Wilson, R. C., Heasly, B. & Gold, J. I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366\u201312378 (2010).","journal-title":"J. Neurosci."},{"key":"4840_CR28","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1016\/j.neuron.2005.04.026","volume":"46","author":"AJ Yu","year":"2005","unstructured":"Yu, A. J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681\u2013692 (2005).","journal-title":"Neuron"},{"key":"4840_CR29","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1901\/jeab.2005.23-05","volume":"84","author":"GS Corrado","year":"2005","unstructured":"Corrado, G. S., Sugrue, L. P., Seung, H. S. & Newsome, W. T. Linear-nonlinear-poisson models of primate choice dynamics. J. Exp. Anal. Behav. 84, 581\u2013617 (2005).","journal-title":"J. Exp. Anal. Behav."},{"key":"4840_CR30","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1038\/nn1901","volume":"10","author":"KP Kording","year":"2007","unstructured":"Kording, K. P., Tenenbaum, J. B. & Shadmehr, R. The dynamics of memory as a consequence of optimal adaptation to a changing body. Nat. Neurosci. 10, 779\u2013786 (2007).","journal-title":"Nat. Neurosci."},{"key":"4840_CR31","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1016\/j.neuron.2007.03.017","volume":"54","author":"S Fusi","year":"2007","unstructured":"Fusi, S., Asaad, W. F., Miller, E. K. & Wang, X. J. A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales. Neuron 54, 319\u2013333 (2007).","journal-title":"Neuron"},{"key":"4840_CR32","doi-asserted-by":"publisher","unstructured":"Iigaya, K. et al. Learning fast and slow: deviations from the matching law can reflect an optimal strategy under uncertainty. Preprint at bioRxiv\u00a0https:\/\/doi.org\/10.1101\/141309 (2017).","DOI":"10.1101\/141309"},{"key":"4840_CR33","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","volume":"8","author":"ND Daw","year":"2005","unstructured":"Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704\u20131711 (2005).","journal-title":"Nat. Neurosci."},{"key":"4840_CR34","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.neuroimage.2012.04.024","volume":"62","author":"M Guitart-Masip","year":"2012","unstructured":"Guitart-Masip, M. et al. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage 62, 154\u2013166 (2012).","journal-title":"Neuroimage"},{"key":"4840_CR35","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.1111\/j.1460-9568.2011.07980.x","volume":"35","author":"AG Collins","year":"2012","unstructured":"Collins, A. G. & Frank, M. J. How much of reinforcement learning is working memory, not reinforcement learning? a behavioral, computational, and neurogenetic analysis. Eur. J. Neurosci. 35, 1024\u20131035 (2012).","journal-title":"Eur. J. Neurosci."},{"key":"4840_CR36","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","volume":"81","author":"SW Lee","year":"2014","unstructured":"Lee, S. W., Shimojo, S. & ODoherty, J. P. Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687\u2013699 (2014).","journal-title":"Neuron"},{"key":"4840_CR37","doi-asserted-by":"publisher","first-page":"9817","DOI":"10.1523\/JNEUROSCI.2707-07.2007","volume":"27","author":"S Gong","year":"2007","unstructured":"Gong, S. et al. Targeting cre recombinase to specific neuron populations with bacterial artificial chromosome constructs. J. Neurosci. 27, 9817\u20139823 (2007).","journal-title":"J. Neurosci."},{"key":"4840_CR38","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1901\/jeab.2005.110-04","volume":"84","author":"B Lau","year":"2005","unstructured":"Lau, B. & Glimcher, P. W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 84, 555\u2013579 (2005).","journal-title":"J. Exp. Anal. Behav."},{"key":"4840_CR39","doi-asserted-by":"publisher","first-page":"1782","DOI":"10.1126\/science.1094765","volume":"304","author":"LP Sugrue","year":"2004","unstructured":"Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Matching behavior and the representation of value in the parietal cortex. Science 304, 1782\u20131787 (2004).","journal-title":"Science"},{"key":"4840_CR40","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1162\/neco.2008.20.1.227","volume":"20","author":"Y Sakai","year":"2008","unstructured":"Sakai, Y. & Fukai, T. The actor-critic learning is behind the matching law: matching versus optimal behaviors. Neural Comput. 20, 227\u2013251 (2008).","journal-title":"Neural Comput."},{"key":"4840_CR41","doi-asserted-by":"publisher","first-page":"2755","DOI":"10.1162\/neco.2009.09-08-854","volume":"21","author":"Y Loewenstein","year":"2009","unstructured":"Loewenstein, Y., Prelec, D. & Seung, H. S. Operant matching as a Nash equilibrium of an intertemporal game. Neural Comput. 21, 2755\u20132773 (2009).","journal-title":"Neural Comput."},{"key":"4840_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/NECO_a_00522","volume":"25","author":"K Iigaya","year":"2013","unstructured":"Iigaya, K. & Fusi, S. Dynamical regimes in neural network models of matching behavior. Neural Comput. 25, 1\u201320 (2013).","journal-title":"Neural Comput."},{"key":"4840_CR43","doi-asserted-by":"crossref","unstructured":"Smethells, J. R. & Reilly, M. P. Intertrial interval duration and impulsive choice.\u00a0J. Exp. Anal. Behav.\u00a0103, 153\u2013165 (2015).","DOI":"10.1002\/jeab.131"},{"key":"4840_CR44","doi-asserted-by":"publisher","first-page":"e19551","DOI":"10.1371\/journal.pone.0019551","volume":"6","author":"P Ashourian","year":"2011","unstructured":"Ashourian, P. & Loewenstein, Y. Bayesian inference underlies the contraction bias in delayed comparison tasks. PLoS ONE 6, e19551 (2011).","journal-title":"PLoS ONE"},{"key":"4840_CR45","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1038\/nn1209","volume":"7","author":"DJ Barraclough","year":"2004","unstructured":"Barraclough, D. J., Conroy, M. L. & Lee, D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 7, 404 (2004).","journal-title":"Nat. Neurosci."},{"key":"4840_CR46","doi-asserted-by":"publisher","first-page":"364","DOI":"10.3758\/s13423-012-0324-9","volume":"20","author":"DA Worthy","year":"2013","unstructured":"Worthy, D. A., Hawthorne, M. J. & Otto, A. R. Heterogeneity of strategy use in the iowa gambling task: a comparison of win-stay\/lose-shift and reinforcement learning models. Psychon. Bull. Rev. 20, 364\u2013371 (2013).","journal-title":"Psychon. Bull. Rev."},{"key":"4840_CR47","doi-asserted-by":"publisher","first-page":"1165","DOI":"10.1016\/j.cell.2015.10.063","volume":"163","author":"HF Kim","year":"2015","unstructured":"Kim, H. F., Ghazizadeh, A. & Hikosaka, O. Dopamine neurons encoding long-term memory of object value for habitual behavior. Cell 163, 1165\u20131175 (2015).","journal-title":"Cell"},{"key":"4840_CR48","doi-asserted-by":"publisher","first-page":"e1002028","DOI":"10.1371\/journal.pcbi.1002028","volume":"7","author":"QJ Huys","year":"2011","unstructured":"Huys, Q. J. et al. Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput. Biol. 7, e1002028 (2011).","journal-title":"PLoS Comput. Biol."},{"key":"4840_CR49","doi-asserted-by":"publisher","first-page":"e13747","DOI":"10.7554\/eLife.13747","volume":"5","author":"K Iigaya","year":"2016","unstructured":"Iigaya, K., Story, G. W., Kurth-Nelson, Z., Dolan, R. J. & Dayan, P. The modulation of savouring by prediction error and its effects on choice. eLife 5, e13747 (2016).","journal-title":"eLife"},{"key":"4840_CR50","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"1998","unstructured":"Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction 1 (MIT Press, Cambridge, 1998)."},{"key":"4840_CR51","doi-asserted-by":"publisher","first-page":"e06346","DOI":"10.7554\/eLife.06346","volume":"4","author":"JY Cohen","year":"2015","unstructured":"Cohen, J. Y., Amoroso, M. W. & Uchida, N. Serotonergic neurons signal reward and punishment on multiple timescales. eLife 4, e06346 (2015).","journal-title":"eLife"},{"key":"4840_CR52","doi-asserted-by":"publisher","first-page":"820","DOI":"10.1016\/S0959-4388(97)80141-9","volume":"7","author":"BL Jacobs","year":"1997","unstructured":"Jacobs, B. L. & Fornal, C. A. Serotonin and motor activity. Curr. Opin. Neurobiol. 7, 820\u2013825 (1997).","journal-title":"Curr. Opin. Neurobiol."},{"key":"4840_CR53","first-page":"149","volume":"2","author":"J Deakin","year":"1983","unstructured":"Deakin, J. Roles of serotonergic systems in escape, avoidance and other behaviours. Theory Psychopharmacol. 2, 149\u2013193 (1983).","journal-title":"Theory Psychopharmacol."},{"key":"4840_CR54","doi-asserted-by":"publisher","first-page":"1209","DOI":"10.1016\/j.neuroscience.2010.09.058","volume":"171","author":"J Schweimer","year":"2010","unstructured":"Schweimer, J. & Ungless, M. Phasic responses in dorsal raphe serotonin neurons to noxious stimuli. Neuroscience 171, 1209\u20131215 (2010).","journal-title":"Neuroscience"},{"key":"4840_CR55","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.bandc.2009.03.009","volume":"71","author":"M Tops","year":"2009","unstructured":"Tops, M., Russo, S., Boksem, M. A. & Tucker, D. M. Serotonin: modulator of a drive to withdraw. Brain Cogn. 71, 427\u2013436 (2009).","journal-title":"Brain Cogn."},{"key":"4840_CR56","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1016\/S0893-6080(02)00044-8","volume":"15","author":"K Doya","year":"2002","unstructured":"Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495\u2013506 (2002).","journal-title":"Neural Netw."},{"key":"4840_CR57","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1016\/j.nlm.2016.08.008","volume":"135","author":"M Luo","year":"2016","unstructured":"Luo, M., Li, Y. & Zhong, W. Do dorsal raphe 5-ht neurons encode beneficialness? Neurobiol. Learn. Mem. 135, 40\u201349 (2016).","journal-title":"Neurobiol. Learn. Mem."},{"key":"4840_CR58","doi-asserted-by":"publisher","first-page":"698","DOI":"10.1523\/JNEUROSCI.2489-13.2014","volume":"34","author":"AS Hart","year":"2014","unstructured":"Hart, A. S., Rutledge, R. B., Glimcher, P. W. & Phillips, P. E. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J. Neurosci. 34, 698\u2013704 (2014).","journal-title":"J. Neurosci."},{"key":"4840_CR59","doi-asserted-by":"publisher","first-page":"e23291","DOI":"10.1371\/journal.pone.0023291","volume":"6","author":"KT Kishida","year":"2011","unstructured":"Kishida, K. T. et al. Sub-second dopamine detection in human striatum. PLoS ONE 6, e23291 (2011).","journal-title":"PLoS ONE"},{"key":"4840_CR60","first-page":"354","volume":"27","author":"CR Gallistel","year":"2001","unstructured":"Gallistel, C. R., Mark, T. A., King, A. P. & Latham, P. E. The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. J. Exp. Psychol. 27, 354\u2013372 (2001).","journal-title":"J. Exp. Psychol."},{"key":"4840_CR61","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1196\/annals.1390.005","volume":"1104","author":"K Preuschoff","year":"2007","unstructured":"Preuschoff, K. & Bossaerts, P. Adding prediction risk to the theory of reward learning. Ann. N. Y. Acad. Sci. 1104, 135\u2013146 (2007).","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"4840_CR62","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1146\/annurev.neuro.28.061604.135709","volume":"28","author":"G Aston-Jones","year":"2005","unstructured":"Aston-Jones, G. & Cohen, J. D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403\u2013450 (2005).","journal-title":"Annu. Rev. Neurosci."},{"key":"4840_CR63","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1016\/j.pneurobio.2016.03.004","volume":"151","author":"P De Deurwaerd\u00e8re","year":"2017","unstructured":"De Deurwaerd\u00e8re, P. & Di Giovanni, G. Serotonergic modulation of the activity of mesencephalic dopaminergic systems: therapeutic implications. Prog. Neurobiol. 151, 175\u2013236 (2017).","journal-title":"Prog. Neurobiol."},{"key":"4840_CR64","doi-asserted-by":"publisher","first-page":"997","DOI":"10.1038\/nn.4573","volume":"20","author":"AM Bornstein","year":"2017","unstructured":"Bornstein, A. M. & Norman, K. A. Reinstated episodic context guides sampling-based decisions for reward. Nat. Neurosci. 20, 997\u20131003 (2017).","journal-title":"Nat. Neurosci."},{"key":"4840_CR65","doi-asserted-by":"crossref","unstructured":"Lengyel, M. & Dayan, P. Uncertainty, phase and oscillatory hippocampal recall. Adv. Neural Inf. Process. Syst. 19, 833\u2013840 (2007).","DOI":"10.7551\/mitpress\/7503.003.0109"},{"key":"4840_CR66","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.cogbrainres.2004.07.007","volume":"22","author":"D Lee","year":"2004","unstructured":"Lee, D., Conroy, M. L., McGreevy, B. P. & Barraclough, D. J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res. Cogn. Brain Res. 22, 45\u201358 (2004).","journal-title":"Brain Res. Cogn. Brain Res."},{"key":"4840_CR67","unstructured":"Herrnstein, R. J., Laibson, D. & Rachlin, H. The Matching Law: Papers In Psychology and Economics (Harvard University Press, Cambridge, MA. 1997)."},{"key":"4840_CR68","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1016\/j.cpr.2005.07.003","volume":"26","author":"AC Butler","year":"2006","unstructured":"Butler, A. C., Chapman, J. E., Forman, E. M. & Beck, A. T. The empirical status of cognitive-behavioral therapy: a review of meta-analyses. Clin. Psychol. Rev. 26, 17\u201331 (2006).","journal-title":"Clin. Psychol. Rev."},{"key":"4840_CR69","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1901\/jeab.1961.4-267","volume":"4","author":"RJ Herrnstein","year":"1961","unstructured":"Herrnstein, R. J. Relative and absolute strength of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 4, 267\u2013272 (1961).","journal-title":"J. Exp. Anal. Behav."},{"key":"4840_CR70","volume-title":"Learning from Delayed Rewards","author":"CJCH Watkins","year":"1989","unstructured":"Watkins, C. J. C. H. Learning from Delayed Rewards (King\u2019s College, Cambridge, 1989)."},{"key":"4840_CR71","doi-asserted-by":"publisher","first-page":"e0165840","DOI":"10.1371\/journal.pone.0165840","volume":"11","author":"K Iigaya","year":"2016","unstructured":"Iigaya, K. et al. Cognitive bias in ambiguity judgements: using computational models to dissect the effects of mild mood manipulation in humans. PLoS ONE 11, e0165840 (2016).","journal-title":"PLoS ONE"}],"container-title":["Nature Communications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41467-018-04840-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41467-018-04840-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41467-018-04840-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,3]],"date-time":"2023-09-03T03:58:09Z","timestamp":1693713489000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41467-018-04840-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,26]]},"references-count":71,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2018,12]]}},"alternative-id":["4840"],"URL":"https:\/\/doi.org\/10.1038\/s41467-018-04840-2","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/215400","asserted-by":"object"}],"has-review":[{"id-type":"doi","id":"10.3410\/f.733505841.793549124","asserted-by":"object"}]},"ISSN":["2041-1723"],"issn-type":[{"value":"2041-1723","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,26]]},"assertion":[{"value":"4 December 2017","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 May 2018","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 June 2018","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2477"}}