{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T06:18:07Z","timestamp":1772173087468,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1010340","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T00:00:00Z","timestamp":1659571200000}}],"reference-count":142,"publisher":"Public Library of Science (PLoS)","issue":"7","license":[{"start":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T00:00:00Z","timestamp":1658707200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"James S. McDonnell Foundation 21st Century Science Initiative Understanding Dynamic"},{"name":"Multi-scale Systems - Postdoctoral Fellowship Award"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Studying the brain circuits that control behavior is challenging, since in addition to their structural complexity there are continuous feedback interactions between actions and sensed inputs from the environment. It is therefore important to identify mathematical principles that can be used to develop testable hypotheses. In this study, we use ideas and concepts from systems biology to study the dopamine system, which controls learning, motivation, and movement. Using data from neuronal recordings in behavioral experiments, we developed a mathematical model for dopamine responses and the effect of dopamine on movement. We show that the dopamine system shares core functional analogies with bacterial chemotaxis. Just as chemotaxis robustly climbs chemical attractant gradients, the dopamine circuit performs \u2018reward-taxis\u2019 where the attractant is the expected value of reward. The reward-taxis mechanism provides a simple explanation for scale-invariant dopaminergic responses and for matching in free operant settings, and makes testable quantitative predictions. We propose that reward-taxis is a simple and robust navigation strategy that complements other, more goal-directed navigation mechanisms.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010340","type":"journal-article","created":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T13:38:55Z","timestamp":1658756335000},"page":"e1010340","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":9,"title":["The dopamine circuit as a reward-taxis navigation system"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9426-5362","authenticated-orcid":true,"given":"Omer","family":"Karin","sequence":"first","affiliation":[]},{"given":"Uri","family":"Alon","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,7,25]]},"reference":[{"key":"pcbi.1010340.ref001","author":"AG Barto","year":"1995","journal-title":"Adaptive critics and the basal ganglia"},{"key":"pcbi.1010340.ref002","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1038\/377725a0","article-title":"Bee foraging in uncertain environments using predictive hebbian learning","volume":"377","author":"PR Montague","year":"1995","journal-title":"Nature"},{"key":"pcbi.1010340.ref003","volume-title":"Models of information processing in the basal ganglia","author":"JC Houk","year":"1995"},{"key":"pcbi.1010340.ref004","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive Hebbian learning","volume":"16","author":"PR Montague","year":"1996","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref005","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"key":"pcbi.1010340.ref006","doi-asserted-by":"crossref","first-page":"966","DOI":"10.1038\/nn.3413","article-title":"A Causal Link Between Prediction Errors, Dopamine Neurons and Learning","volume":"16","author":"EE Steinberg","year":"2013","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref007","doi-asserted-by":"crossref","DOI":"10.1109\/TNN.1998.712192","volume-title":"Introduction to reinforcement learning","author":"RS Sutton","year":"1998"},{"key":"pcbi.1010340.ref008","doi-asserted-by":"crossref","first-page":"15647","DOI":"10.1073\/pnas.1014269108","article-title":"Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis","volume":"108","author":"PW Glimcher","year":"2011","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref009","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1038\/nature12475","article-title":"Prolonged dopamine signalling in striatum signals proximity and value of distant rewards","volume":"500","author":"MW Howe","year":"2013","journal-title":"nature"},{"key":"pcbi.1010340.ref010","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1038\/nn.4173","article-title":"Mesolimbic dopamine signals the value of work","volume":"19","author":"AA Hamid","year":"2016","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref011","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1038\/s41586-019-1235-y","article-title":"Dissociable dopamine dynamics for learning and motivation","volume":"570","author":"A Mohebi","year":"2019","journal-title":"Nature"},{"key":"pcbi.1010340.ref012","article-title":"A Unified Framework for Dopamine Signals across Timescales","author":"HR Kim","year":"2020","journal-title":"Cell"},{"key":"pcbi.1010340.ref013","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1007\/s00213-006-0502-4","article-title":"Tonic dopamine: opportunity costs and the control of response vigor","volume":"191","author":"Y Niv","year":"2007","journal-title":"Psychopharmacology (Berl)"},{"key":"pcbi.1010340.ref014","doi-asserted-by":"crossref","first-page":"7105","DOI":"10.1523\/JNEUROSCI.0264-07.2007","article-title":"Why don\u2019t we move faster? Parkinson\u2019s disease, movement vigor, and implicit motivation","volume":"27","author":"P Mazzoni","year":"2007","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref015","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1007\/s00213-006-0578-x","article-title":"The debate over dopamine\u2019s role in reward: the case for incentive salience","volume":"191","author":"KC Berridge","year":"2007","journal-title":"Psychopharmacology (Berl)"},{"key":"pcbi.1010340.ref016","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.conb.2016.02.005","article-title":"The basal ganglia: from motor commands to the control of vigor","volume":"37","author":"JT Dudman","year":"2016","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1010340.ref017","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1038\/nature25457","article-title":"Dopamine neuron activity before action initiation gates and invigorates future movements","volume":"554","author":"JA da Silva","year":"2018","journal-title":"Nature"},{"key":"pcbi.1010340.ref018","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/12940.001.0001","volume-title":"Vigor: neuroeconomics of movement control","author":"R Shadmehr","year":"2020"},{"key":"pcbi.1010340.ref019","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.neuroimage.2018.11.021","article-title":"The role of dopamine in the brain-lessons learned from Parkinson\u2019s disease","volume":"190","author":"D Meder","year":"2019","journal-title":"Neuroimage"},{"key":"pcbi.1010340.ref020","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1038\/s41593-018-0152-y","article-title":"What does dopamine mean?","volume":"21","author":"JD Berke","year":"2018","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref021","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: a unified brain theory?","volume":"11","author":"K. Friston","year":"2010","journal-title":"Nat Rev Neurosci."},{"key":"pcbi.1010340.ref022","doi-asserted-by":"crossref","first-page":"e53262","DOI":"10.7554\/eLife.53262","article-title":"Dopamine role in learning and action inference","volume":"9","author":"R. Bogacz","year":"2020","journal-title":"Elife"},{"key":"pcbi.1010340.ref023","first-page":"1019","article-title":"How fast to work: Response vigor, motivation and tonic dopamine","volume":"18","author":"Y Niv","year":"2005","journal-title":"Adv Neural Inf Process Syst"},{"key":"pcbi.1010340.ref024","doi-asserted-by":"crossref","first-page":"E10476","DOI":"10.1073\/pnas.1812979115","article-title":"Control of movement vigor and decision making during foraging","volume":"115","author":"T Yoon","year":"2018","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref025","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1038\/nature04766","article-title":"Cortical substrates for exploratory decisions in humans","volume":"441","author":"ND Daw","year":"2006","journal-title":"Nature"},{"key":"pcbi.1010340.ref026","doi-asserted-by":"crossref","first-page":"1642","DOI":"10.1126\/science.1105370","article-title":"Adaptive Coding of Reward Value by Dopamine Neurons","volume":"307","author":"PN Tobler","year":"2005","journal-title":"Science"},{"key":"pcbi.1010340.ref027","doi-asserted-by":"crossref","DOI":"10.1201\/9780429283321","volume-title":"An introduction to systems biology: design principles of biological circuits.","author":"U. Alon","year":"2019"},{"key":"pcbi.1010340.ref028","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1152\/jn.1998.80.1.1","article-title":"Predictive reward signal of dopamine neurons","volume":"80","author":"W. Schultz","year":"1998","journal-title":"J Neurophysiol"},{"key":"pcbi.1010340.ref029","doi-asserted-by":"crossref","first-page":"4894","DOI":"10.1073\/pnas.0811507106","article-title":"Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli","volume":"106","author":"F Brischoux","year":"2009","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref030","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1038\/nn.4239","article-title":"Dopamine neurons share common response function for reward prediction error","volume":"19","author":"N Eshel","year":"2016","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref031","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1038\/nn.4287","article-title":"Reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target","volume":"19","author":"NF Parker","year":"2016","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref032","doi-asserted-by":"crossref","first-page":"e42992","DOI":"10.7554\/eLife.42992","article-title":"Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons","volume":"8","author":"RS Lee","year":"2019","journal-title":"eLife"},{"key":"pcbi.1010340.ref033","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1038\/s41586-019-1261-9","article-title":"Specialized coding of sensory, motor and cognitive variables in VTA dopamine neurons","volume":"570","author":"B Engelhard","year":"2019","journal-title":"Nature"},{"key":"pcbi.1010340.ref034","doi-asserted-by":"crossref","first-page":"928","DOI":"10.1523\/JNEUROSCI.15-01-00928.1995","article-title":"Eye movements in monkeys with local dopamine depletion in the caudate nucleus. II. Deficits in voluntary saccades","volume":"15","author":"A Kori","year":"1995","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref035","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1038\/nature08028","article-title":"Two types of dopamine neuron distinctly convey positive and negative motivational signals","volume":"459","author":"M Matsumoto","year":"2009","journal-title":"Nature"},{"key":"pcbi.1010340.ref036","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1162\/NECO_a_00559","article-title":"Dopamine ramps are a consequence of reward prediction errors","volume":"26","author":"SJ Gershman","year":"2014","journal-title":"Neural Comput"},{"key":"pcbi.1010340.ref037","first-page":"283","volume-title":"Neuroeconomics (Second Edition)","author":"ND Daw","year":"2014"},{"key":"pcbi.1010340.ref038","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1038\/nature14855","article-title":"Arithmetic and local circuitry underlying dopamine prediction errors","volume":"525","author":"N Eshel","year":"2015","journal-title":"Nature"},{"key":"pcbi.1010340.ref039","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/S1364-6613(03)00055-X","article-title":"The neural basis of the Weber\u2013Fechner law: a logarithmic mental number line","volume":"7","author":"S. Dehaene","year":"2003","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1010340.ref040","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/S0896-6273(02)01144-3","article-title":"Coding of cognitive magnitude: Compressed scaling of numerical information in the primate prefrontal cortex","volume":"37","author":"A Nieder","year":"2003","journal-title":"Neuron"},{"key":"pcbi.1010340.ref041","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0167-2789(02)00734-0","article-title":"On the foundations of vision modeling: I. Weber\u2019s law and Weberized TV restoration","volume":"175","author":"J. Shen","year":"2003","journal-title":"Phys Nonlinear Phenom."},{"key":"pcbi.1010340.ref042","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1126\/science.1156540","article-title":"Log or linear? Distinct intuitions of the number scale in Western and Amazonian indigene cultures","volume":"320","author":"S Dehaene","year":"2008","journal-title":"Science"},{"key":"pcbi.1010340.ref043","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1146\/annurev.neuro.051508.135550","article-title":"Representation of number in the brain","volume":"32","author":"A Nieder","year":"2009","journal-title":"Annu Rev Neurosci"},{"key":"pcbi.1010340.ref044","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1242\/jeb.146.1.39","article-title":"The role of sensory adaptation in the retina","volume":"146","author":"SB Laughlin","year":"1989","journal-title":"J Exp Biol"},{"key":"pcbi.1010340.ref045","article-title":"Specimen theoriae novae de mensura sortis","author":"D. Bernoulli","year":"1968","journal-title":"Gregg"},{"key":"pcbi.1010340.ref046","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/B978-0-12-445850-5.50007-6","volume-title":"Financial Dec Making Under Uncertainty","author":"M. Rubinstein","year":"1977"},{"key":"pcbi.1010340.ref047","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1038\/nrn.2016.165","article-title":"Ventral tegmental area: cellular heterogeneity, connectivity and behaviour","volume":"18","author":"M Morales","year":"2017","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1010340.ref048","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1038\/s41583-019-0189-2","article-title":"Striatal circuits for reward learning and decision-making","volume":"20","author":"J Cox","year":"2019","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1010340.ref049","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1038\/nature10754","article-title":"Neuron-type-specific signals for reward and punishment in the ventral tegmental area","volume":"482","author":"JY Cohen","year":"2012","journal-title":"nature"},{"key":"pcbi.1010340.ref050","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1016\/j.cell.2009.06.013","article-title":"Defining network topologies that can achieve biochemical adaptation","volume":"138","author":"W Ma","year":"2009","journal-title":"Cell"},{"key":"pcbi.1010340.ref051","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/j.cels.2016.12.009","article-title":"Optimal regulatory circuit topologies for fold-change detection","volume":"4","author":"M Adler","year":"2017","journal-title":"Cell Syst"},{"key":"pcbi.1010340.ref052","doi-asserted-by":"crossref","first-page":"13329","DOI":"10.1073\/pnas.0405084101","article-title":"Firing properties of dopamine neurons in freely moving dopamine-deficient mice: Effects of dopamine receptor activation and anesthesia","volume":"101","author":"S Robinson","year":"2004","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref053","doi-asserted-by":"crossref","first-page":"3311","DOI":"10.1162\/neco_a_01023","article-title":"Dopamine, inference, and uncertainty","volume":"29","author":"SJ Gershman","year":"2017","journal-title":"Neural Comput"},{"key":"pcbi.1010340.ref054","doi-asserted-by":"crossref","first-page":"15995","DOI":"10.1073\/pnas.1002352107","article-title":"Fold-change detection and scalar symmetry of sensory input fields","volume":"107","author":"O Shoval","year":"2010","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref055","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.coisb.2017.12.005","article-title":"Fold-change detection in biological systems","volume":"8","author":"M Adler","year":"2018","journal-title":"Curr Opin Syst Biol"},{"key":"pcbi.1010340.ref056","doi-asserted-by":"crossref","first-page":"14855","DOI":"10.1073\/pnas.0807569105","article-title":"Modeling the chemotactic response of Escherichia coli to time-varying stimuli","volume":"105","author":"Y Tu","year":"2008","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref057","doi-asserted-by":"crossref","first-page":"e1003781","DOI":"10.1371\/journal.pcbi.1003781","article-title":"Logarithmic and power law input-output relations in sensory systems with fold-change detection","volume":"10","author":"M Adler","year":"2014","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010340.ref058","doi-asserted-by":"crossref","first-page":"6676","DOI":"10.1109\/ACC.2016.7526722","volume-title":"2016 American Control Conference (ACC)","author":"M Lang","year":"2016"},{"key":"pcbi.1010340.ref059","doi-asserted-by":"crossref","first-page":"1495","DOI":"10.1038\/npp.2013.48","article-title":"Dopamine modulates reward-related vigor","volume":"38","author":"U Beierholm","year":"2013","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1010340.ref060","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1016\/j.cell.2015.08.014","article-title":"Dopamine is required for the neural representation and control of movement vigor","volume":"162","author":"B Panigrahi","year":"2015","journal-title":"Cell"},{"key":"pcbi.1010340.ref061","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1021\/acschemneuro.6b00014","article-title":"Behavioral Analysis of Dopaminergic Activation in Zebrafish and Rats Reveals Similar Phenotypes","volume":"7","author":"F Ek","year":"2016","journal-title":"ACS Chem Neurosci."},{"key":"pcbi.1010340.ref062","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1901\/jeab.1970.13-243","article-title":"On the law of effect 1","volume":"13","author":"RJ Herrnstein","year":"1970","journal-title":"J Exp Anal Behav."},{"key":"pcbi.1010340.ref063","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1901\/jeab.1974.22-231","article-title":"On two types of deviation from the matching law: bias and undermatching 1","volume":"22","author":"WM Baum","year":"1974","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref064","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1901\/jeab.1981.36-387","article-title":"Optimization and the matching law as accounts of instrumental behavior","volume":"36","author":"WM Baum","year":"1981","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref065","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1901\/jeab.1999.71-355","article-title":"Choice, contingency discrimination, and foraging theory","volume":"71","author":"WM Baum","year":"1999","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref066","doi-asserted-by":"crossref","first-page":"1782","DOI":"10.1126\/science.1094765","article-title":"Matching behavior and the representation of value in the parietal cortex","volume":"304","author":"LP Sugrue","year":"2004","journal-title":"science"},{"key":"pcbi.1010340.ref067","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1097\/00008877-200411000-00001","article-title":"Herrnstein\u2019s hyperbolic matching equation and behavioral pharmacology: Review and critique","volume":"15","author":"J Dallery","year":"2004","journal-title":"Behav Pharmacol"},{"key":"pcbi.1010340.ref068","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1901\/jeab.2005.110-04","article-title":"Dynamic response-by-response models of matching behavior in rhesus monkeys","volume":"84","author":"B Lau","year":"2005","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref069","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1037\/a0029924","article-title":"On the theoretical and empirical status of the matching law and matching theory","volume":"139","author":"JJ McDowell","year":"2013","journal-title":"Psychol Bull"},{"key":"pcbi.1010340.ref070","article-title":"Matching Behaviours and Rewards","author":"AI Houston","year":"2021","journal-title":"Trends Cogn Sci."},{"key":"pcbi.1010340.ref071","volume-title":"The matching law: a research review","author":"M Davison","year":"1988"},{"key":"pcbi.1010340.ref072","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1901\/jeab.1969.12-861","article-title":"Choice as time allocation 1","volume":"12","author":"WM Baum","year":"1969","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref073","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1901\/jeab.1961.4-267","article-title":"Relative and absolute strength of response as a function of frequency of reinforcement","volume":"4","author":"RJ Herrnstein","year":"1961","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref074","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1901\/jeab.1979.32-269","article-title":"Matching, undermatching, and overmatching in studies of choice","volume":"32","author":"BM William","year":"1979","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref075","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1901\/jeab.1991.55-47","article-title":"Choice, changeover, and travel: A quantitative model","volume":"55","author":"M Davison","year":"1991","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref076","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1126\/science.185.4145.78","article-title":"Choice in free-ranging wild pigeons","volume":"185","author":"WM Baum","year":"1974","journal-title":"Science"},{"key":"pcbi.1010340.ref077","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1901\/jeab.1986.45-15","article-title":"THE MATCHING LAW APPLIES TO WAGTAILS\u2019FORAGING IN THE WILD","volume":"45","author":"A. Houston","year":"1986","journal-title":"J Exp Anal Behav."},{"key":"pcbi.1010340.ref078","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1901\/jeab.1981.35-367","article-title":"How to maximize reward rate on two variable-interval paradigms","volume":"35","author":"AI Houston","year":"1981","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1010340.ref079","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1901\/jeab.1979.31-41","article-title":"A MARKOV MODEL DESCRIPTION OF CHANGEOVER PROBABILITIES ON CONCURRENT VARIABLE-INTERVAL SCHEDULES 1","volume":"31","author":"GM Heyman","year":"1979","journal-title":"J Exp Anal Behav."},{"key":"pcbi.1010340.ref080","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1257\/jep.5.3.137","article-title":"Melioration: A theory of distributed choice","volume":"5","author":"RJ Herrnstein","year":"1991","journal-title":"J Econ Perspect"},{"key":"pcbi.1010340.ref081","doi-asserted-by":"crossref","first-page":"3731","DOI":"10.1523\/JNEUROSCI.5159-05.2006","article-title":"A biophysically based neural model of matching law behavior: melioration by stochastic synapses","volume":"26","author":"A Soltani","year":"2006","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref082","doi-asserted-by":"crossref","first-page":"15224","DOI":"10.1073\/pnas.0505220103","article-title":"Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity","volume":"103","author":"Y Loewenstein","year":"2006","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref083","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.brainres.2009.07.017","article-title":"Explicit melioration by a neural diffusion model","volume":"1299","author":"P Simen","year":"2009","journal-title":"Brain Res"},{"key":"pcbi.1010340.ref084","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/239500a0","article-title":"Chemotaxis in Escherichia coli analysed by three-dimensional tracking","volume":"239","author":"HC Berg","year":"1972","journal-title":"Nature"},{"key":"pcbi.1010340.ref085","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.ceb.2011.11.008","article-title":"Responding to chemical gradients: bacterial chemotaxis","volume":"24","author":"V Sourjik","year":"2012","journal-title":"Curr Opin Cell Biol"},{"key":"pcbi.1010340.ref086","doi-asserted-by":"crossref","first-page":"13870","DOI":"10.1073\/pnas.1108608108","article-title":"Response rescaling in bacterial chemotaxis","volume":"108","author":"MD Lazova","year":"2011","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref087","volume-title":"Expanded ed.","author":"HC Berg","year":"1993"},{"key":"pcbi.1010340.ref088","doi-asserted-by":"crossref","first-page":"048101","DOI":"10.1103\/PhysRevLett.109.048101","article-title":"Pathway-Based Mean-Field Model for Escherichia coli Chemotaxis","volume":"109","author":"G Si","year":"2012","journal-title":"Phys Rev Lett"},{"key":"pcbi.1010340.ref089","doi-asserted-by":"crossref","first-page":"e1003694","DOI":"10.1371\/journal.pcbi.1003694","article-title":"Limits of Feedback Control in Bacterial Chemotaxis","volume":"10","author":"YS Dufour","year":"2014","journal-title":"PLOS Comput Biol."},{"key":"pcbi.1010340.ref090","doi-asserted-by":"crossref","first-page":"16036","DOI":"10.1038\/npjsba.2016.36","article-title":"Logarithmic sensing in Bacillus subtilis aerotaxis","volume":"3","author":"F Menolascina","year":"2017","journal-title":"NPJ Syst Biol Appl"},{"key":"pcbi.1010340.ref091","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/0022-5193(71)90050-6","article-title":"Model for chemotaxis","volume":"30","author":"EF Keller","year":"1971","journal-title":"J Theor Biol"},{"key":"pcbi.1010340.ref092","doi-asserted-by":"crossref","first-page":"341","DOI":"10.2307\/3318418","article-title":"Exponential convergence of Langevin distributions and their discrete approximations","volume":"2","author":"GO Roberts","year":"1996","journal-title":"Bernoulli"},{"key":"pcbi.1010340.ref093","first-page":"2","article-title":"MCMC using Hamiltonian dynamics.","volume":"2","author":"RM Neal","year":"2011","journal-title":"Handb Markov Chain Monte Carlo"},{"key":"pcbi.1010340.ref094","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/j.1467-9868.2010.00765.x","article-title":"Riemann manifold langevin and hamiltonian monte carlo methods","volume":"73","author":"M Girolami","year":"2011","journal-title":"J R Stat Soc Ser B Stat Methodol."},{"key":"pcbi.1010340.ref095","article-title":"Theoretical guarantees for approximate sampling from smooth and log-concave densities","author":"AS Dalalyan","year":"2014","journal-title":"ArXiv Prepr ArXiv14127392"},{"key":"pcbi.1010340.ref096","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/s00213-004-1877-8","article-title":"Reduced sensitivity to reward in CB1 knockout mice","volume":"176","author":"C Sanchis-Segura","year":"2004","journal-title":"Psychopharmacology"},{"key":"pcbi.1010340.ref097","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00213-008-1432-0","article-title":"Attenuation of basal and cocaine-enhanced locomotion and nucleus accumbens dopamine in cannabinoid CB1-receptor-knockout mice","volume":"204","author":"X Li","year":"2009","journal-title":"Psychopharmacology (Berl)"},{"key":"pcbi.1010340.ref098","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1146\/annurev-neuro-072116-031109","article-title":"Neural circuitry of reward prediction error","volume":"40","author":"M Watabe-Uchida","year":"2017","journal-title":"Annu Rev Neurosci"},{"key":"pcbi.1010340.ref099","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1080\/00223980.1959.9916336","article-title":"Weber\u2019s law and related functions","volume":"47","author":"Ekman G\u00f6s","year":"1959","journal-title":"J Psychol."},{"key":"pcbi.1010340.ref100","doi-asserted-by":"crossref","first-page":"e57455","DOI":"10.1371\/journal.pone.0057455","article-title":"Comparing apples and oranges: fold-change detection of multiple simultaneous inputs","volume":"8","author":"Y Hart","year":"2013","journal-title":"PloS One"},{"key":"pcbi.1010340.ref101","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1038\/s41586-019-1924-6","article-title":"A distributional code for value in dopamine-based reinforcement learning","volume":"577","author":"W Dabney","year":"2020","journal-title":"Nature"},{"key":"pcbi.1010340.ref102","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1038\/s41593-021-00807-7","article-title":"Rare rewards amplify dopamine responses","volume":"24","author":"KM Rothenhoefer","year":"2021","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref103","doi-asserted-by":"crossref","first-page":"1877","DOI":"10.1038\/s41467-019-09521-2","article-title":"Bacterial chemotaxis in a microfluidic T-maze reveals strong phenotypic heterogeneity in chemotactic sensitivity","volume":"10","author":"MM Salek","year":"2019","journal-title":"Nat Commun"},{"key":"pcbi.1010340.ref104","doi-asserted-by":"crossref","first-page":"9557","DOI":"10.1523\/JNEUROSCI.19-21-09557.1999","article-title":"The Fundamental Role of Pirouettes in Caenorhabditis elegans Chemotaxis","volume":"19","author":"JT Pierce-Shimomura","year":"1999","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref105","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1126\/science.1172667","article-title":"Chlamydomonas Swims with Two \u201cGears\u201d in a Eukaryotic Version of Run-and-Tumble Locomotion","volume":"325","author":"M Polin","year":"2009","journal-title":"Science"},{"key":"pcbi.1010340.ref106","doi-asserted-by":"crossref","first-page":"2776","DOI":"10.1073\/pnas.1315205111","article-title":"Bidirectional thermotaxis in Caenorhabditis elegans is mediated by distinct sensorimotor strategies driven by the AFD thermosensory neurons","volume":"111","author":"L Luo","year":"2014","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1010340.ref107","doi-asserted-by":"crossref","first-page":"e18109","DOI":"10.7554\/eLife.18109","article-title":"Aerotaxis in the closest relatives of animals","volume":"5","author":"JB Kirkegaard","year":"2016","journal-title":"Elife"},{"key":"pcbi.1010340.ref108","doi-asserted-by":"crossref","first-page":"e1003672","DOI":"10.1371\/journal.pcbi.1003672","article-title":"Behaviors and strategies of bacterial navigation in chemical and nonchemical gradients","volume":"10","author":"B Hu","year":"2014","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010340.ref109","doi-asserted-by":"crossref","first-page":"102796","DOI":"10.1016\/j.isci.2021.102796","article-title":"Temporal fluctuations in chemotaxis gain implement a simulated-tempering strategy for efficient navigation in complex environments","volume":"24","author":"O Karin","year":"2021","journal-title":"Iscience"},{"key":"pcbi.1010340.ref110","doi-asserted-by":"crossref","first-page":"e05360","DOI":"10.7554\/eLife.05360","article-title":"VTA neurons coordinate with the hippocampal reactivation of spatial experience","volume":"4","author":"SN Gomperts","year":"2015","journal-title":"eLife"},{"key":"pcbi.1010340.ref111","doi-asserted-by":"crossref","first-page":"R37","DOI":"10.1016\/j.cub.2017.10.073","article-title":"The role of hippocampal replay in memory and planning","volume":"28","author":"HF \u00d3lafsd\u00f3ttir","year":"2018","journal-title":"Curr Biol"},{"key":"pcbi.1010340.ref112","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.neuron.2019.01.052","article-title":"Hippocampal reactivation of random trajectories resembling Brownian diffusion","volume":"102","author":"F Stella","year":"2019","journal-title":"Neuron"},{"key":"pcbi.1010340.ref113","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1016\/S0896-6273(02)01096-6","article-title":"Memory of sequential experience in the hippocampus during slow wave sleep","volume":"36","author":"AK Lee","year":"2002","journal-title":"Neuron"},{"key":"pcbi.1010340.ref114","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1038\/nature12112","article-title":"Hippocampal place-cell sequences depict future paths to remembered goals","volume":"497","author":"BE Pfeiffer","year":"2013","journal-title":"Nature"},{"key":"pcbi.1010340.ref115","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.neuron.2009.07.027","article-title":"Hippocampal replay of extended experience","volume":"63","author":"TJ Davidson","year":"2009","journal-title":"Neuron"},{"key":"pcbi.1010340.ref116","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1016\/j.neuropsychologia.2004.06.026","article-title":"Deficits in saccadic eye-movement control in Parkinson\u2019s disease","volume":"43","author":"F Chan","year":"2005","journal-title":"Neuropsychologia"},{"key":"pcbi.1010340.ref117","doi-asserted-by":"crossref","first-page":"592","DOI":"10.3389\/fneur.2017.00592","article-title":"Eye movements in Parkinson\u2019s disease and inherited parkinsonian syndromes","volume":"8","author":"E Pretegiani","year":"2017","journal-title":"Front Neurol"},{"key":"pcbi.1010340.ref118","doi-asserted-by":"crossref","first-page":"5010","DOI":"10.1523\/JNEUROSCI.0432-19.2019","article-title":"Reward prediction error modulates saccade vigor","volume":"39","author":"E Sedaghat-Nejad","year":"2019","journal-title":"J Neurosci"},{"key":"pcbi.1010340.ref119","doi-asserted-by":"crossref","first-page":"056114","DOI":"10.1103\/PhysRevE.79.056114","article-title":"L\u00e9vy-like diffusion in eye movements during spoken-language comprehension","volume":"79","author":"DG Stephen","year":"2009","journal-title":"Phys Rev E"},{"key":"pcbi.1010340.ref120","doi-asserted-by":"crossref","first-page":"797","DOI":"10.3389\/fpsyg.2013.00797","article-title":"Fixational eye movements during viewing of dynamic natural scenes","volume":"4","author":"JA Roberts","year":"2013","journal-title":"Front Psychol."},{"key":"pcbi.1010340.ref121","doi-asserted-by":"crossref","first-page":"e0139379","DOI":"10.1371\/journal.pone.0139379","article-title":"Temporal structure of human gaze dynamics is invariant during free viewing","volume":"10","author":"CA Marlow","year":"2015","journal-title":"PloS One"},{"key":"pcbi.1010340.ref122","doi-asserted-by":"crossref","first-page":"1080","DOI":"10.1126\/science.1168878","article-title":"Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning","volume":"324","author":"H-C Tsai","year":"2009","journal-title":"Science"},{"key":"pcbi.1010340.ref123","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1137\/0325042","article-title":"Diffusion for Global Optimization in $\\mathbb{R}^n $","volume":"25","author":"T-S Chiang","year":"1987","journal-title":"SIAM J Control Optim"},{"key":"pcbi.1010340.ref124","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1137\/0329055","article-title":"Recursive Stochastic Algorithms for Global Optimization in $\\mathbb{R}^d $","volume":"29","author":"SB Gelfand","year":"1991","journal-title":"SIAM J Control Optim"},{"key":"pcbi.1010340.ref125","first-page":"7847","volume-title":"Advances in Neural Information Processing Systems 31","author":"H Lee","year":"2018"},{"key":"pcbi.1010340.ref126","first-page":"9671","volume-title":"Advances in Neural Information Processing Systems 31","author":"MA Erdogdu","year":"2018"},{"key":"pcbi.1010340.ref127","doi-asserted-by":"crossref","first-page":"20881","DOI":"10.1073\/pnas.1820003116","article-title":"Sampling can be faster than optimization","volume":"116","author":"Y-A Ma","year":"2019","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1010340.ref128","article-title":"Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusion","author":"Y Chen","year":"2020","journal-title":"ArXiv200701990 Cs Math Stat."},{"key":"pcbi.1010340.ref129","doi-asserted-by":"crossref","first-page":"e1005429","DOI":"10.1371\/journal.pcbi.1005429","article-title":"Feedback between motion and sensation provides nonlinear boost in run-and-tumble navigation","volume":"13","author":"J Long","year":"2017","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1010340.ref130","doi-asserted-by":"crossref","first-page":"2366","DOI":"10.1038\/npp.2014.84","article-title":"Role of dopamine D2 receptors in human reinforcement learning","volume":"39","author":"C Eisenegger","year":"2014","journal-title":"Neuropsychopharmacology"},{"key":"pcbi.1010340.ref131","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-019-43245-z","article-title":"Dopamine blockade impairs the exploration-exploitation trade-off in rats","volume":"9","author":"F Cinotti","year":"2019","journal-title":"Sci Rep"},{"key":"pcbi.1010340.ref132","doi-asserted-by":"crossref","first-page":"1062","DOI":"10.1038\/nn.2342","article-title":"The neurogenetics of exploration and exploitation: Prefrontal and striatal dopaminergic components","volume":"12","author":"MJ Frank","year":"2009","journal-title":"Nat Neurosci"},{"key":"pcbi.1010340.ref133","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1037\/a0037128","article-title":"Dopamine modulates novelty seeking behavior during decision making","volume":"128","author":"VD Costa","year":"2014","journal-title":"Behav Neurosci"},{"key":"pcbi.1010340.ref134","article-title":"Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis","author":"M Raginsky","year":"2017","journal-title":"ArXiv170203849 Cs Math Stat."},{"key":"pcbi.1010340.ref135","first-page":"3122","volume-title":"Advances in Neural Information Processing Systems 31","author":"P Xu","year":"2018"},{"key":"pcbi.1010340.ref136","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1038\/43199","article-title":"Robustness in simple biochemical networks","volume":"387","author":"N Barkai","year":"1997","journal-title":"Nature"},{"key":"pcbi.1010340.ref137","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1038\/16483","article-title":"Robustness in bacterial chemotaxis","volume":"397","author":"U Alon","year":"1999","journal-title":"Nature"},{"key":"pcbi.1010340.ref138","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1016\/j.cels.2016.02.006","article-title":"Perfect and near-perfect adaptation in cell signaling","volume":"2","author":"JE Ferrell","year":"2016","journal-title":"Cell Syst"},{"key":"pcbi.1010340.ref139","doi-asserted-by":"crossref","first-page":"2491","DOI":"10.1016\/j.cub.2014.08.064","article-title":"Dopamine reward prediction error responses reflect marginal utility","volume":"24","author":"WR Stauffer","year":"2014","journal-title":"Curr Biol"},{"key":"pcbi.1010340.ref140","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1162\/089976600300015961","article-title":"Reinforcement learning in continuous time and space","volume":"12","author":"K. Doya","year":"2000","journal-title":"Neural Comput"},{"key":"pcbi.1010340.ref141","volume-title":"Learning and sequential decision making","author":"AG Barto","year":"1989"},{"key":"pcbi.1010340.ref142","doi-asserted-by":"crossref","first-page":"3721","DOI":"10.1093\/brain\/aws273","article-title":"Distinct roles of dopamine and subthalamic nucleus in learning and probabilistic decision making","volume":"135","author":"EJ Coulthard","year":"2012","journal-title":"Brain"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1010340","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T00:00:00Z","timestamp":1659571200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010340","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,4]],"date-time":"2022-08-04T14:01:52Z","timestamp":1659621712000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010340"}},"subtitle":[],"editor":[{"given":"Blake A.","family":"Richards","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,7,25]]},"references-count":142,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7,25]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010340","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.04.15.439955","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,25]]}}}