{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"bioRxiv"}],"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T08:17:00Z","timestamp":1777623420477,"version":"3.51.4"},"posted":{"date-parts":[[2026,4,11]]},"group-title":"Neuroscience","reference-count":46,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T00:00:00Z","timestamp":1775865600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"crossref","award":["R90DA060339"],"award-info":[{"award-number":["R90DA060339"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,4,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>Adaptive behavior requires learning the value of environmental features while selectively attending to those most likely to yield reward. Reward prediction errors (RPEs) drive value learning and learned values guide attention, yet the computational function linking RPEs to attentional modulation remains unspecified. Here, we developed a reinforcement learning model with a perceptual front-end to investigate how value and RPE signals modulate attentional gain during learning. We compared five candidate RPE-attention transfer functions, each combined with either single- or multi-focus attention, against behavioral data from two adult male rhesus macaques performing a color-value learning task with shifting reward contingencies. Monkeys exhibited rapid initial learning followed by sub-optimal asymptotic accuracy. Overall, single-focus architectures consistently outperformed multi-focus counterparts on matching monkey errors, indicating that macaques collapse the value distribution into a winner-take-all attentional focus. Furthermore, the \u201cSwitch\u201d model, in which attention targets the highest-valued feature but transiently inverts following negative RPEs, produced the fastest exploration dynamics following target switches and, together with the Absolute Value model, yielded decision confidence trajectories that positively correlated with empirical reaction times. In support of this, single-neuron correlation analyses revealed that 27 \u2212 42% of neurons in prefrontal cortex, frontal eye fields, and lateral intraparietal area encoded previous-trial RPE at the time of next trial onset. In total, we conclude that capacity constrained attention that inverts its focus after negative RPE best explains value learning dynamics. These results provide a normative account for why biological learners sacrifice asymptotic precision for rapid adaptation in volatile environments.<\/jats:p>\n                <jats:sec>\n                  <jats:title>Significance Statement<\/jats:title>\n                  <jats:p>Learning which features of the environment predict reward requires both reinforcement learning and selective attention, yet how these processes interact algorithmically remains unknown. We developed a computational model that specifies how reward prediction errors dynamically adjust the gain of feature-based attention during value learning. Testing competing hypotheses against macaque behavioral data, we show that a \u201cSwitch\u201d mechanism, in which negative prediction errors transiently invert attentional focus away from the highest-valued feature, best captures primate learning dynamics. This architecture reveals a principled trade-off: the brain sacrifices asymptotic accuracy for rapid detection of environmental change, using error-triggered attentional inversion as a directed exploration strategy. These findings bridge reinforcement learning theory and attention research by identifying the transfer function linking prediction errors to sensory gain modulation.<\/jats:p>\n                <\/jats:sec>","DOI":"10.64898\/2026.04.10.717847","type":"posted-content","created":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T07:50:12Z","timestamp":1775980212000},"source":"Crossref","is-referenced-by-count":1,"title":["Modulation of feature attention by reward prediction error explains value learning behavior"],"prefix":"10.64898","author":[{"given":"Mingze L.","family":"Leukos (\u674e\u94ed\u6cfd)","sequence":"first","affiliation":[{"name":"Department of Psychology, New York University, New York, NY 10003, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Albert","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Psychology, New York University, New York, NY 10003, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9904-7471","authenticated-orcid":false,"given":"Grace W.","family":"Lindsay","sequence":"additional","affiliation":[{"name":"Department of Psychology, New York University, New York, NY 10003, USA"},{"name":"Center for Data Science, New York University, New York, NY 10011, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"54368","reference":[{"key":"2026041511201591000_2026.04.10.717847v1.1","doi-asserted-by":"publisher","DOI":"10.1111\/nyas.12957"},{"key":"2026041511201591000_2026.04.10.717847v1.2","doi-asserted-by":"publisher","DOI":"10.1093\/scan\/nsw141"},{"key":"2026041511201591000_2026.04.10.717847v1.3","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1104047108"},{"issue":"8","key":"2026041511201591000_2026.04.10.717847v1.4","doi-asserted-by":"crossref","first-page":"1644","DOI":"10.3758\/s13414-012-0348-2","article-title":"Value-driven attentional and oculomotor capture during goal-directed, unconstrained viewing","volume":"74","year":"2012","journal-title":"Attention, Perception, & Psychophysics"},{"key":"2026041511201591000_2026.04.10.717847v1.5","doi-asserted-by":"crossref","first-page":"108","DOI":"10.3389\/fncir.2017.00108","article-title":"Neuromodulatory systems and their interactions: a review of models, theories, and experiments","volume":"11","year":"2017","journal-title":"Frontiers in neural circuits"},{"key":"2026041511201591000_2026.04.10.717847v1.6","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2015.10.001"},{"key":"2026041511201591000_2026.04.10.717847v1.7","first-page":"4","article-title":"Representation of perceptual color space in macaque posterior inferior temporal cortex (the v4 complex)","volume":"3","year":"2016","journal-title":"Eneuro"},{"key":"2026041511201591000_2026.04.10.717847v1.8","doi-asserted-by":"publisher","DOI":"10.1126\/science.1138071"},{"key":"2026041511201591000_2026.04.10.717847v1.9","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2007.2098"},{"key":"2026041511201591000_2026.04.10.717847v1.10","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.ne.18.030195.001205"},{"issue":"1","key":"2026041511201591000_2026.04.10.717847v1.11","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1177\/1073858420907591","article-title":"Dopamine, prediction error and beyond","volume":"27","year":"2021","journal-title":"The Neuroscientist"},{"key":"2026041511201591000_2026.04.10.717847v1.12","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cobeha.2021.06.004","article-title":"What do reinforcement learning models measure? interpreting model parameters in cognition and neuroscience","volume":"41","year":"2021","journal-title":"Current opinion in behavioral sciences"},{"key":"2026041511201591000_2026.04.10.717847v1.13","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainres.2005.09.068"},{"key":"2026041511201591000_2026.04.10.717847v1.14","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1014269108"},{"key":"2026041511201591000_2026.04.10.717847v1.15","doi-asserted-by":"publisher","DOI":"10.1162\/jocn"},{"key":"2026041511201591000_2026.04.10.717847v1.16","doi-asserted-by":"publisher","DOI":"10.1037\/xhp0000019"},{"issue":"1","key":"2026041511201591000_2026.04.10.717847v1.17","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1080\/17470215208416600","article-title":"On the rate of gain of information","volume":"4","year":"1952","journal-title":"Quarterly Journal of experimental psychology"},{"key":"2026041511201591000_2026.04.10.717847v1.18","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00458.2005"},{"key":"2026041511201591000_2026.04.10.717847v1.19","doi-asserted-by":"publisher","DOI":"10.1016\/j.cell.2024.01.041"},{"key":"2026041511201591000_2026.04.10.717847v1.20","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2016.12.040"},{"issue":"1","key":"2026041511201591000_2026.04.10.717847v1.21","doi-asserted-by":"crossref","first-page":"1474","DOI":"10.1038\/s42003-025-08871-6","article-title":"Spatially global effects of feature-based attention in functional subdivisions of human subcortical nuclei","volume":"8","year":"2025","journal-title":"Communications Biology"},{"key":"2026041511201591000_2026.04.10.717847v1.22","doi-asserted-by":"crossref","first-page":"29","DOI":"10.3389\/fncom.2020.00029","article-title":"Attention in psychology, neuroscience, and machine learning","volume":"14","year":"2020","journal-title":"Frontiers in computational neuroscience"},{"key":"2026041511201591000_2026.04.10.717847v1.23","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.38105"},{"issue":"8","key":"2026041511201591000_2026.04.10.717847v1.24","doi-asserted-by":"crossref","first-page":"2327","DOI":"10.3758\/s13414-017-1390-x","article-title":"Limits in feature-based attention to multiple colors","volume":"79","year":"2017","journal-title":"Attention, Perception, & Psychophysics"},{"key":"2026041511201591000_2026.04.10.717847v1.25","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2004.04.028"},{"key":"2026041511201591000_2026.04.10.717847v1.26","doi-asserted-by":"crossref","unstructured":"Martinez-Trujillo, J. C. and Treue, S . (2005). The feature similarity gain model of attention: Unifying multiplicative effects of spatial and feature-based attention. In Neurobiology of attention, pages 300\u2013304. Elsevier.","DOI":"10.1016\/B978-012375731-9\/50053-7"},{"key":"2026041511201591000_2026.04.10.717847v1.27","doi-asserted-by":"publisher","DOI":"10.1016\/j.tins.2006.04.001"},{"key":"2026041511201591000_2026.04.10.717847v1.28","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0192-16.2016"},{"key":"2026041511201591000_2026.04.10.717847v1.29","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3002-07.2007"},{"key":"2026041511201591000_2026.04.10.717847v1.30","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2978-14.2015"},{"issue":"1","key":"2026041511201591000_2026.04.10.717847v1.31","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1038\/s41467-018-08184-9","article-title":"Feature-specific prediction errors and surprise across macaque fronto-striatal circuits","volume":"10","year":"2019","journal-title":"Nature Communications"},{"key":"2026041511201591000_2026.04.10.717847v1.32","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhm109"},{"key":"2026041511201591000_2026.04.10.717847v1.33","doi-asserted-by":"crossref","unstructured":"Perkins, A. Q. and Rich, E. L . (2024). Attention-dependent attribute comparisons underlie multi-attribute decision-making in orbitofrontal cortex. bioRxiv, pages 2024\u201311.","DOI":"10.1101\/2024.11.12.623291"},{"key":"2026041511201591000_2026.04.10.717847v1.34","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2019.01.010"},{"key":"2026041511201591000_2026.04.10.717847v1.35","first-page":"209","article-title":"Modulation of feedback-related negativity during trial-and-error exploration and encoding of behavioral shifts","volume":"7","year":"2013","journal-title":"Frontiers in neuroscience"},{"key":"2026041511201591000_2026.04.10.717847v1.36","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00108.2016"},{"key":"2026041511201591000_2026.04.10.717847v1.37","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.6274-10.2011"},{"key":"2026041511201591000_2026.04.10.717847v1.38","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"2026041511201591000_2026.04.10.717847v1.39","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2005.05.014"},{"key":"2026041511201591000_2026.04.10.717847v1.40","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2006.05.044"},{"key":"2026041511201591000_2026.04.10.717847v1.41","unstructured":"Sutton, R. S. , Barto, A. G ., et al. (1998). Reinforcement learning: An introduction, volume 1. MIT press Cambridge."},{"key":"2026041511201591000_2026.04.10.717847v1.42","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopsycho.2020.108007"},{"key":"2026041511201591000_2026.04.10.717847v1.43","doi-asserted-by":"publisher","DOI":"10.1038\/21176"},{"key":"2026041511201591000_2026.04.10.717847v1.44","doi-asserted-by":"publisher","DOI":"10.1167\/15.1.15"},{"issue":"11","key":"2026041511201591000_2026.04.10.717847v1.45","first-page":"1430","article-title":"The specificity of feature-based attentional guidance is equivalent under single- and dual-target search","volume":"49","year":"2023","journal-title":"Journal of Experimental Psychology: Human Perception and Performance"},{"key":"2026041511201591000_2026.04.10.717847v1.46","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2008.02.034"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.04.10.717847","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T18:20:25Z","timestamp":1776277225000},"score":1,"resource":{"primary":{"URL":"http:\/\/biorxiv.org\/lookup\/doi\/10.64898\/2026.04.10.717847"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,11]]},"references-count":46,"URL":"https:\/\/doi.org\/10.64898\/2026.04.10.717847","relation":{},"subject":[],"published":{"date-parts":[[2026,4,11]]},"subtype":"preprint"}}