{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T05:51:49Z","timestamp":1773553909359,"version":"3.50.1"},"reference-count":63,"publisher":"MIT Press","issue":"2","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2016,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Attention includes processes that evaluate stimuli relevance, select the most relevant stimulus against less relevant stimuli, and bias choice behavior toward the selected information. It is not clear how these processes interact. Here, we captured these processes in a reinforcement learning framework applied to a feature-based attention task that required macaques to learn and update the value of stimulus features while ignoring nonrelevant sensory features, locations, and action plans. We found that value-based reinforcement learning mechanisms could account for feature-based attentional selection and choice behavior but required a value-independent stickiness selection process to explain selection errors while at asymptotic behavior. By comparing different reinforcement learning schemes, we found that trial-by-trial selections were best predicted by a model that only represents expected values for the task-relevant feature dimension, with nonrelevant stimulus features and action plans having only a marginal influence on covert selections. These findings show that attentional control subprocesses can be described by (1) the reinforcement learning of feature values within a restricted feature space that excludes irrelevant feature dimensions, (2) a stochastic selection process on feature-specific value representations, and (3) value-independent stickiness toward previous feature selections akin to perseveration in the motor domain. We speculate that these three mechanisms are implemented by distinct but interacting brain circuits and that the proposed formal account of feature-based stimulus selection will be important to understand how attentional subprocesses are implemented in primate brain networks.<\/jats:p>","DOI":"10.1162\/jocn_a_00894","type":"journal-article","created":{"date-parts":[[2015,12,23]],"date-time":"2015-12-23T12:35:20Z","timestamp":1450874120000},"page":"333-349","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":29,"title":["Attentional Selection Can Be Predicted by Reinforcement Learning of Task-relevant Stimulus Features Weighted by Value-independent Stickiness"],"prefix":"10.1162","volume":"28","author":[{"given":"Matthew","family":"Balcarras","sequence":"first","affiliation":[{"name":"1York University, Toronto, Canada"}]},{"given":"Salva","family":"Ardid","sequence":"additional","affiliation":[{"name":"1York University, Toronto, Canada"},{"name":"2Boston University"}]},{"given":"Daniel","family":"Kaping","sequence":"additional","affiliation":[{"name":"1York University, Toronto, Canada"}]},{"given":"Stefan","family":"Everling","sequence":"additional","affiliation":[{"name":"3University of Western Ontario"}]},{"given":"Thilo","family":"Womelsdorf","sequence":"additional","affiliation":[{"name":"1York University, Toronto, Canada"},{"name":"3University of Western Ontario"}]}],"member":"281","published-online":{"date-parts":[[2016,2,1]]},"reference":[{"key":"2021073020593236800_R1","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1080\/03640210802352992","article-title":"Comparison of decision learning models using the generalization criterion method","volume":"32","author":"Ahn","year":"2008","journal-title":"Cognitive Science"},{"key":"2021073020593236800_R2","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1177\/1059712307078663","article-title":"Shifting attention using a temporal difference prediction error and high-dimensional input","volume":"15","author":"Alexander","year":"2007","journal-title":"Adaptive Behavior"},{"key":"2021073020593236800_R3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1167\/13.3.7","article-title":"A value-driven mechanism of attentional selection","volume":"13","author":"Anderson","year":"2013","journal-title":"Journal of Vision"},{"key":"2021073020593236800_R4","doi-asserted-by":"crossref","first-page":"10367","DOI":"10.1073\/pnas.1104047108","article-title":"Value-driven attentional capture","volume":"108","author":"Anderson","year":"2011","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2021073020593236800_R5","doi-asserted-by":"crossref","first-page":"P8","DOI":"10.1186\/1471-2202-15-S1-P8","article-title":"\u201cAdaptive learning\u201d as a mechanistic candidate for reaching optimal task-set representations flexibly","volume":"15","author":"Ardid","year":"2014","journal-title":"BMC Neuroscience"},{"key":"2021073020593236800_R6","doi-asserted-by":"crossref","first-page":"19504","DOI":"10.1523\/JNEUROSCI.1356-13.2013","article-title":"A tweaking principle for executive control: Neuronal circuit mechanism for rule-based task switching and conflict resolution","volume":"33","author":"Ardid","year":"2013","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R7","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/j.jneumeth.2008.07.014","article-title":"A flexible software tool for temporally-precise behavioral control in Matlab","volume":"174","author":"Asaad","year":"2008","journal-title":"Journal of Neuroscience Methods"},{"key":"2021073020593236800_R8","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1016\/j.tics.2012.06.010","article-title":"Top\u2013down versus bottom\u2013up attentional control: A failed theoretical dichotomy","volume":"16","author":"Awh","year":"2012","journal-title":"Trends in Cognitive Sciences"},{"key":"2021073020593236800_R9","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1016\/j.neuron.2014.01.008","article-title":"Contributions of orbitofrontal and lateral prefrontal cortices to economic choice and the good-to-action transformation","volume":"81","author":"Cai","year":"2014","journal-title":"Neuron"},{"key":"2021073020593236800_R10","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.visres.2012.12.005","article-title":"Rewards teach visual selective attention","volume":"85","author":"Chelazzi","year":"2013","journal-title":"Vision Research"},{"key":"2021073020593236800_R11","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1037\/a0030852","article-title":"Cognitive control over learning: Creating, clustering, and generalizing task-set structure","volume":"120","author":"Collins","year":"2013","journal-title":"Psychological Review"},{"key":"2021073020593236800_R12","doi-asserted-by":"crossref","first-page":"473","DOI":"10.3758\/s13415-014-0277-8","article-title":"Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation","volume":"14","author":"Dayan","year":"2014","journal-title":"Cognitive, Affective & Behavioral Neuroscience"},{"key":"2021073020593236800_R13","doi-asserted-by":"crossref","first-page":"1218","DOI":"10.1038\/81504","article-title":"Learning and selective attention","volume":"3(Suppl.)","author":"Dayan","year":"2000","journal-title":"Nature Neuroscience"},{"key":"2021073020593236800_R14","doi-asserted-by":"crossref","first-page":"12066","DOI":"10.1523\/JNEUROSCI.1088-12.2012","article-title":"Corticostriatal connectivity underlies individual differences in the balance between habitual and goal-directed action control","volume":"32","author":"de Wit","year":"2012","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R15","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/S0079-6123(00)26016-0","article-title":"Reward-dependent learning in neuronal networks for planning and decision making","volume":"126","author":"Dehaene","year":"2000","journal-title":"Progress in Brain Research"},{"key":"2021073020593236800_R16","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1111\/j.1467-9280.2009.02360.x","article-title":"Learning to attend and to ignore is a matter of gains and losses","volume":"20","author":"Della Libera","year":"2009","journal-title":"Psychological Science"},{"key":"2021073020593236800_R17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via EM algorithm","volume":"39","author":"Dempster","year":"1977","journal-title":"Journal of the Royal Statistical Society, Series B: Methodological"},{"key":"2021073020593236800_R18","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.neuron.2013.09.007","article-title":"Goals and habits in the brain","volume":"80","author":"Dolan","year":"2013","journal-title":"Neuron"},{"key":"2021073020593236800_R19","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1126\/science.1252254","article-title":"Human cognition. Foundations of human reasoning in the prefrontal cortex","volume":"344","author":"Donoso","year":"2014","journal-title":"Science"},{"key":"2021073020593236800_R20","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1038\/nrn1114","article-title":"Exploring the consequences of the previous trial","volume":"4","author":"Fecteau","year":"2003","journal-title":"Nature Reviews Neuroscience"},{"key":"2021073020593236800_R21","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.conb.2010.02.008","article-title":"Learning latent structure: Carving nature at its joints","volume":"20","author":"Gershman","year":"2010","journal-title":"Current Opinion in Neurobiology"},{"key":"2021073020593236800_R22","doi-asserted-by":"crossref","first-page":"15647","DOI":"10.1073\/pnas.1014269108","article-title":"Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis","volume":"108(Suppl. 3)","author":"Glimcher","year":"2011","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2021073020593236800_R23","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1016\/j.neuron.2012.09.034","article-title":"Attention, learning, and the value of information","volume":"76","author":"Gottlieb","year":"2012","journal-title":"Neuron"},{"key":"2021073020593236800_R24","doi-asserted-by":"crossref","first-page":"15497","DOI":"10.1523\/JNEUROSCI.3270-14.2014","article-title":"Attention, reward, and information seeking","volume":"34","author":"Gottlieb","year":"2014","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R25","doi-asserted-by":"crossref","first-page":"18120","DOI":"10.1073\/pnas.1109322108","article-title":"Transformation of stimulus value signals into motor commands during simple choice","volume":"108","author":"Hare","year":"2011","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2021073020593236800_R26","doi-asserted-by":"crossref","first-page":"e1002028","DOI":"10.1371\/journal.pcbi.1002028","article-title":"Disentangling the roles of approach, activation and valence in instrumental and Pavlovian responding","volume":"7","author":"Huys","year":"2011","journal-title":"PLoS Computational Biology"},{"key":"2021073020593236800_R27","doi-asserted-by":"crossref","first-page":"e1001224","DOI":"10.1371\/journal.pbio.1001224","article-title":"Specific contributions of ventromedial, anterior cingulate, and lateral prefrontal cortex for attentional selection and stimulus valuation","volume":"9","author":"Kaping","year":"2011","journal-title":"PLoS Biology"},{"key":"2021073020593236800_R28","doi-asserted-by":"crossref","first-page":"1581","DOI":"10.1038\/nn.2961","article-title":"Double dissociation of value computations in orbitofrontal and anterior cingulate neurons","volume":"14","author":"Kennerley","year":"2011","journal-title":"Nature Neuroscience"},{"key":"2021073020593236800_R29","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1016\/j.tics.2014.05.008","article-title":"Attention as an effect not a cause","volume":"18","author":"Krauzlis","year":"2014","journal-title":"Trends in Cognitive Sciences"},{"key":"2021073020593236800_R30","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1080\/13506280544000039","article-title":"Rapid learning in attention shifts: A review","volume":"13","author":"Kristjansson","year":"2006","journal-title":"Visual Cognition"},{"key":"2021073020593236800_R31","doi-asserted-by":"crossref","first-page":"5","DOI":"10.3758\/APP.72.1.5","article-title":"Where perception meets memory: A review of repetition priming in visual search tasks","volume":"72","author":"Kristjansson","year":"2010","journal-title":"Attention, Perception, & Psychophysics"},{"key":"2021073020593236800_R32","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1017\/CBO9780511760402.002","article-title":"Evolution of attention in learning","volume-title":"Computational models of conditioning","author":"Kruschke","year":"2010"},{"key":"2021073020593236800_R33","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1901\/jeab.2005.110-04","article-title":"Dynamic response-by-response models of matching behavior in rhesus monkeys","volume":"84","author":"Lau","year":"2005","journal-title":"Journal of the Experimental Analysis of Behavior"},{"key":"2021073020593236800_R34","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/j.neuron.2008.02.021","article-title":"Value representations in the primate striatum during matching behavior","volume":"58","author":"Lau","year":"2008","journal-title":"Neuron"},{"key":"2021073020593236800_R35","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pcbi.1000894","article-title":"Reinforcement learning on slow features of high-dimensional input streams","volume":"6","author":"Legenstein","year":"2010","journal-title":"PLoS Computational Biology"},{"key":"2021073020593236800_R36","doi-asserted-by":"crossref","first-page":"1864","DOI":"10.1523\/JNEUROSCI.4920-12.2013","article-title":"Choice coding in frontal cortex during stimulus-guided or action-guided decision-making","volume":"33","author":"Luk","year":"2013","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R37","doi-asserted-by":"crossref","first-page":"5232","DOI":"10.1073\/pnas.0911972107","article-title":"Optimal reward harvesting in complex perceptual environments","volume":"107","author":"Navalpakkam","year":"2010","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2021073020593236800_R38","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1146\/annurev-neuro-061010-113648","article-title":"Neurobiology of economic choice: A good-based model","volume":"34","author":"Padoa-Schioppa","year":"2011","journal-title":"Annual Review of Neuroscience"},{"key":"2021073020593236800_R39","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:osobl\/9780199552917.001.0001","volume-title":"The neurobiology of the prefrontal cortex: Anatomy, evolution, and the origin of insight","author":"Passingham","year":"2012"},{"key":"2021073020593236800_R40","doi-asserted-by":"crossref","first-page":"11182","DOI":"10.1523\/JNEUROSCI.1929-09.2009","article-title":"Reward modulates attention independently of action value in posterior parietal cortex","volume":"29","author":"Peck","year":"2009","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R41","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1038\/nn.3328","article-title":"The primate amygdala combines information about space and value","volume":"16","author":"Peck","year":"2013","journal-title":"Nature Neuroscience"},{"key":"2021073020593236800_R42","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/B978-0-12-416008-8.00008-5","article-title":"The computation of stimulus values in simple choice","volume-title":"Neuroeconomics: Decision making and the brain","author":"Rangel","year":"2014","edition":"2nd ed."},{"key":"2021073020593236800_R43","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1016\/j.conb.2010.03.001","article-title":"Neural computations associated with goal-directed choice","volume":"20","author":"Rangel","year":"2010","journal-title":"Current Opinion in Neurobiology"},{"key":"2021073020593236800_R44","doi-asserted-by":"crossref","first-page":"2176","DOI":"10.1162\/0899766054615699","article-title":"Attention-gated reinforcement learning of internal representations for classification","volume":"17","author":"Roelfsema","year":"2005","journal-title":"Neural Computation"},{"key":"2021073020593236800_R45","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.tics.2009.11.005","article-title":"Perceptual learning rules based on reinforcers and attention","volume":"14","author":"Roelfsema","year":"2010","journal-title":"Trends in Cognitive Sciences"},{"key":"2021073020593236800_R46","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1080\/13506285.2015.1010462","article-title":"A learning rule that explains how rewards teach attention","volume":"23","author":"Rombouts","year":"2015","journal-title":"Visual Cognition"},{"key":"2021073020593236800_R47","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1038\/nn2066","article-title":"Choice, uncertainty and value in prefrontal and cingulate cortex","volume":"11","author":"Rushworth","year":"2008","journal-title":"Nature Neuroscience"},{"key":"2021073020593236800_R48","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1016\/j.neuron.2011.05.014","article-title":"Frontal cortex and reward-guided learning and decision-making","volume":"70","author":"Rushworth","year":"2011","journal-title":"Neuron"},{"key":"2021073020593236800_R49","doi-asserted-by":"crossref","first-page":"668","DOI":"10.3389\/fnhum.2013.00668","article-title":"Learning what to expect (in visual perception)","volume":"7","author":"Series","year":"2013","journal-title":"Frontiers in Human Neuroscience"},{"key":"2021073020593236800_R50","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.conb.2008.07.010","article-title":"Anchors, scales and the relative coding of value in the brain","volume":"18","author":"Seymour","year":"2008","journal-title":"Current Opinion in Neurobiology"},{"key":"2021073020593236800_R51","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: An integrative theory of anterior cingulate cortex function","volume":"79","author":"Shenhav","year":"2013","journal-title":"Neuron"},{"key":"2021073020593236800_R52","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.conb.2013.12.004","article-title":"Reinforcement learning and human behavior","volume":"25C","author":"Shteingart","year":"2014","journal-title":"Current Opinion in Neurobiology"},{"key":"2021073020593236800_R53","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1162\/089976603765202622","article-title":"Estimating a state-space model from point process observations","volume":"15","author":"Smith","year":"2003","journal-title":"Neural Computation"},{"key":"2021073020593236800_R54","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1523\/JNEUROSCI.2908-03.2004","article-title":"Dynamic analysis of learning in behavioral experiments","volume":"24","author":"Smith","year":"2004","journal-title":"Journal of Neuroscience"},{"key":"2021073020593236800_R55","doi-asserted-by":"crossref","first-page":"1782","DOI":"10.1126\/science.1094765","article-title":"Matching behavior and the representation of value in the parietal cortex","volume":"304","author":"Sugrue","year":"2004","journal-title":"Science"},{"key":"2021073020593236800_R56","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1016\/j.tics.2009.06.003","article-title":"Expectation (and attention) in visual cognition","volume":"13","author":"Summerfield","year":"2009","journal-title":"Trends in Cognitive Sciences"},{"key":"2021073020593236800_R57","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1167\/11.5.5","article-title":"Eye guidance in natural vision: Reinterpreting salience","volume":"11","author":"Tatler","year":"2011","journal-title":"Journal of Vision"},{"key":"2021073020593236800_R58","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9780262015417.001.0001","volume-title":"A computational perspective on visual attention","author":"Tsotsos","year":"2011","edition":"1st ed."},{"key":"2021073020593236800_R59","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1177\/1073858411435128","article-title":"Information processing in decision-making systems","volume":"18","author":"van der Meer","year":"2012","journal-title":"The Neuroscientist"},{"key":"2021073020593236800_R60","first-page":"189","article-title":"Inferring relevance in a changing world","volume":"5","author":"Wilson","year":"2011","journal-title":"Frontiers in Human Neuroscience"},{"key":"2021073020593236800_R61","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1126\/science.1084324","article-title":"Single neurons in the monkey hippocampus and learning of new associations","volume":"300","author":"Wirth","year":"2003","journal-title":"Science"},{"key":"2021073020593236800_R62","doi-asserted-by":"crossref","DOI":"10.1016\/j.tins.2015.08.009","article-title":"Long-range attention networks: Circuit motifs underlying endogenously controlled stimulus selection","author":"Womelsdorf","year":"2015","journal-title":"Trends in Neurosciences"},{"key":"2021073020593236800_R63","doi-asserted-by":"crossref","first-page":"15005","DOI":"10.1073\/pnas.1002258107","article-title":"Economic choices can be made using only stimulus values","volume":"107","author":"Wunderlich","year":"2010","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/direct.mit.edu\/jocn\/article-pdf\/28\/2\/333\/1950604\/jocn_a_00894.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/direct.mit.edu\/jocn\/article-pdf\/28\/2\/333\/1950604\/jocn_a_00894.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T10:05:34Z","timestamp":1718273134000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/28\/2\/333\/28469\/Attentional-Selection-Can-Be-Predicted-by"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,2,1]]},"references-count":63,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2016,2,1]]},"published-print":{"date-parts":[[2016,2,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_00894","relation":{},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2016,2]]},"published":{"date-parts":[[2016,2,1]]}}}