{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T22:35:31Z","timestamp":1790807731706,"version":"4.1.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1014052","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T00:00:00Z","timestamp":1773619200000}}],"reference-count":56,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T00:00:00Z","timestamp":1772755200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Reinforcement learning models can be combined with sequential sampling models to fit choice-RT data. The combined models, known as RL-SSMs, explain a wide range of choice-RT patterns in repeated decision tasks. The present study shows how constraining an RL-SSM with eye gaze data can further enhance its predictive ability. Our model allows learned option values and relative gaze to jointly influence the accumulation of evidence prior to choice. We evaluate the model on data from two eye-tracking experiments (total N\u2009=\u2009133) and test several variants of the model that assume different mechanisms for integrating values and gaze at the decision stage. Further, we show that it captures a variety of empirical effects, including gaze biases on choice and response time, as well as individual differences in absolute versus relative valuation. The model can be used to understand how learned option values interact with visual attention to influence choice, joining together two major (but mostly separate) modeling traditions.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1014052","type":"journal-article","created":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T18:52:33Z","timestamp":1772823153000},"page":"e1014052","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["A reinforcement learning and sequential sampling model constrained by gaze data"],"prefix":"10.1371","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5378-656X","authenticated-orcid":true,"given":"William M.","family":"Hayes","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Melanie J.","family":"Touchard","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,3,6]]},"reference":[{"key":"pcbi.1014052.ref001","doi-asserted-by":"crossref","first-page":"107261","DOI":"10.1016\/j.neuropsychologia.2019.107261","article-title":"Mutual benefits: combining reinforcement learning with sequential sampling models","volume":"136","author":"S Mileti\u0107","year":"2020","journal-title":"Neuropsychologia"},{"issue":"1","key":"pcbi.1014052.ref002","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1177\/17456916231179138","article-title":"The future of decisions from experience: connecting real-world decision problems to cognitive processes","volume":"19","author":"S Olschewski","year":"2024","journal-title":"Perspect Psychol Sci"},{"key":"pcbi.1014052.ref003","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"1998"},{"key":"pcbi.1014052.ref004","volume-title":"Individual Choice behaviour","author":"RD Luce","year":"1959"},{"issue":"1","key":"pcbi.1014052.ref005","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/0001-6918(77)90012-9","article-title":"Speed-accuracy tradeoff and information processing dynamics","volume":"41","author":"WA Wickelgren","year":"1977","journal-title":"Acta Psychologica"},{"issue":"4","key":"pcbi.1014052.ref006","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1016\/j.tics.2016.01.007","article-title":"Diffusion decision model: current issues and history","volume":"20","author":"R Ratcliff","year":"2016","journal-title":"Trends Cogn Sci"},{"issue":"12","key":"pcbi.1014052.ref007","doi-asserted-by":"crossref","first-page":"1810","DOI":"10.1037\/xge0000482","article-title":"Attention and choice across domains","volume":"147","author":"SM Smith","year":"2018","journal-title":"J Exp Psychol Gen"},{"key":"pcbi.1014052.ref008","author":"R Cecchi","year":"2025"},{"issue":"6","key":"pcbi.1014052.ref009","doi-asserted-by":"crossref","first-page":"3291","DOI":"10.1073\/pnas.1911348117","article-title":"Uncertainty in learning, choice, and visual fixation","volume":"117","author":"H Stoji\u0107","year":"2020","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"4","key":"pcbi.1014052.ref010","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1162\/neco.2008.12-06-420","article-title":"The diffusion decision model: theory and data for two-choice decision tasks","volume":"20","author":"R Ratcliff","year":"2008","journal-title":"Neural Comput"},{"issue":"3","key":"pcbi.1014052.ref011","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1037\/0033-295X.100.3.432","article-title":"Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment","volume":"100","author":"JR Busemeyer","year":"1993","journal-title":"Psychol Rev"},{"issue":"3","key":"pcbi.1014052.ref012","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1037\/0033-295X.108.3.550","article-title":"The time course of perceptual choice: the leaky, competing accumulator model","volume":"108","author":"M Usher","year":"2001","journal-title":"Psychol Rev"},{"issue":"3","key":"pcbi.1014052.ref013","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.cogpsych.2007.12.002","article-title":"The simplest complete model of choice response time: linear ballistic accumulation","volume":"57","author":"SD Brown","year":"2008","journal-title":"Cogn Psychol"},{"issue":"10","key":"pcbi.1014052.ref014","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1038\/nn.2635","article-title":"Visual fixations and the computation and comparison of value in simple choice","volume":"13","author":"I Krajbich","year":"2010","journal-title":"Nat Neurosci"},{"issue":"33","key":"pcbi.1014052.ref015","doi-asserted-by":"crossref","first-page":"13852","DOI":"10.1073\/pnas.1101328108","article-title":"Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions","volume":"108","author":"I Krajbich","year":"2011","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1014052.ref016","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1016\/j.copsyc.2018.10.008","article-title":"Accounting for attention in sequential sampling models of decision making","volume":"29","author":"I Krajbich","year":"2019","journal-title":"Curr Opin Psychol"},{"key":"pcbi.1014052.ref017","doi-asserted-by":"crossref","first-page":"1384713","DOI":"10.3389\/frbhe.2024.1384713","article-title":"Unraveling information processes of decision-making with eye-tracking data","volume":"3","author":"CC Ting","year":"2024","journal-title":"Front Behav Econ"},{"issue":"5","key":"pcbi.1014052.ref018","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1017\/S1930297500000413","article-title":"Biasing simple choices by manipulating relative visual attention","volume":"3","author":"KC Armel","year":"2008","journal-title":"Judgm Decis Mak"},{"issue":"4","key":"pcbi.1014052.ref019","doi-asserted-by":"crossref","first-page":"1476","DOI":"10.1037\/a0035813","article-title":"Eye tracking and pupillometry are indicators of dissociable latent decision processes","volume":"143","author":"JF Cavanagh","year":"2014","journal-title":"J Exp Psychol Gen"},{"issue":"6","key":"pcbi.1014052.ref020","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1038\/s41562-019-0584-8","article-title":"Gaze bias differences capture individual choice behaviour","volume":"3","author":"AW Thomas","year":"2019","journal-title":"Nat Hum Behav"},{"issue":"40","key":"pcbi.1014052.ref021","doi-asserted-by":"crossref","DOI":"10.1073\/pnas.1304429110","article-title":"Simultaneous modeling of visual saliency and value computation improves predictions of economic choice","volume":"110","author":"RB Towal","year":"2013","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"1","key":"pcbi.1014052.ref022","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1177\/0956797618810521","article-title":"Gaze amplifies value in decision making","volume":"30","author":"SM Smith","year":"2019","journal-title":"Psychol Sci"},{"key":"pcbi.1014052.ref023","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1002\/bdm.1854","article-title":"Eye movements in risky choice","volume":"29","author":"N Stewart","year":"2016","journal-title":"J Behav Decis Mak"},{"issue":"2","key":"pcbi.1014052.ref024","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1523\/JNEUROSCI.2036-14.2015","article-title":"fMRI and EEG predictors of dynamic decision parameters during human reinforcement learning","volume":"35","author":"MJ Frank","year":"2015","journal-title":"J Neurosci"},{"issue":"4","key":"pcbi.1014052.ref025","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.3758\/s13423-016-1199-y","article-title":"The drift diffusion model as the choice rule in reinforcement learning","volume":"24","author":"ML Pedersen","year":"2017","journal-title":"Psychon Bull Rev"},{"issue":"4","key":"pcbi.1014052.ref026","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.3758\/s13423-018-1554-2","article-title":"A reinforcement learning diffusion decision model for value-based decisions","volume":"26","author":"L Fontanesi","year":"2019","journal-title":"Psychon Bull Rev"},{"issue":"3","key":"pcbi.1014052.ref027","doi-asserted-by":"crossref","first-page":"490","DOI":"10.3758\/s13415-019-00723-1","article-title":"Decomposing the effects of context valence and feedback information on speed and accuracy during reinforcement learning: a meta-analytical approach using diffusion decision modeling","volume":"19","author":"L Fontanesi","year":"2019","journal-title":"Cogn Affect Behav Neurosci"},{"issue":"1","key":"pcbi.1014052.ref028","doi-asserted-by":"crossref","first-page":"20","DOI":"10.3758\/s13423-020-01774-z","article-title":"Modeling the influence of working memory, reinforcement, and action uncertainty on reaction time and choice during instrumental learning","volume":"28","author":"SD McDougle","year":"2021","journal-title":"Psychon Bull Rev"},{"issue":"3","key":"pcbi.1014052.ref029","doi-asserted-by":"crossref","first-page":"868","DOI":"10.3758\/s13423-019-01570-4","article-title":"Combining error-driven models of associative learning with evidence accumulation models of decision-making","volume":"26","author":"DK Sewell","year":"2019","journal-title":"Psychon Bull Rev"},{"key":"pcbi.1014052.ref030","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.83891","article-title":"The functional form of value normalization in human reinforcement learning","volume":"12","author":"S Bavard","year":"2023","journal-title":"Elife"},{"key":"pcbi.1014052.ref031","doi-asserted-by":"crossref","first-page":"105280","DOI":"10.1016\/j.cognition.2022.105280","article-title":"Testing models of context-dependent outcome encoding in reinforcement learning","volume":"230","author":"WM Hayes","year":"2023","journal-title":"Cognition"},{"key":"pcbi.1014052.ref032","doi-asserted-by":"crossref","first-page":"8096","DOI":"10.1038\/ncomms9096","article-title":"Contextual modulation of value signals in reward and punishment learning","volume":"6","author":"S Palminteri","year":"2015","journal-title":"Nat Commun"},{"issue":"16","key":"pcbi.1014052.ref033","doi-asserted-by":"crossref","first-page":"3268","DOI":"10.1523\/JNEUROSCI.1712-19.2020","article-title":"The effect of counterfactual information on outcome value coding in medial prefrontal and cingulate cortex: from an absolute to a relative neural code","volume":"40","author":"D Pischedda","year":"2020","journal-title":"J Neurosci"},{"issue":"8","key":"pcbi.1014052.ref034","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1037\/xlm0001145","article-title":"Reinforcement learning in and out of context: the effects of attentional focus","volume":"49","author":"WM Hayes","year":"2023","journal-title":"J Exp Psychol Learn Mem Cogn"},{"key":"pcbi.1014052.ref035","unstructured":"Touchard MJ, Hayes WM. Can visual fixations explain context-dependent reinforcement learning?. In: Proceedings of the Annual Meeting of the Cognitive Science Society; 2025."},{"key":"pcbi.1014052.ref036","doi-asserted-by":"crossref","first-page":"12438","DOI":"10.1038\/ncomms12438","article-title":"Gaze data reveal distinct choice processes underlying model-based and model-free reinforcement learning","volume":"7","author":"A Konovalov","year":"2016","journal-title":"Nat Commun"},{"issue":"2","key":"pcbi.1014052.ref037","doi-asserted-by":"crossref","first-page":"588","DOI":"10.3758\/s13423-024-02587-0","article-title":"The Tweedledum and Tweedledee of dynamic decisions: Discriminating between diffusion decision and accumulator models","volume":"32","author":"PD Kvam","year":"2025","journal-title":"Psychon Bull Rev"},{"key":"pcbi.1014052.ref038","doi-asserted-by":"crossref","DOI":"10.1037\/14496-000","volume-title":"Stochastic models for learning","author":"RR Bush","year":"1955"},{"issue":"3","key":"pcbi.1014052.ref039","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1037\/0033-295X.109.3.472","article-title":"Toward a method of selecting among computational models of cognition","volume":"109","author":"MA Pitt","year":"2002","journal-title":"Psychol Rev"},{"issue":"2","key":"pcbi.1014052.ref040","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.jmp.2006.01.004","article-title":"Accumulative prediction error and the selection of time series models","volume":"50","author":"EJ Wagenmakers","year":"2006","journal-title":"J Math Psychol"},{"issue":"2","key":"pcbi.1014052.ref041","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s11299-025-00329-w","article-title":"Human reinforcement learning processes and biases: computational characterization and possible applications to behavioral public policy","volume":"24","author":"S Palminteri","year":"2025","journal-title":"Mind Soc"},{"key":"pcbi.1014052.ref042","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cobeha.2021.06.006","article-title":"Context-dependent outcome encoding in human reinforcement learning","volume":"41","author":"S Palminteri","year":"2021","journal-title":"Curr Opin Behav Sci"},{"issue":"1","key":"pcbi.1014052.ref043","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jcps.2011.10.002","article-title":"Relative visual saliency differences induce sizable bias in consumer choice","volume":"22","author":"M Milosavljevic","year":"2012","journal-title":"J Consum Psychol"},{"key":"pcbi.1014052.ref044","author":"JS Trueblood","year":"2022","journal-title":"PsyArXiv"},{"issue":"1","key":"pcbi.1014052.ref045","doi-asserted-by":"crossref","first-page":"6163","DOI":"10.1038\/s41467-024-50214-2","article-title":"Gaze-centered gating, reactivation, and reevaluation of economic value in orbitofrontal cortex","volume":"15","author":"D Ferro","year":"2024","journal-title":"Nat Commun"},{"issue":"5","key":"pcbi.1014052.ref046","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1016\/j.neuron.2016.04.036","article-title":"Decision making and sequential sampling from memory","volume":"90","author":"MN Shadlen","year":"2016","journal-title":"Neuron"},{"issue":"1","key":"pcbi.1014052.ref047","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.tics.2021.10.006","article-title":"Magnitude-sensitivity: rethinking decision-making","volume":"26","author":"A Pirrone","year":"2022","journal-title":"Trends Cogn Sci"},{"issue":"6484","key":"pcbi.1014052.ref048","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1126\/science.aaz5891","article-title":"Dopamine promotes cognitive effort by biasing the benefits versus costs of cognitive work","volume":"367","author":"A Westbrook","year":"2020","journal-title":"Science"},{"key":"pcbi.1014052.ref049","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cogpsych.2018.02.002","article-title":"Modeling 2-alternative forced-choice tasks: accounting for both magnitude and difference effects","volume":"103","author":"R Ratcliff","year":"2018","journal-title":"Cogn Psychol"},{"issue":"6","key":"pcbi.1014052.ref050","doi-asserted-by":"crossref","first-page":"634","DOI":"10.1038\/s41562-020-0822-0","article-title":"Value-based attention but not divisive normalization influences decisions with multiple alternatives","volume":"4","author":"S Gluth","year":"2020","journal-title":"Nat Hum Behav"},{"key":"pcbi.1014052.ref051","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cogpsych.2015.11.002","article-title":"A new framework for modeling decisions about changing information: the Piecewise Linear Ballistic Accumulator model","volume":"85","author":"WR Holmes","year":"2016","journal-title":"Cogn Psychol"},{"key":"pcbi.1014052.ref052","volume-title":"afex: Analysis of Factorial Experiments","author":"H Singmann","year":"2024"},{"issue":"3","key":"pcbi.1014052.ref053","article-title":"Random effects structure for confirmatory hypothesis testing: keep it maximal","volume":"68","author":"DJ Barr","year":"2013","journal-title":"J Mem Lang"},{"key":"pcbi.1014052.ref054","doi-asserted-by":"crossref","DOI":"10.7554\/eLife.49547","article-title":"Ten simple rules for the computational modeling of behavioral data","volume":"8","author":"RC Wilson","year":"2019","journal-title":"Elife"},{"key":"pcbi.1014052.ref055","article-title":"Non-decision time: The Higgs boson of decision","author":"A Bompas","year":"2024","journal-title":"Psychological Review"},{"key":"pcbi.1014052.ref056","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v040.i06","article-title":"DEoptim: an R package for global optimization by differential evolution","volume":"40","author":"KM Mullen","year":"2011","journal-title":"J Stat Softw"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1014052","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T00:00:00Z","timestamp":1773619200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014052","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T18:00:53Z","timestamp":1773684053000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1014052"}},"subtitle":[],"editor":[{"given":"Bastien","family":"Blain","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,3,6]]},"references-count":56,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3,6]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1014052","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,6]]}}}