{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T19:16:25Z","timestamp":1791486985493,"version":"4.3.4"},"reference-count":436,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T00:00:00Z","timestamp":1711670400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF\u201119\u20112\u20110026"],"award-info":[{"award-number":["W911NF\u201119\u20112\u20110026"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000183","name":"Army Research Office","doi-asserted-by":"publisher","award":["W911NF\u201116\u20111\u20110474"],"award-info":[{"award-number":["W911NF\u201116\u20111\u20110474"]}],"id":[{"id":"10.13039\/100000183","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01 DA040011"],"award-info":[{"award-number":["R01 DA040011"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["P50 MH094258"],"award-info":[{"award-number":["P50 MH094258"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants\u2014even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011950","type":"journal-article","created":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T20:01:37Z","timestamp":1711742497000},"page":"e1011950","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":7,"title":["Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1872-7614","authenticated-orcid":true,"given":"Jaron T.","family":"Colas","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0016-3531","authenticated-orcid":true,"given":"John P.","family":"O\u2019Doherty","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4015-3151","authenticated-orcid":true,"given":"Scott T.","family":"Grafton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,3,29]]},"reference":[{"issue":"5","key":"pcbi.1011950.ref001","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1037\/h0054388","article-title":"A mathematical model for simple learning.","volume":"58","author":"RR Bush","year":"1951","journal-title":"Psychol Rev"},{"key":"pcbi.1011950.ref002","first-page":"64","volume-title":"Classical conditioning II: Current research and theory.","author":"RA Rescorla","year":"1972"},{"key":"pcbi.1011950.ref003","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"1998"},{"issue":"6","key":"pcbi.1011950.ref004","doi-asserted-by":"crossref","first-page":"1100","DOI":"10.1177\/1745691617693393","article-title":"Choosing prediction over explanation in psychology: lessons from machine learning.","volume":"12","author":"T Yarkoni","year":"2017","journal-title":"Perspect Psychol Sci"},{"key":"pcbi.1011950.ref005","article-title":"Predicting human decisions with behavioral theories and machine learning.","author":"O Plonsky","journal-title":"arXiv. 2019"},{"issue":"1","key":"pcbi.1011950.ref006","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1146\/annurev-psych-010416-044216","article-title":"Learning, reward, and decision making.","volume":"68","author":"JP O\u2019Doherty","year":"2017","journal-title":"Annu Rev Psychol"},{"key":"pcbi.1011950.ref007","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.neubiorev.2020.10.022","article-title":"Why and how the brain weights contributions from a mixture of experts","volume":"123","author":"JP O\u2019Doherty","year":"2021","journal-title":"Neurosci Biobehav Rev"},{"issue":"1","key":"pcbi.1011950.ref008","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1038\/s41386-021-01108-0","article-title":"Reinforcement-learning in fronto-striatal circuits","volume":"47","author":"B Averbeck","year":"2022","journal-title":"Neuropsychopharmacology"},{"issue":"4","key":"pcbi.1011950.ref009","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"J Gl\u00e4scher","year":"2010","journal-title":"Neuron"},{"key":"pcbi.1011950.ref010","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1038\/s41562-017-0180-8","article-title":"The successor representation in human reinforcement learning.","volume":"1","author":"I Momennejad","year":"2017","journal-title":"Nat Hum Behav"},{"issue":"47","key":"pcbi.1011950.ref011","doi-asserted-by":"crossref","first-page":"29381","DOI":"10.1073\/pnas.1912330117","article-title":"Computational evidence for hierarchically structured reinforcement learning in humans","volume":"117","author":"MK Eckstein","year":"2020","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"15","key":"pcbi.1011950.ref012","doi-asserted-by":"crossref","first-page":"4750","DOI":"10.1002\/hbm.25988","article-title":"Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T","volume":"43","author":"JT Colas","year":"2022","journal-title":"Hum Brain Mapp"},{"key":"pcbi.1011950.ref013","first-page":"3","volume-title":"Decision making, affect, and learning: attention and performance XXIII","author":"ND Daw","year":"2011"},{"issue":"32","key":"pcbi.1011950.ref014","doi-asserted-by":"crossref","first-page":"8360","DOI":"10.1523\/JNEUROSCI.1010-06.2006","article-title":"The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans","volume":"26","author":"AN Hampton","year":"2006","journal-title":"J Neurosci"},{"issue":"4","key":"pcbi.1011950.ref015","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1016\/j.neuron.2007.07.022","article-title":"Contributions of the amygdala to reward expectancy and choice signals in human prefrontal cortex","volume":"55","author":"AN Hampton","year":"2007","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011950.ref016","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1093\/cercor\/bhn098","article-title":"Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making","volume":"19","author":"J Gl\u00e4scher","year":"2009","journal-title":"Cereb Cortex"},{"issue":"3","key":"pcbi.1011950.ref017","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1901\/jeab.2005.110-04","article-title":"Dynamic response-by-response models of matching behavior in rhesus monkeys.","volume":"84","author":"B Lau","year":"2005","journal-title":"J Exp Anal Behav"},{"issue":"47","key":"pcbi.1011950.ref018","doi-asserted-by":"crossref","first-page":"12860","DOI":"10.1523\/JNEUROSCI.2496-07.2007","article-title":"Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making","volume":"27","author":"T Sch\u00f6nberg","year":"2007","journal-title":"J Neurosci"},{"issue":"43","key":"pcbi.1011950.ref019","doi-asserted-by":"crossref","first-page":"13524","DOI":"10.1523\/JNEUROSCI.2469-09.2009","article-title":"Human reinforcement learning subdivides structured action spaces by learning effector-specific values","volume":"29","author":"SJ Gershman","year":"2009","journal-title":"J Neurosci"},{"issue":"48","key":"pcbi.1011950.ref020","doi-asserted-by":"crossref","first-page":"15104","DOI":"10.1523\/JNEUROSCI.3524-09.2009","article-title":"Dopaminergic drugs modulate learning rates and perseveration in Parkinson\u2019s patients in a dynamic foraging task","volume":"29","author":"RB Rutledge","year":"2009","journal-title":"J Neurosci"},{"issue":"10","key":"pcbi.1011950.ref021","doi-asserted-by":"crossref","first-page":"e1005810","DOI":"10.1371\/journal.pcbi.1005810","article-title":"Distinct prediction errors in mesostriatal circuits of the human brain mediate learning about the values of both states and actions: evidence from high-resolution fMRI","volume":"13","author":"JT Colas","year":"2017","journal-title":"PLOS Comput Biol"},{"issue":"5669","key":"pcbi.1011950.ref022","doi-asserted-by":"crossref","first-page":"452","DOI":"10.1126\/science.1094285","article-title":"Dissociable roles of ventral and dorsal striatum in instrumental conditioning","volume":"304","author":"JP O\u2019Doherty","year":"2004","journal-title":"Science"},{"issue":"4","key":"pcbi.1011950.ref023","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1007\/BF02288967","article-title":"Stimulus and response generalization: a stochastic model relating generalization to distance in psychological space.","volume":"22","author":"RN Shepard","year":"1957","journal-title":"Psychometrika"},{"key":"pcbi.1011950.ref024","volume-title":"Individual choice behavior: a theoretical analysis","author":"RD Luce","year":"1959"},{"issue":"3","key":"pcbi.1011950.ref025","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/0022-2496(77)90032-3","article-title":"The choice axiom after twenty years.","volume":"15","author":"RD Luce","year":"1977","journal-title":"J Math Psychol"},{"key":"pcbi.1011950.ref026","volume-title":"Cognitive modeling.","author":"JR Busemeyer","year":"2010"},{"issue":"6","key":"pcbi.1011950.ref027","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.tics.2017.03.011","article-title":"The importance of falsification in computational cognitive modeling.","volume":"21","author":"S Palminteri","year":"2017","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011950.ref028","doi-asserted-by":"crossref","first-page":"e49547","DOI":"10.7554\/eLife.49547","article-title":"Ten simple rules for the computational modeling of behavioral data.","volume":"8","author":"RC Wilson","year":"2019","journal-title":"eLife."},{"issue":"3","key":"pcbi.1011950.ref029","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1177\/2167702614565359","article-title":"Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification","volume":"3","author":"TV Wiecki","year":"2015","journal-title":"Clinical Psychol Sci"},{"key":"pcbi.1011950.ref030","first-page":"1277","volume-title":"Proceedings of the 35th Annual Conference of the Cognitive Science Society","author":"B Scheibehenne","year":"2013"},{"key":"pcbi.1011950.ref031","doi-asserted-by":"crossref","first-page":"391","DOI":"10.3758\/s13423-014-0684-4","article-title":"Using Bayesian hierarchical parameter estimation to assess the generalizability of cognitive models of choice.","volume":"22","author":"B Scheibehenne","year":"2015","journal-title":"Psychon Bull Rev"},{"issue":"6","key":"pcbi.1011950.ref032","doi-asserted-by":"crossref","first-page":"2047","DOI":"10.3758\/s13423-018-1446-5","article-title":"The relative merit of empirical priors in non-identifiable and sloppy models: applications to models of learning and decision-making.","volume":"25","author":"MS Spektor","year":"2018","journal-title":"Psychon Bull Rev"},{"key":"pcbi.1011950.ref033","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.jneumeth.2019.01.006","article-title":"Joint modeling of reaction times and choice improves parameter identifiability in reinforcement learning models","volume":"317","author":"IC Ballard","year":"2019","journal-title":"J Neurosci Methods"},{"key":"pcbi.1011950.ref034","first-page":"4e2d9","article-title":"reliability","author":"K Katahira","year":"2022","journal-title":"Canof computational models be truly improved by placing priors on parametersPsyArXiv"},{"key":"pcbi.1011950.ref035","article-title":"Troubleshooting Bayesian cognitive models. Psychol Methods.","author":"B Baribault","year":"2023"},{"issue":"1","key":"pcbi.1011950.ref036","doi-asserted-by":"crossref","first-page":"122","DOI":"10.2307\/1416673","article-title":"On the fallacy of imputing the correlations found for groups to the individuals or smaller groups composing them.","volume":"52","author":"EL Thorndike","year":"1939","journal-title":"Am J Psychol"},{"issue":"3","key":"pcbi.1011950.ref037","doi-asserted-by":"crossref","first-page":"351","DOI":"10.2307\/2087176","article-title":"Ecological correlations and the behavior of individuals.","volume":"15","author":"WS Robinson","year":"1950","journal-title":"Am Sociol Rev"},{"issue":"6","key":"pcbi.1011950.ref038","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1086\/222356","article-title":"Durkheim\u2019s Suicide and problems of empirical research.","volume":"63","author":"HC Selvin","year":"1958","journal-title":"Am J Sociol."},{"issue":"3","key":"pcbi.1011950.ref039","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1901\/jeab.2005.23-05","article-title":"Linear-nonlinear-Poisson models of primate choice dynamics.","volume":"84","author":"GS Corrado","year":"2005","journal-title":"J Exp Anal Behav"},{"key":"pcbi.1011950.ref040","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/j.jmp.2015.03.006","article-title":"The relation between reinforcement learning parameters and the influence of reinforcement history on choice behavior.","volume":"66","author":"K. Katahira","year":"2015","journal-title":"J Math Psychol"},{"key":"pcbi.1011950.ref041","article-title":"Pseudo-learning effects in reinforcement learning model-based analysis: a problem of misspecification of initial preference.","author":"K Katahira","year":"2017","journal-title":"PsyArXiv"},{"key":"pcbi.1011950.ref042","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.jmp.2018.09.002","article-title":"The statistical structures of reinforcement learning with asymmetric value updates.","volume":"87","author":"K. Katahira","year":"2018","journal-title":"J Math Psychol"},{"key":"pcbi.1011950.ref043","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.jmp.2019.03.007","article-title":"Biases in estimating the balance between model-free and model-based learning systems due to model misspecification.","volume":"91","author":"A Toyama","year":"2019","journal-title":"J Math Psychol"},{"key":"pcbi.1011950.ref044","doi-asserted-by":"crossref","first-page":"3574","DOI":"10.1038\/s41598-020-80593-7","article-title":"Dissociation between asymmetric value updating and perseverance in human reinforcement learning.","volume":"11","author":"M Sugawara","year":"2021","journal-title":"Sci Rep"},{"key":"pcbi.1011950.ref045","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1007\/s42113-022-00145-2","article-title":"Influences of reinforcement and choice histories on choice behavior in actor-critic learning","volume":"6","author":"K Katahira","year":"2023","journal-title":"Comput Brain Behav"},{"issue":"1","key":"pcbi.1011950.ref046","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1037\/bne0000541","article-title":"Choice-confirmation bias and gradual perseveration in human reinforcement learning.","volume":"137","author":"S. Palminteri","year":"2023","journal-title":"Behav Neurosci."},{"key":"pcbi.1011950.ref047","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1007\/s42113-023-00175-4","article-title":"Examinations of biases by model misspecification and parameter reliability of reinforcement learning models","volume":"6","author":"A Toyama","year":"2023","journal-title":"Comput Brain Behav"},{"issue":"1","key":"pcbi.1011950.ref048","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1006\/jmps.1999.1283","article-title":"The importance of complexity in model selection.","volume":"44","author":"IJ Myung","year":"2000","journal-title":"J Math Psychol"},{"issue":"1","key":"pcbi.1011950.ref049","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","article-title":"Adaptive mixtures of local experts.","volume":"3","author":"RA Jacobs","year":"1991","journal-title":"Neural Comput"},{"issue":"6","key":"pcbi.1011950.ref050","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1162\/089976602753712972","article-title":"Multiple model-based reinforcement learning.","volume":"14","author":"K Doya","year":"2002","journal-title":"Neural Comput"},{"issue":"8","key":"pcbi.1011950.ref051","doi-asserted-by":"crossref","first-page":"1177","DOI":"10.1109\/TNNLS.2012.2200299","article-title":"Twenty years of mixture of experts","volume":"23","author":"SE Yuksel","year":"2012","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"pcbi.1011950.ref052","article-title":"Metacontrol for adaptive imagination-based optimization.","author":"JB Hamrick","year":"2017","journal-title":"arXiv"},{"key":"pcbi.1011950.ref053","article-title":"Outrageously large neural networks: the sparsely-gated mixture-of-experts layer.","author":"N Shazeer","year":"2017","journal-title":"arXiv"},{"key":"pcbi.1011950.ref054","article-title":"A review of sparse expert models in deep learning.","author":"W Fedus","year":"2022","journal-title":"arXiv"},{"issue":"5180","key":"pcbi.1011950.ref055","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1126\/science.8091209","article-title":"The basal ganglia and adaptive motor control","volume":"265","author":"AM Graybiel","year":"1994","journal-title":"Science"},{"issue":"6623","key":"pcbi.1011950.ref056","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1038\/386392a0","article-title":"Modular decomposition in visuomotor learning","volume":"386","author":"Z Ghahramani","year":"1997","journal-title":"Nature"},{"issue":"12","key":"pcbi.1011950.ref057","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"ND Daw","year":"2005","journal-title":"Nat Neurosci"},{"issue":"31","key":"pcbi.1011950.ref058","doi-asserted-by":"crossref","first-page":"9861","DOI":"10.1523\/JNEUROSCI.6157-08.2009","article-title":"Validation of decision-making models and analysis of decision variables in the rat basal ganglia","volume":"29","author":"M Ito","year":"2009","journal-title":"J Neurosci"},{"issue":"47","key":"pcbi.1011950.ref059","doi-asserted-by":"crossref","first-page":"14701","DOI":"10.1523\/JNEUROSCI.2728-09.2009","article-title":"Role of striatum in updating values of chosen actions","volume":"29","author":"H Kim","year":"2009","journal-title":"J Neurosci"},{"issue":"3","key":"pcbi.1011950.ref060","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/j.cub.2014.12.002","article-title":"Activation of dorsal raphe serotonergic neurons promotes waiting but is not reinforcing","volume":"25","author":"MS Fonseca","year":"2015","journal-title":"Curr Biol"},{"issue":"15","key":"pcbi.1011950.ref061","doi-asserted-by":"crossref","first-page":"e2113961119","DOI":"10.1073\/pnas.2113961119","article-title":"Mice exhibit stochastic and efficient action switching during probabilistic decision making","volume":"119","author":"CC Beron","year":"2022","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"6","key":"pcbi.1011950.ref062","doi-asserted-by":"crossref","first-page":"716","DOI":"10.1109\/TAC.1974.1100705","article-title":"A new look at the statistical model identification","volume":"19","author":"H. Akaike","year":"1974","journal-title":"IEEE Trans Automat Contr"},{"issue":"2","key":"pcbi.1011950.ref063","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1093\/biomet\/76.2.297","article-title":"Regression and time series model selection in small samples","volume":"76","author":"CM Hurvich","year":"1989","journal-title":"Biometrika"},{"key":"pcbi.1011950.ref064","doi-asserted-by":"crossref","DOI":"10.1037\/10976-000","volume-title":"The fundamentals of learning.","author":"EL Thorndike","year":"1932"},{"issue":"1989","key":"pcbi.1011950.ref065","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1126\/science.77.1989.173.b","article-title":"A proof of the law of effect","volume":"77","author":"EL Thorndike","year":"1933","journal-title":"Science"},{"issue":"5703","key":"pcbi.1011950.ref066","doi-asserted-by":"crossref","first-page":"1940","DOI":"10.1126\/science.1102941","article-title":"By carrot or by stick: cognitive reinforcement learning in parkinsonism","volume":"306","author":"MJ Frank","year":"2004","journal-title":"Science"},{"issue":"23","key":"pcbi.1011950.ref067","doi-asserted-by":"crossref","first-page":"R941","DOI":"10.1016\/j.cub.2011.10.030","article-title":"The optimism bias","volume":"21","author":"T. Sharot","year":"2011","journal-title":"Curr Biol"},{"issue":"11","key":"pcbi.1011950.ref068","doi-asserted-by":"crossref","first-page":"1475","DOI":"10.1038\/nn.2949","article-title":"How unrealistic optimism is maintained in the face of reality","volume":"14","author":"T Sharot","year":"2011","journal-title":"Nat Neurosci"},{"issue":"4\u20136","key":"pcbi.1011950.ref069","first-page":"603","article-title":"Opponent interactions between serotonin and dopamine.","volume":"15","author":"ND Daw","year":"2002","journal-title":"Neural Netw"},{"issue":"41","key":"pcbi.1011950.ref070","doi-asserted-by":"crossref","first-page":"16311","DOI":"10.1073\/pnas.0706111104","article-title":"Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning","volume":"104","author":"MJ Frank","year":"2007","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"8","key":"pcbi.1011950.ref071","doi-asserted-by":"crossref","first-page":"1062","DOI":"10.1038\/nn.2342","article-title":"Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation","volume":"12","author":"MJ Frank","year":"2009","journal-title":"Nat Neurosci"},{"issue":"2","key":"pcbi.1011950.ref072","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1523\/JNEUROSCI.5498-10.2012","article-title":"Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain","volume":"32","author":"Y Niv","year":"2012","journal-title":"J Neurosci"},{"issue":"6","key":"pcbi.1011950.ref073","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1007\/s00422-013-0571-5","article-title":"Adaptive properties of differential learning rates for positive and negative outcomes.","volume":"107","author":"RD Caz\u00e9","year":"2013","journal-title":"Biol Cybern"},{"issue":"4","key":"pcbi.1011950.ref074","doi-asserted-by":"crossref","first-page":"0067","DOI":"10.1038\/s41562-017-0067","article-title":"Behavioural and neural characterization of optimistic reinforcement learning.","volume":"1","author":"G Lefebvre","year":"2017","journal-title":"Nat Hum Behav"},{"issue":"8","key":"pcbi.1011950.ref075","doi-asserted-by":"crossref","first-page":"e1005684","DOI":"10.1371\/journal.pcbi.1005684","article-title":"Confirmation bias in human reinforcement learning: evidence from counterfactual feedback processing.","volume":"13","author":"S Palminteri","year":"2017","journal-title":"PLOS Comput Biol"},{"issue":"7","key":"pcbi.1011950.ref076","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.tics.2022.04.005","article-title":"The computational roots of positivity and confirmation biases in reinforcement learning.","volume":"26","author":"S Palminteri","year":"2022","journal-title":"Trends Cogn Sci"},{"issue":"4","key":"pcbi.1011950.ref077","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.1037\/rev0000424","article-title":"Linking confidence biases to reinforcement-learning processes.","volume":"130","author":"N Salem-Garcia","year":"2023","journal-title":"Psychol Rev"},{"issue":"1","key":"pcbi.1011950.ref078","doi-asserted-by":"crossref","first-page":"6896","DOI":"10.1038\/s41467-023-42589-5","article-title":"Neural and computational underpinnings of biased confidence in human reinforcement learning","volume":"14","author":"CC Ting","year":"2023","journal-title":"Nat Commun."},{"key":"pcbi.1011950.ref079","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jmp.2016.01.006","article-title":"Empirical priors for reinforcement learning models.","volume":"71","author":"SJ Gershman","year":"2016","journal-title":"J Math Psychol"},{"issue":"10","key":"pcbi.1011950.ref080","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1038\/s41562-020-0919-5","article-title":"Information about action outcomes differentially affects learning from self-determined versus imposed choices.","volume":"4","author":"V Chambon","year":"2020","journal-title":"Nat Hum Behav"},{"key":"pcbi.1011950.ref081","first-page":"547684","article-title":"Brain-wide representations of prior information in mouse decision-making.","author":"International Brain Laboratory","year":"2023","journal-title":"bioRxiv"},{"issue":"9","key":"pcbi.1011950.ref082","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"TE Behrens","year":"2007","journal-title":"Nat Neurosci"},{"issue":"42","key":"pcbi.1011950.ref083","doi-asserted-by":"crossref","first-page":"17951","DOI":"10.1073\/pnas.0905191106","article-title":"Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions","volume":"106","author":"LK Krugel","year":"2009","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"37","key":"pcbi.1011950.ref084","doi-asserted-by":"crossref","first-page":"12366","DOI":"10.1523\/JNEUROSCI.0822-10.2010","article-title":"An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment","volume":"30","author":"MR Nassar","year":"2010","journal-title":"J Neurosci"},{"issue":"4","key":"pcbi.1011950.ref085","doi-asserted-by":"crossref","first-page":"e1003015","DOI":"10.1371\/journal.pcbi.1003015","article-title":"A healthy fear of the unknown: perspectives on the interpretation of parameter fits from computational models in neuroscience","volume":"9","author":"MR Nassar","year":"2013","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref086","article-title":"Adaptive optimal training of animal behavior.","volume":"29","author":"JH Bak","year":"2016","journal-title":"Adv Neural Inf Process Syst."},{"key":"pcbi.1011950.ref087","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.cobeha.2016.04.003","article-title":"Taming the beast: extracting generalizable knowledge from computational models of cognition.","volume":"11","author":"MR Nassar","year":"2016","journal-title":"Curr Opin Behav Sci"},{"key":"pcbi.1011950.ref088","article-title":"Efficient inference for time-varying behavior during learning.","author":"NA Roy","year":"2018","journal-title":"Adv Neural Inf Process Syst."},{"issue":"4","key":"pcbi.1011950.ref089","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1016\/j.neuron.2020.12.004","article-title":"International Brain Laboratory, Akrami A, Brody CD, Pillow JW. Extracting the dynamics of behavior in sensory decision-making experiments","volume":"109","author":"NA Roy","year":"2021","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011950.ref090","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1038\/s41593-021-01007-z","article-title":"International Brain Laboratory, Urai AE, Churchland AK, Pouget A, Pillow JW. Mice alternate between discrete strategies during perceptual decision-making","volume":"25","author":"ZC Ashwood","year":"2022","journal-title":"Nat Neurosci"},{"key":"pcbi.1011950.ref091","doi-asserted-by":"crossref","first-page":"e86491","DOI":"10.7554\/eLife.86491","article-title":"Tracking subjects\u2019 strategies in behavioural choice experiments at trial resolution","volume":"13","author":"S Maggi","year":"2024","journal-title":"eLife"},{"key":"pcbi.1011950.ref092","article-title":"Dissecting the complexities of learning with infinite hidden Markov models","author":"International Brain Laboratory","year":"2023","journal-title":"bioRxiv"},{"issue":"9","key":"pcbi.1011950.ref093","doi-asserted-by":"crossref","first-page":"e1011430","DOI":"10.1371\/journal.pcbi.1011430","article-title":"Mixtures of strategies underlie rodent behavior during reversal learning.","volume":"19","author":"NM Le","year":"2023","journal-title":"PLOS Comput Biol"},{"issue":"9","key":"pcbi.1011950.ref094","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1038\/nn.4613","article-title":"Dorsal hippocampus contributes to model-based planning","volume":"20","author":"KJ Miller","year":"2017","journal-title":"Nat Neurosci"},{"issue":"32","key":"pcbi.1011950.ref095","doi-asserted-by":"crossref","first-page":"15871","DOI":"10.1073\/pnas.1821647116","article-title":"NSPN Consortium, Dolan RJ. Credit assignment to state-independent task representations and its relationship with model-based decision making","volume":"116","author":"N Shahar","year":"2019","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1011950.ref096","article-title":"From predictive models to cognitive models: separable behavioral processes underlying reward learning in the rat.","author":"KJ Miller","year":"2018","journal-title":"bioRxiv"},{"issue":"1","key":"pcbi.1011950.ref097","first-page":"1","article-title":"Assigning the right credit to the wrong action: compulsivity in the general population is associated with augmented outcome-irrelevant value-based learning.","volume":"11","author":"NSPN Consortium","year":"2021","journal-title":"Transl Psychiatry."},{"key":"pcbi.1011950.ref098","doi-asserted-by":"crossref","first-page":"e64575","DOI":"10.7554\/eLife.64575","article-title":"Value representations in the rodent orbitofrontal cortex drive learning, not choice.","volume":"11","author":"KJ Miller","year":"2022","journal-title":"eLife."},{"key":"pcbi.1011950.ref099","article-title":"Computational and neural mechanisms underlying the influence of action affordances on value-based choice","author":"Yi S","year":"2023","journal-title":"bioRxiv"},{"issue":"1","key":"pcbi.1011950.ref100","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.neuroimage.2012.04.024","article-title":"Go and no-go learning in reward and punishment: interactions between affect and effect.","volume":"62","author":"M Guitart-Masip","year":"2012","journal-title":"NeuroImage."},{"issue":"5","key":"pcbi.1011950.ref101","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1007\/s00213-013-3313-4","article-title":"Differential, but not opponent, effects of L-DOPA and citalopram on action learning with reward and punishment","volume":"231","author":"M Guitart-Masip","year":"2014","journal-title":"Psychopharmacology"},{"issue":"10","key":"pcbi.1011950.ref102","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.1162\/jocn_a_01224","article-title":"Pavlovian control of escape and avoidance.","volume":"30","author":"AJ Millner","year":"2018","journal-title":"J Cogn Neurosci"},{"issue":"2","key":"pcbi.1011950.ref103","doi-asserted-by":"crossref","first-page":"e1008553","DOI":"10.1371\/journal.pcbi.1008553","article-title":"Neural signatures of arbitration between Pavlovian and instrumental action selection.","volume":"17","author":"SJ Gershman","year":"2021","journal-title":"PLOS Comput Biol"},{"issue":"7","key":"pcbi.1011950.ref104","doi-asserted-by":"crossref","first-page":"1343","DOI":"10.1080\/02699931.2022.2109600","article-title":"The effects of induced positive and negative affect on Pavlovian-instrumental interactions.","volume":"36","author":"ID Weber","year":"2022","journal-title":"Cogn Emot"},{"key":"pcbi.1011950.ref105","first-page":"eb697","article-title":"Improving the reliability of the Pavlovian go\/no-go task.","author":"S Zorowitz","year":"2023","journal-title":"PsyArXiv"},{"key":"pcbi.1011950.ref106","doi-asserted-by":"crossref","first-page":"2000","DOI":"10.3389\/fpsyg.2017.02000","article-title":"Learning where to look for high value improves decision making asymmetrically.","volume":"8","author":"JT Colas","year":"2017","journal-title":"Front Psychol"},{"issue":"3","key":"pcbi.1011950.ref107","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1348\/000711009X477581","article-title":"Separating response-execution bias from decision bias: arguments for an additional parameter in Ratcliff\u2019s diffusion model.","volume":"63","author":"A Voss","year":"2010","journal-title":"Br J Math Stat Psychol"},{"issue":"31","key":"pcbi.1011950.ref108","doi-asserted-by":"crossref","first-page":"11351","DOI":"10.1523\/JNEUROSCI.6689-10.2011","article-title":"The detection of visual contrast in the behaving mouse","volume":"31","author":"L Busse","year":"2011","journal-title":"J Neurosci"},{"issue":"1","key":"pcbi.1011950.ref109","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/srep07569","article-title":"Stimulus similarity determines the prevalence of behavioral laterality in a visual discrimination task for mice.","volume":"4","author":"M. Trevi\u00f1o","year":"2014","journal-title":"Sci Rep"},{"key":"pcbi.1011950.ref110","doi-asserted-by":"crossref","first-page":"147138","DOI":"10.1016\/j.brainres.2020.147138","article-title":"Distributed processing of side-choice biases","volume":"1749","author":"M Trevi\u00f1o","year":"2020","journal-title":"Brain Res"},{"key":"pcbi.1011950.ref111","doi-asserted-by":"crossref","first-page":"99","DOI":"10.3389\/fnbeh.2020.00099","article-title":"Adaptive choice biases in mice and humans.","volume":"14","author":"M Trevi\u00f1o","year":"2020","journal-title":"Front Behav Neurosci"},{"issue":"1","key":"pcbi.1011950.ref112","doi-asserted-by":"crossref","first-page":"e0245890","DOI":"10.1371\/journal.pone.0245890","article-title":"Isomorphic decisional biases across perceptual tasks.","volume":"16","author":"M Trevi\u00f1o","year":"2021","journal-title":"PLOS ONE"},{"key":"pcbi.1011950.ref113","doi-asserted-by":"crossref","first-page":"6486","DOI":"10.1038\/s41598-023-33008-2","article-title":"Decision heuristics in contexts integrating action selection and execution.","volume":"13","author":"NM Dundon","year":"2023","journal-title":"Sci Rep"},{"issue":"1","key":"pcbi.1011950.ref114","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1016\/0028-3932(71)90067-4","article-title":"The assessment and analysis of handedness: the Edinburgh inventory.","volume":"9","author":"RC Oldfield","year":"1971","journal-title":"Neuropsychologia."},{"issue":"3","key":"pcbi.1011950.ref115","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1006\/nimg.1996.0073","article-title":"Asymmetry in the human motor cortex and handedness.","volume":"4","author":"K Amunts","year":"1996","journal-title":"NeuroImage."},{"issue":"3","key":"pcbi.1011950.ref116","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/S0028-3932(99)00075-5","article-title":"Interhemispheric asymmetry of the human motor cortex related to handedness and gender.","volume":"38","author":"K Amunts","year":"2000","journal-title":"Neuropsychologia."},{"issue":"5","key":"pcbi.1011950.ref117","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1016\/S0028-3932(99)00120-7","article-title":"The effects of hand preference and gender on finger tapping performance asymmetry by the use of an infra-red light measurement device.","volume":"38","author":"SL Schmidt","year":"2000","journal-title":"Neuropsychologia."},{"issue":"3","key":"pcbi.1011950.ref118","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s00221-002-1181-y","article-title":"Motor sequence learning with the nondominant left hand","volume":"146","author":"ST Grafton","year":"2002","journal-title":"Exp Brain Res"},{"issue":"10","key":"pcbi.1011950.ref119","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1038\/nn.2635","article-title":"Visual fixations and the computation and comparison of value in simple choice","volume":"13","author":"I Krajbich","year":"2010","journal-title":"Nat Neurosci"},{"issue":"33","key":"pcbi.1011950.ref120","doi-asserted-by":"crossref","first-page":"13852","DOI":"10.1073\/pnas.1101328108","article-title":"Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions","volume":"108","author":"I Krajbich","year":"2011","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"2","key":"pcbi.1011950.ref121","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1257\/aer.101.2.900","article-title":"Search dynamics in consumer choice under time pressure: an eye-tracking study.","volume":"101","author":"E Reutskaja","year":"2011","journal-title":"Am Econ Rev"},{"issue":"1","key":"pcbi.1011950.ref122","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/S0028-3932(99)00045-7","article-title":"Pseudoneglect: a review and meta-analysis of performance factors in line bisection tasks.","volume":"38","author":"G Jewell","year":"2000","journal-title":"Neuropsychologia."},{"issue":"3","key":"pcbi.1011950.ref123","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1212\/WNL.30.3.327","article-title":"Right hemisphere dominance for attention: the mechanism underlying hemispheric asymmetries of inattention (neglect).","volume":"30","author":"KM Heilman","year":"1980","journal-title":"Neurology"},{"issue":"4","key":"pcbi.1011950.ref124","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1002\/ana.410100402","article-title":"A cortical network for directed attention and unilateral neglect","volume":"10","author":"MM Mesulam","year":"1981","journal-title":"Ann Neurol"},{"issue":"6","key":"pcbi.1011950.ref125","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1002\/dev.20166","article-title":"The evolutionary psychology of left and right: costs and benefits of lateralization.","volume":"48","author":"G. Vallortigara","year":"2006","journal-title":"Dev Psychobiol"},{"issue":"10","key":"pcbi.1011950.ref126","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1038\/nn.2905","article-title":"A lateralized brain network for visuospatial attention","volume":"14","author":"MT de Schotten","year":"2011","journal-title":"Nat Neurosci"},{"issue":"4","key":"pcbi.1011950.ref127","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/0926-6410(93)90005-P","article-title":"Influence of reading habits on line bisection","volume":"1","author":"S Chokron","year":"1993","journal-title":"Cogn Brain Res"},{"issue":"1","key":"pcbi.1011950.ref128","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0926-6410(95)00018-6","article-title":"Reading habits and line bisection: a developmental approach","volume":"3","author":"S Chokron","year":"1995","journal-title":"Cogn Brain Res"},{"issue":"2","key":"pcbi.1011950.ref129","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/S0926-6410(98)00022-6","article-title":"Scanning direction and line bisection: a study of normal subjects and unilateral neglect patients with opposite reading habits","volume":"7","author":"S Chokron","year":"1998","journal-title":"Cogn Brain Res"},{"issue":"6","key":"pcbi.1011950.ref130","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1016\/0028-3932(84)90098-8","article-title":"Varieties of perseveration.","volume":"22","author":"J Sandson","year":"1984","journal-title":"Neuropsychologia."},{"issue":"11","key":"pcbi.1011950.ref131","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1212\/WNL.37.11.1736","article-title":"Perseveration in behavioral neurology","volume":"37","author":"J Sandson","year":"1987","journal-title":"Neurology"},{"issue":"2","key":"pcbi.1011950.ref132","doi-asserted-by":"crossref","first-page":"151","DOI":"10.3109\/02699059509008188","article-title":"Perseveration. Part I: a review.","volume":"9","author":"G Hotz","year":"1995","journal-title":"Brain Inj"},{"issue":"3","key":"pcbi.1011950.ref133","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1006\/brln.1999.2032","article-title":"Frequency of perseveration in normal subjects","volume":"66","author":"A Ramage","year":"1999","journal-title":"Brain Lang"},{"issue":"47","key":"pcbi.1011950.ref134","doi-asserted-by":"crossref","first-page":"14891","DOI":"10.1523\/JNEUROSCI.4060-09.2009","article-title":"The dorsomedial striatum reflects response bias during learning","volume":"29","author":"EY Kimchi","year":"2009","journal-title":"J Neurosci"},{"key":"pcbi.1011950.ref135","volume-title":"Habits: their definition, neurobiology and role in addiction.","author":"NV Banavar","year":"2024"},{"issue":"4","key":"pcbi.1011950.ref136","first-page":"1","article-title":"Animal intelligence: an experimental study of the associative processes in animals.","volume":"2","author":"EL Thorndike","year":"1898","journal-title":"Psychol Rev Monogr Suppl."},{"key":"pcbi.1011950.ref137","volume-title":"Animal intelligence: experimental studies","author":"EL Thorndike","year":"1911"},{"issue":"1135","key":"pcbi.1011950.ref138","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1098\/rstb.1985.0010","article-title":"Actions and habits: the development of behavioural autonomy","volume":"308","author":"A. Dickinson","year":"1985","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"issue":"2","key":"pcbi.1011950.ref139","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1016\/S0896-6273(02)00963-7","article-title":"Reward, motivation, and reinforcement learning","volume":"36","author":"P Dayan","year":"2002","journal-title":"Neuron"},{"key":"pcbi.1011950.ref140","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1038\/npp.2009.131","article-title":"\u2019Doherty JP. Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action","volume":"35","author":"O Balleine BW","year":"2010","journal-title":"Neuropsychopharmacology"},{"issue":"6","key":"pcbi.1011950.ref141","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1002\/ejsp.674","article-title":"How are habits formed: modelling habit formation in the real world.","volume":"40","author":"P Lally","year":"2010","journal-title":"Eur J Soc Psychol"},{"key":"pcbi.1011950.ref142","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1146\/annurev-psych-122414-033417","article-title":"Psychology of habit.","volume":"67","author":"W Wood","year":"2016","journal-title":"Annu Rev Psychol"},{"issue":"2","key":"pcbi.1011950.ref143","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1037\/rev0000120","article-title":"Habits without values.","volume":"126","author":"KJ Miller","year":"2019","journal-title":"Psychol Rev"},{"key":"pcbi.1011950.ref144","doi-asserted-by":"crossref","first-page":"e53262","DOI":"10.7554\/eLife.53262","article-title":"Dopamine role in learning and action inference.","volume":"9","author":"R. Bogacz","year":"2020","journal-title":"eLife."},{"issue":"4","key":"pcbi.1011950.ref145","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1007\/BF02686603","article-title":"Spontaneous alternation behavior in animals: a review.","volume":"5","author":"CL Richman","year":"1986","journal-title":"Curr Psychol Res Rev"},{"issue":"1","key":"pcbi.1011950.ref146","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/S0149-7634(01)00041-0","article-title":"The neurobiological basis of spontaneous alternation.","volume":"26","author":"R. Lalonde","year":"2002","journal-title":"Neurosci Biobehav Rev"},{"issue":"1","key":"pcbi.1011950.ref147","first-page":"1","article-title":"Motor cortex activity predicts response alternation during sensorimotor decisions.","volume":"7","author":"AA Pape","year":"2016","journal-title":"Nat Commun"},{"issue":"1","key":"pcbi.1011950.ref148","first-page":"1","article-title":"Motor actions influence subsequent sensorimotor decisions.","volume":"7","author":"AA Pape","year":"2017","journal-title":"Sci Rep"},{"issue":"4","key":"pcbi.1011950.ref149","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1037\/0278-7393.11.1-4.675","article-title":"On the ability to inhibit simple thoughts and actions: II. Stop-signal studies of repetition priming.","volume":"11","author":"GD Logan","year":"1985","journal-title":"J Exp Psychol Learn Mem Cogn"},{"issue":"6","key":"pcbi.1011950.ref150","doi-asserted-by":"crossref","first-page":"1573","DOI":"10.1016\/j.neuropsychologia.2008.05.019","article-title":"Hand path priming in manual obstacle avoidance: rapid decay of dorsal stream information.","volume":"47","author":"SA Jax","year":"2009","journal-title":"Neuropsychologia."},{"issue":"1","key":"pcbi.1011950.ref151","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/a0026192","article-title":"Repetition effects in grasping","volume":"66","author":"P Dixon","year":"2012","journal-title":"Can J Exp Psychol"},{"key":"pcbi.1011950.ref152","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s00221-013-3410-y","article-title":"Perseveration effects in reaching and grasping rely on motor priming and not perception","volume":"226","author":"S Glover","year":"2013","journal-title":"Exp Brain Res"},{"key":"pcbi.1011950.ref153","doi-asserted-by":"crossref","first-page":"566","DOI":"10.3758\/s13423-013-0504-2","article-title":"Hand selection for object grasping is influenced by recent motor history.","volume":"21","author":"KF Valyear","year":"2014","journal-title":"Psychon Bull Rev"},{"issue":"2","key":"pcbi.1011950.ref154","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1080\/00222895.2014.961891","article-title":"Two routes to the same action: an action repetition priming study.","volume":"47","author":"J Randerath","year":"2015","journal-title":"J Mot Behav"},{"key":"pcbi.1011950.ref155","doi-asserted-by":"crossref","first-page":"305","DOI":"10.3758\/s13423-018-1510-1","article-title":"Now and then: hand choice is influenced by recent action history.","volume":"26","author":"KF Valyear","year":"2019","journal-title":"Psychon Bull Rev."},{"issue":"24","key":"pcbi.1011950.ref156","doi-asserted-by":"crossref","first-page":"13494","DOI":"10.1073\/pnas.93.24.13494","article-title":"Neural mechanisms for visual memory and their role in attention","volume":"93","author":"R. Desimone","year":"1996","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"1","key":"pcbi.1011950.ref157","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.tics.2005.11.006","article-title":"Repetition and the brain: neural models of stimulus-specific effects","volume":"10","author":"K Grill-Spector","year":"2006","journal-title":"Trends Cogn Sci"},{"issue":"4","key":"pcbi.1011950.ref158","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1016\/j.humov.2007.05.009","article-title":"Evidence for a distributed hierarchy of action representation in the brain","volume":"26","author":"ST Grafton","year":"2007","journal-title":"Hum Mov Sci"},{"issue":"9","key":"pcbi.1011950.ref159","doi-asserted-by":"crossref","first-page":"2898","DOI":"10.1002\/hbm.20717","article-title":"Repetition suppression for performed hand gestures revealed by fMRI","volume":"30","author":"AF Hamilton","year":"2009","journal-title":"Hum Brain Mapp"},{"issue":"11","key":"pcbi.1011950.ref160","doi-asserted-by":"crossref","first-page":"2736","DOI":"10.1093\/cercor\/bhp049","article-title":"Movement-specific repetition suppression in ventral and dorsal premotor cortex during action observation","volume":"19","author":"J Majdand\u017ei\u0107","year":"2009","journal-title":"Cereb Cortex"},{"issue":"3","key":"pcbi.1011950.ref161","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1016\/j.neuroimage.2006.06.005","article-title":"Post-movement beta rebound is generated in motor cortex: evidence from neuromagnetic recordings.","volume":"32","author":"MT Jurkiewicz","year":"2006","journal-title":"NeuroImage."},{"issue":"8","key":"pcbi.1011950.ref162","doi-asserted-by":"crossref","first-page":"1512","DOI":"10.3758\/BF03212152","article-title":"Inhibition of return in manual and saccadic response systems.","volume":"62","author":"KA Briand","year":"2000","journal-title":"Percept Psychophys."},{"issue":"6","key":"pcbi.1011950.ref163","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1038\/nrn1114","article-title":"Exploring the consequences of the previous trial.","volume":"4","author":"JH Fecteau","year":"2003","journal-title":"Nat Rev Neurosci"},{"issue":"1","key":"pcbi.1011950.ref164","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1162\/jocn.2008.20010","article-title":"Inhibition of return arises from inhibition of response processes: an analysis of oscillatory beta activity.","volume":"20","author":"B Past\u00f6tter","year":"2008","journal-title":"J Cogn Neurosci"},{"issue":"3","key":"pcbi.1011950.ref165","doi-asserted-by":"crossref","first-page":"705","DOI":"10.2466\/pms.1964.18.3.705","article-title":"A brief survey of variables that influence random-generation.","volume":"18","author":"GS Tune","year":"1964","journal-title":"Percept Mot Skills"},{"issue":"2","key":"pcbi.1011950.ref166","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1080\/14640746608400019","article-title":"The capacity for generating information by randomization.","volume":"18","author":"AD Baddeley","year":"1966","journal-title":"Q J Exp Psychol"},{"issue":"1","key":"pcbi.1011950.ref167","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1037\/h0032060","article-title":"Generation of random sequences by human subjects: a critical survey of literature.","volume":"77","author":"WA Wagenaar","year":"1972","journal-title":"Psychol Bull."},{"issue":"6","key":"pcbi.1011950.ref168","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1037\/0278-7393.8.6.626","article-title":"Doing the impossible: a note on induction and the experience of randomness.","volume":"8","author":"LL Lopes","year":"1982","journal-title":"J Exp Psychol Learn Mem Cogn"},{"issue":"3","key":"pcbi.1011950.ref169","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/0001-6918(82)90011-7","article-title":"Sequential response bias in randomized response sequences: a computer simulation","volume":"52","author":"S. Wiegersma","year":"1982","journal-title":"Acta Psychol"},{"issue":"4","key":"pcbi.1011950.ref170","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1037\/0096-1523.18.4.1189","article-title":"Not that bad after all: generation of random sequences.","volume":"18","author":"Y. Kareev","year":"1992","journal-title":"J Exp Psychol Hum Percept Perform"},{"issue":"2","key":"pcbi.1011950.ref171","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1037\/0033-295X.109.2.330","article-title":"The production and perception of randomness.","volume":"109","author":"RS Nickerson","year":"2002","journal-title":"Psychol Rev"},{"key":"pcbi.1011950.ref172","doi-asserted-by":"crossref","first-page":"56","DOI":"10.3389\/fpsyg.2012.00056","article-title":"How predictable are \u201cspontaneous decisions\u201d and \u201chidden intentions\u201d? Comparing classification results based on previous responses with multivariate pattern analysis of fMRI BOLD signals.","volume":"3","author":"M Lages","year":"2012","journal-title":"Front Psychol"},{"key":"pcbi.1011950.ref173","article-title":"Sequential dependencies between trials in free choice tasks.","author":"C Allefeld","year":"2013","journal-title":"arXiv"},{"key":"pcbi.1011950.ref174","doi-asserted-by":"crossref","first-page":"1113654","DOI":"10.3389\/fpsyg.2023.1113654","article-title":"Instruction effects on randomness in sequence generation.","volume":"14","author":"M Guseva","year":"2023","journal-title":"Front Psychol"},{"issue":"1","key":"pcbi.1011950.ref175","doi-asserted-by":"crossref","first-page":"e1011739","DOI":"10.1371\/journal.pcbi.1011739","article-title":"Explaining the flaws in human random generation as local sampling with momentum.","volume":"20","author":"L Castillo","year":"2024","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref176","doi-asserted-by":"crossref","first-page":"22","DOI":"10.3389\/fnsys.2011.00022","article-title":"Dopaminergic balance between reward maximization and policy complexity.","volume":"5","author":"N Parush","year":"2011","journal-title":"Front Syst Neurosci"},{"issue":"4","key":"pcbi.1011950.ref177","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1016\/j.neuron.2013.08.030","article-title":"Dissociable effects of dopamine and serotonin on reversal learning","volume":"80","author":"HE den Ouden","year":"2013","journal-title":"Neuron"},{"key":"pcbi.1011950.ref178","first-page":"507572","article-title":"Action prediction error: a value-free dopaminergic teaching signal that drives stable learning.","author":"F Greenstreet","year":"2022","journal-title":"bioRxiv"},{"issue":"3","key":"pcbi.1011950.ref179","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1523\/JNEUROSCI.1003-22.2022","article-title":"Undermatching is a consequence of policy compression","volume":"43","author":"BA Bari","year":"2023","journal-title":"J Neurosci"},{"key":"pcbi.1011950.ref180","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1038\/s41467-023-44358-w","article-title":"Dopamine release in human associative striatum during reversal learning.","volume":"15","author":"F Grill","year":"2024","journal-title":"Nat Commun"},{"key":"pcbi.1011950.ref181","doi-asserted-by":"crossref","first-page":"1302842","DOI":"10.3389\/fnbeh.2023.1302842","article-title":"A reinforcement learning model with choice traces for a progressive ratio schedule.","volume":"17","author":"K Ihara","year":"2024","journal-title":"Front Behav Neurosci"},{"key":"pcbi.1011950.ref182","doi-asserted-by":"crossref","first-page":"e51260","DOI":"10.7554\/eLife.51260","article-title":"Dopaminergic modulation of the exploration\/exploitation trade-off in human decision-making.","volume":"9","author":"K Chakroun","year":"2020","journal-title":"eLife"},{"issue":"17","key":"pcbi.1011950.ref183","doi-asserted-by":"crossref","first-page":"5833","DOI":"10.1523\/JNEUROSCI.0053-12.2012","article-title":"Serotonin selectively modulates reward value in human decision-making","volume":"32","author":"B Seymour","year":"2012","journal-title":"J Neurosci"},{"issue":"5","key":"pcbi.1011950.ref184","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive Hebbian learning","volume":"16","author":"PR Montague","year":"1996","journal-title":"J Neurosci"},{"issue":"5306","key":"pcbi.1011950.ref185","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"issue":"3","key":"pcbi.1011950.ref186","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1152\/physrev.00023.2014","article-title":"Neuronal reward and decision signals: from theories to data","volume":"95","author":"W. Schultz","year":"2015","journal-title":"Physiol Rev"},{"key":"pcbi.1011950.ref187","doi-asserted-by":"crossref","first-page":"81","DOI":"10.3389\/fnins.2011.00081","article-title":"Dopamine and effort-based decision making.","volume":"5","author":"IT Kurniawan","year":"2011","journal-title":"Front Neurosci."},{"key":"pcbi.1011950.ref188","doi-asserted-by":"crossref","first-page":"39","DOI":"10.3389\/fnint.2015.00039","article-title":"Beyond reward prediction errors: the role of dopamine in movement kinematics.","volume":"9","author":"JW Barter","year":"2015","journal-title":"Front Integr Neurosci"},{"issue":"8","key":"pcbi.1011950.ref189","doi-asserted-by":"crossref","first-page":"a021691","DOI":"10.1101\/cshperspect.a021691","article-title":"The striatum: where skills and habits meet.","volume":"7","author":"AM Graybiel","year":"2015","journal-title":"Cold Spring Harb Perspect Biol"},{"issue":"6","key":"pcbi.1011950.ref190","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1016\/j.cell.2015.08.014","article-title":"Dopamine is required for the neural representation and control of movement vigor","volume":"162","author":"B Panigrahi","year":"2015","journal-title":"Cell"},{"issue":"2","key":"pcbi.1011950.ref191","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.tins.2018.10.001","article-title":"What is the relationship between dopamine and effort","volume":"42","author":"ME Walton","year":"2019","journal-title":"Trends Neurosci"},{"key":"pcbi.1011950.ref192","first-page":"537994","article-title":"Force tuning explains changes in phasic dopamine signaling during stimulus-reward learning.","author":"K Bakhurin","year":"2023","journal-title":"bioRxiv"},{"issue":"3","key":"pcbi.1011950.ref193","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1037\/h0041006","article-title":"Postdecision changes in the desirability of alternatives.","volume":"52","author":"JW Brehm","year":"1956","journal-title":"J Abnorm Soc Psychol"},{"key":"pcbi.1011950.ref194","doi-asserted-by":"crossref","DOI":"10.1515\/9781503620766","volume-title":"A theory of cognitive dissonance","author":"L. Festinger","year":"1957"},{"issue":"51","key":"pcbi.1011950.ref195","doi-asserted-by":"crossref","first-page":"22014","DOI":"10.1073\/pnas.1011879108","article-title":"Neural correlates of cognitive dissonance and choice-induced preference change","volume":"107","author":"K Izuma","year":"2010","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1011950.ref196","doi-asserted-by":"crossref","first-page":"31","DOI":"10.3389\/fnins.2012.00031","article-title":"Distinction between externally vs. internally guided decision-making: operational differences, meta-analytical comparisons and their theoretical implications.","volume":"6","author":"T Nakao","year":"2012","journal-title":"Front Neurosci"},{"key":"pcbi.1011950.ref197","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3389\/fpsyg.2013.00041","article-title":"Choice-induced preference change in the free-choice paradigm: a critical methodological review.","volume":"4","author":"K Izuma","year":"2013","journal-title":"Front Psychol"},{"key":"pcbi.1011950.ref198","doi-asserted-by":"crossref","first-page":"32477","DOI":"10.1038\/srep32477","article-title":"Post-response \u03b2\u03b3 power predicts the degree of choice-based learning in internally guided decision-making.","volume":"6","author":"T Nakao","year":"2016","journal-title":"Sci Rep."},{"issue":"1","key":"pcbi.1011950.ref199","doi-asserted-by":"crossref","first-page":"e0244434","DOI":"10.1371\/journal.pone.0244434","article-title":"Computational modeling of choice-induced preference change: a reinforcement-learning-based approach.","volume":"16","author":"J Zhu","year":"2021","journal-title":"PLOS ONE"},{"key":"pcbi.1011950.ref200","doi-asserted-by":"crossref","first-page":"153","DOI":"10.3389\/fnhum.2019.00153","article-title":"Reinforcement learning with parsimonious computation and a forgetting process.","volume":"13","author":"A Toyama","year":"2019","journal-title":"Front Hum Neurosci"},{"issue":"1","key":"pcbi.1011950.ref201","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.neuron.2020.10.013","article-title":"The anterior cingulate cortex predicts future states to mediate model-based action selection","volume":"109","author":"T Akam","year":"2021","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011950.ref202","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1162\/jocn_a_01947","article-title":"Choice type impacts human reinforcement learning.","volume":"35","author":"M Rmus","year":"2023","journal-title":"J Cogn Neurosci"},{"issue":"2","key":"pcbi.1011950.ref203","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1037\/xge0001491","article-title":"The construction and use of cognitive maps in model-based control","volume":"153","author":"AB Karagoz","year":"2024","journal-title":"J Exp Psychol Gen"},{"key":"pcbi.1011950.ref204","first-page":"e50469","article-title":"Temporal chunking as a mechanism for unsupervised learning of task-sets.","volume":"9","author":"F Bouchacourt","year":"2020"},{"key":"pcbi.1011950.ref205","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/bs.plm.2021.02.004","volume-title":"The psychology of learning and motivation","author":"L Lai","year":"2021"},{"key":"pcbi.1011950.ref206","author":"L Lai","year":"2022","journal-title":"Action chunking as policy compression"},{"issue":"1","key":"pcbi.1011950.ref207","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/j.neuron.2013.10.018","article-title":"Autonomous mechanism of internal choice estimate underlies decision inertia","volume":"81","author":"R Akaishi","year":"2014","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011950.ref208","doi-asserted-by":"crossref","first-page":"e89802","DOI":"10.1371\/journal.pone.0089802","article-title":"Hysteresis as an implicit prior in tactile spatial decision making.","volume":"9","author":"SD Thiel","year":"2014","journal-title":"PLOS ONE."},{"key":"pcbi.1011950.ref209","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3389\/fnhum.2015.00261","article-title":"Dissociation in decision bias mechanism between probabilistic information and previous decision.","volume":"9","author":"Y Kaneko","year":"2015","journal-title":"Front Hum Neurosci"},{"issue":"25","key":"pcbi.1011950.ref210","doi-asserted-by":"crossref","first-page":"E3548","DOI":"10.1073\/pnas.1518786113","article-title":"Adaptable history biases in human perceptual decisions","volume":"113","author":"A Abrahamyan","year":"2016","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"4","key":"pcbi.1011950.ref211","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1016\/j.cub.2017.01.006","article-title":"Opposite effects of recent history on perception and decision","volume":"27","author":"M Fritsche","year":"2017","journal-title":"Curr Biol"},{"issue":"10","key":"pcbi.1011950.ref212","doi-asserted-by":"crossref","first-page":"2418","DOI":"10.1523\/JNEUROSCI.2189-17.2017","article-title":"Adaptive history biases result from confidence-weighted accumulation of past choices","volume":"38","author":"A Braun","year":"2018","journal-title":"J Neurosci"},{"key":"pcbi.1011950.ref213","doi-asserted-by":"crossref","first-page":"107123","DOI":"10.1016\/j.isci.2023.107123","article-title":"Probabilistic integration of preceding responses explains response bias in perceptual decision making.","volume":"26","author":"D Schlunegger","year":"2023","journal-title":"iScience"},{"issue":"5","key":"pcbi.1011950.ref214","doi-asserted-by":"crossref","first-page":"1322","DOI":"10.1016\/j.neuron.2013.09.013","article-title":"Neuronal origins of choice variability in economic decisions","volume":"80","author":"C. Padoa-Schioppa","year":"2013","journal-title":"Neuron"},{"issue":"5","key":"pcbi.1011950.ref215","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1017\/S1930297500004575","article-title":"Process dynamics in delay discounting decisions: an attractor dynamics approach.","volume":"11","author":"S Scherbaum","year":"2016","journal-title":"Judgm Decis Mak"},{"key":"pcbi.1011950.ref216","article-title":"Choice history bias in intertemporal choice.","author":"M Schoemann","year":"2020","journal-title":"PsyArXiv."},{"key":"pcbi.1011950.ref217","article-title":"Independent, not irrelevant: trial order causes systematic misestimation of economic choice traits.","author":"NV Banavar","year":"2023","journal-title":"PsyArXiv"},{"issue":"980","key":"pcbi.1011950.ref218","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1038\/206217a0","article-title":"Serial choice reaction-time as a function of response versus signal-and-response repetition","volume":"206","author":"P. Bertelson","year":"1965","journal-title":"Nature"},{"issue":"1","key":"pcbi.1011950.ref219","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1037\/0278-7393.17.1.33","article-title":"Procedural learning: II. Intertrial repetition effects in speeded-choice tasks.","volume":"17","author":"H Pashler","year":"1991","journal-title":"J Exp Psychol Learn Mem Cogn"},{"issue":"4","key":"pcbi.1011950.ref220","doi-asserted-by":"crossref","first-page":"283","DOI":"10.3758\/CABN.2.4.283","article-title":"Mechanisms underlying dependencies of performance on stimulus history in a two-alternative forced-choice task.","volume":"2","author":"RY Cho","year":"2002","journal-title":"Cogn Affect Behav Neurosci"},{"issue":"7","key":"pcbi.1011950.ref221","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1167\/14.7.9","article-title":"Quantifying the effect of intertrial dependence on perceptual decisions.","volume":"14","author":"I Fr\u00fcnd","year":"2014","journal-title":"J Vis"},{"issue":"1","key":"pcbi.1011950.ref222","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-017-01356-z","article-title":"History-based action selection bias in posterior parietal cortex.","volume":"8","author":"EJ Hwang","year":"2017","journal-title":"Nat Commun"},{"issue":"7692","key":"pcbi.1011950.ref223","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1038\/nature25510","article-title":"Posterior parietal cortex represents sensory history and mediates its effects on behaviour","volume":"554","author":"A Akrami","year":"2018","journal-title":"Nature"},{"issue":"12","key":"pcbi.1011950.ref224","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1167\/jov.20.12.9","article-title":"Opposite effects of choice history and evidence history resolve a paradox of sequential choice bias.","volume":"20","author":"E Bosch","year":"2020","journal-title":"J Vis"},{"key":"pcbi.1011950.ref225","article-title":"Choice repetition bias in intertemporal choice: an eye-tracking study.","author":"U Senftleben","year":"2024","journal-title":"PsyArXiv."},{"key":"pcbi.1011950.ref226","volume-title":"The ecological approach to visual perception","author":"JJ Gibson","year":"1979"},{"issue":"1485","key":"pcbi.1011950.ref227","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1098\/rstb.2007.2054","article-title":"Cortical mechanisms of action selection: the affordance competition hypothesis","volume":"362","author":"P. Cisek","year":"2007","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"key":"pcbi.1011950.ref228","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1146\/annurev.neuro.051508.135409","article-title":"Neural mechanisms for interacting with a world full of action choices","volume":"33","author":"P Cisek","year":"2010","journal-title":"Annu Rev Neurosci"},{"issue":"6","key":"pcbi.1011950.ref229","doi-asserted-by":"crossref","first-page":"927","DOI":"10.1016\/j.conb.2012.05.007","article-title":"Making decisions through a distributed consensus","volume":"22","author":"P. Cisek","year":"2012","journal-title":"Curr Opin Neurobiol"},{"issue":"6","key":"pcbi.1011950.ref230","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans\u2019 choices and striatal prediction errors","volume":"69","author":"ND Daw","year":"2011","journal-title":"Neuron"},{"issue":"7","key":"pcbi.1011950.ref231","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.1111\/j.1460-9568.2012.08017.x","article-title":"Generalization of value in reinforcement learning by humans","volume":"35","author":"GE Wimmer","year":"2012","journal-title":"Eur J Neurosci"},{"issue":"11","key":"pcbi.1011950.ref232","doi-asserted-by":"crossref","first-page":"1807","DOI":"10.1162\/jocn_a_00447","article-title":"Neural and psychological maturation of decision-making in adolescence and young adulthood.","volume":"25","author":"A Christakou","year":"2013","journal-title":"J Cogn Neurosci"},{"issue":"3","key":"pcbi.1011950.ref233","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1038\/mp.2014.44","article-title":"Disorders of compulsivity: a common bias towards learning habits","volume":"20","author":"V Voon","year":"2014","journal-title":"Mol Psychiatry"},{"issue":"45","key":"pcbi.1011950.ref234","doi-asserted-by":"crossref","first-page":"14901","DOI":"10.1523\/JNEUROSCI.0204-14.2014","article-title":"Episodic memory encoding interferes with reward learning and decreases striatal prediction errors","volume":"34","author":"GE Wimmer","year":"2014","journal-title":"J Neurosci"},{"issue":"2","key":"pcbi.1011950.ref235","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1162\/jocn_a_00894","article-title":"Attentional selection can be predicted by reinforcement learning of task-relevant stimulus features weighted by value-independent stickiness.","volume":"28","author":"M Balcarras","year":"2016","journal-title":"J Cogn Neurosci"},{"issue":"8","key":"pcbi.1011950.ref236","doi-asserted-by":"crossref","first-page":"e1005090","DOI":"10.1371\/journal.pcbi.1005090","article-title":"When does model-based control pay off?","volume":"12","author":"W Kool","year":"2016","journal-title":"PLOS Comput Biol"},{"issue":"9","key":"pcbi.1011950.ref237","doi-asserted-by":"crossref","first-page":"1321","DOI":"10.1177\/0956797617708288","article-title":"Cost-benefit arbitration between multiple reinforcement-learning systems.","volume":"28","author":"W Kool","year":"2017","journal-title":"Psychol Sci"},{"issue":"48","key":"pcbi.1011950.ref238","doi-asserted-by":"crossref","first-page":"10338","DOI":"10.1523\/JNEUROSCI.0457-18.2018","article-title":"How the level of reward awareness changes the computational and electrophysiological signatures of reinforcement learning","volume":"38","author":"CM Correa","year":"2018","journal-title":"J Neurosci"},{"key":"pcbi.1011950.ref239","article-title":"Differences in learning across the lifespan emerge via resource-rational computations.","author":"R Bruckner","year":"2020","journal-title":"PsyArXiv"},{"issue":"6","key":"pcbi.1011950.ref240","doi-asserted-by":"crossref","first-page":"e1007944","DOI":"10.1371\/journal.pcbi.1007944","article-title":"Combined model-free and model-sensitive reinforcement learning in non-human primates.","volume":"16","author":"B Miranda","year":"2020","journal-title":"PLOS Comput Biol"},{"issue":"1","key":"pcbi.1011950.ref241","doi-asserted-by":"crossref","first-page":"3344","DOI":"10.1038\/s41467-021-23704-w","article-title":"Anatomical dissociation of intracerebral signals for reward and punishment prediction errors in humans.","volume":"12","author":"MC Gueguen","year":"2021","journal-title":"Nat Commun"},{"key":"pcbi.1011950.ref242","doi-asserted-by":"crossref","first-page":"e75474","DOI":"10.7554\/eLife.75474","article-title":"The interpretation of computational model parameters depends on the context.","volume":"11","author":"MK Eckstein","year":"2022","journal-title":"eLife"},{"issue":"25","key":"pcbi.1011950.ref243","doi-asserted-by":"crossref","first-page":"8434","DOI":"10.1523\/JNEUROSCI.5468-11.2012","article-title":"Anterior prefrontal cortex contributes to action selection through tracking of recent reward trends","volume":"32","author":"CK Kovach","year":"2012","journal-title":"J Neurosci"},{"issue":"8","key":"pcbi.1011950.ref244","doi-asserted-by":"crossref","first-page":"2534","DOI":"10.1111\/cogs.12688","article-title":"The outcome-representation learning model: a novel reinforcement learning model of the Iowa gambling task.","volume":"42","author":"N Haines","year":"2018","journal-title":"Cogn Sci"},{"key":"pcbi.1011950.ref245","doi-asserted-by":"crossref","first-page":"2477","DOI":"10.1038\/s41467-018-04840-2","article-title":"An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals.","volume":"9","author":"K Iigaya","year":"2018","journal-title":"Nat Commun"},{"issue":"11","key":"pcbi.1011950.ref246","doi-asserted-by":"crossref","first-page":"e1007475","DOI":"10.1371\/journal.pcbi.1007475","article-title":"Tonic exploration governs both flexibility and lapses.","volume":"15","author":"RB Ebitz","year":"2019","journal-title":"PLOS Comput Biol"},{"issue":"7553","key":"pcbi.1011950.ref247","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning.","volume":"521","author":"Y LeCun","year":"2015","journal-title":"Nature"},{"key":"pcbi.1011950.ref248","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.neunet.2014.09.003","article-title":"Deep learning in neural networks: an overview.","volume":"61","author":"J. Schmidhuber","year":"2015","journal-title":"Neural Netw"},{"key":"pcbi.1011950.ref249","volume-title":"Deep learning.","author":"I Goodfellow","year":"2016"},{"key":"pcbi.1011950.ref250","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.conb.2017.06.003","article-title":"Recurrent neural networks as versatile tools of neuroscience research","volume":"46","author":"O. Barak","year":"2017","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011950.ref251","article-title":"A neural network walks into a lab: towards using deep nets as models for human behavior.","author":"WJ Ma","year":"2020","journal-title":"arXiv"},{"issue":"3","key":"pcbi.1011950.ref252","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon.","volume":"38","author":"G. Tesauro","year":"1995","journal-title":"Commun ACM"},{"issue":"7540","key":"pcbi.1011950.ref253","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"V Mnih","year":"2015","journal-title":"Nature"},{"issue":"6","key":"pcbi.1011950.ref254","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep reinforcement learning: a brief survey","volume":"34","author":"K Arulkumaran","year":"2017","journal-title":"IEEE Signal Process Mag"},{"key":"pcbi.1011950.ref255","article-title":"Deep reinforcement learning: an overview.","author":"Y. Li","year":"2017","journal-title":"arXiv"},{"issue":"4\u20135","key":"pcbi.1011950.ref256","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1177\/0278364918770733","article-title":"The limits and potentials of deep learning for robotics.","volume":"37","author":"N S\u00fcnderhauf","year":"2018","journal-title":"Int J Rob Res"},{"issue":"5","key":"pcbi.1011950.ref257","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1016\/j.tics.2019.02.006","article-title":"Reinforcement learning, fast and slow.","volume":"23","author":"M Botvinick","year":"2019","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011950.ref258","first-page":"590","article-title":"Review of deep reinforcement learning for robot manipulation.","volume":"3","author":"H Nguyen","year":"2019","journal-title":"IEEE Int Conf Robot Comput"},{"issue":"4","key":"pcbi.1011950.ref259","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1016\/j.neuron.2020.06.014","article-title":"Deep reinforcement learning and its neuroscientific implications","volume":"107","author":"M Botvinick","year":"2020","journal-title":"Neuron"},{"issue":"4\u20135","key":"pcbi.1011950.ref260","doi-asserted-by":"crossref","first-page":"698","DOI":"10.1177\/0278364920987859","article-title":"How to train your robot with deep reinforcement learning: lessons we have learned.","volume":"40","author":"J Ibarz","year":"2021","journal-title":"Int J Rob Res"},{"issue":"11","key":"pcbi.1011950.ref261","doi-asserted-by":"crossref","first-page":"1197","DOI":"10.1109\/T-C.1972.223477","article-title":"Learning patterns and pattern sequences by self-organizing nets of threshold elements","volume":"C-21","author":"SI Amari","year":"1972","journal-title":"IEEE Trans Comput"},{"issue":"8","key":"pcbi.1011950.ref262","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"JJ Hopfield","year":"1982","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"6088","key":"pcbi.1011950.ref263","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back-propagating errors","volume":"323","author":"DE Rumelhart","year":"1986","journal-title":"Nature"},{"issue":"8","key":"pcbi.1011950.ref264","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory.","volume":"9","author":"S Hochreiter","year":"1997","journal-title":"Neural Comput"},{"key":"pcbi.1011950.ref265","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation.","author":"K Cho","year":"2014","journal-title":"arXiv"},{"key":"pcbi.1011950.ref266","doi-asserted-by":"crossref","first-page":"e21492","DOI":"10.7554\/eLife.21492","article-title":"Reward-based training of recurrent neural networks for cognitive and value-based tasks.","volume":"6","author":"HF Song","year":"2017","journal-title":"eLife"},{"issue":"6","key":"pcbi.1011950.ref267","doi-asserted-by":"crossref","first-page":"e1006903","DOI":"10.1371\/journal.pcbi.1006903","article-title":"Models that learn how humans learn: the case of decision-making and its disorders.","volume":"15","author":"A Dezfouli","year":"2019","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref268","article-title":"Disentangled behavioural representations.","volume":"32","author":"A Dezfouli","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"pcbi.1011950.ref269","article-title":"On the reliability and generalizability of brain-inspired reinforcement learning algorithms.","author":"D Kim","journal-title":"arXiv. 2020"},{"key":"pcbi.1011950.ref270","doi-asserted-by":"crossref","first-page":"4736","DOI":"10.1038\/s41598-022-08863-0","article-title":"Using deep learning to predict human decisions and using cognitive models to explain deep learning models.","volume":"12","author":"M Fintz","year":"2022","journal-title":"Sci Rep."},{"key":"pcbi.1011950.ref271","article-title":"Predictive and interpretable: combining artificial neural networks and classic cognitive models to understand human learning and decision making.","author":"MK Eckstein","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011950.ref272","article-title":"Long short-term prediction guides human metacognitive reinforcement learning.","author":"D Kim","year":"2023","journal-title":"Res Sq."},{"key":"pcbi.1011950.ref273","doi-asserted-by":"crossref","first-page":"20269","DOI":"10.1038\/s41598-023-46850-1","article-title":"Using deep neural networks as a guide for modeling human planning.","volume":"13","author":"I Kuperwajs","year":"2023","journal-title":"Sci Rep"},{"key":"pcbi.1011950.ref274","article-title":"Automatic discovery of cognitive strategies with tiny recurrent neural networks.","author":"J Li","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011950.ref275","article-title":"Cognitive model discovery via disentangled RNNs","volume":"36","author":"KJ Miller","year":"2023","journal-title":"Adv Neural Inf Process Syst"},{"key":"pcbi.1011950.ref276","article-title":"Artificial neural networks for model identification and parameter estimation in computational cognitive models.","author":"M Rmus","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011950.ref277","article-title":"Human-level reinforcement learning performance of recurrent neural networks is linked to hyperperseveration, not directed exploration.","author":"D Tuzsus","year":"2023","journal-title":"bioRxiv."},{"issue":"1","key":"pcbi.1011950.ref278","doi-asserted-by":"crossref","first-page":"e1011678","DOI":"10.1371\/journal.pcbi.1011678","article-title":"Harnessing the flexibility of neural networks to predict dynamic theoretical parameters underlying human choice behavior.","volume":"20","author":"Y Ger","year":"2024","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref279","first-page":"e90082","article-title":"Using recurrent neural network to estimate irreducible stochasticity in human choice-behavior. eLife.","volume":"13","author":"Y Ger","year":"2024"},{"key":"pcbi.1011950.ref280","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1007\/s42113-018-0019-z","article-title":"Between the devil and the deep blue sea: tensions between scientific judgement and statistical model selection","volume":"2","author":"DJ Navarro","year":"2019","journal-title":"Comput Brain Behav"},{"key":"pcbi.1011950.ref281","article-title":"Visualizing and understanding recurrent networks.","author":"A Karpathy","year":"2015","journal-title":"arXiv"},{"key":"pcbi.1011950.ref282","doi-asserted-by":"crossref","first-page":"171058","DOI":"10.1109\/ACCESS.2020.3023394","article-title":"Reinforcement learning interpretation methods: a survey.","volume":"8","author":"A Alharin","year":"2020","journal-title":"IEEE Access"},{"key":"pcbi.1011950.ref283","first-page":"417","volume-title":"Workshops of the European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2020).","author":"C Molnar"},{"key":"pcbi.1011950.ref284","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/978-3-030-57321-8_5","volume-title":"Machine Learning and Knowledge Extraction: International Cross-Domain Conference (CD-MAKE 2020).","author":"E Puiutta","year":"2020"},{"key":"pcbi.1011950.ref285","article-title":"A survey on interpretable reinforcement learning.","author":"C Glanois","year":"2021","journal-title":"arXiv"},{"key":"pcbi.1011950.ref286","doi-asserted-by":"crossref","first-page":"106685","DOI":"10.1016\/j.knosys.2020.106685","article-title":"Explainability in deep reinforcement learning.","volume":"214","author":"A Heuillet","year":"2021","journal-title":"Knowl Based Syst."},{"issue":"3","key":"pcbi.1011950.ref287","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/JPROC.2021.3060483","article-title":"Explaining deep neural networks and beyond: A review of methods and applications","volume":"109","author":"W Samek","year":"2021","journal-title":"Proc IEEE"},{"issue":"10","key":"pcbi.1011950.ref288","doi-asserted-by":"crossref","first-page":"6795","DOI":"10.1109\/TPAMI.2021.3103132","article-title":"Continuous action reinforcement learning from a mixture of interpretable experts","volume":"44","author":"R Akrour","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"pcbi.1011950.ref289","article-title":"Towards understanding the mixture-of-experts layer in deep learning.","volume":"35","author":"Z Chen","year":"2022","journal-title":"Adv Neural Inf Process Syst."},{"key":"pcbi.1011950.ref290","article-title":"A survey of explainable reinforcement learning.","author":"S Milani","year":"2022","journal-title":"arXiv"},{"key":"pcbi.1011950.ref291","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.neunet.2022.03.022","article-title":"Mo\u00cbT: Mixture of Expert Trees and its application to verifiable reinforcement learning","volume":"151","author":"M Vasi\u0107","year":"2022","journal-title":"Neural Netw"},{"key":"pcbi.1011950.ref292","doi-asserted-by":"crossref","unstructured":"R\u00e4uker T, Ho A, Casper S, Hadfield-Menell D. Toward transparent AI: a survey on interpreting the inner structures of deep neural networks. In: 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). Raleigh (NC): IEEE; 2023. p. 464\u2013483. https:\/\/doi.org\/10.1109\/SaTML54575.2023.00039","DOI":"10.1109\/SaTML54575.2023.00039"},{"issue":"37","key":"pcbi.1011950.ref293","doi-asserted-by":"crossref","first-page":"11560","DOI":"10.1523\/JNEUROSCI.1844-09.2009","article-title":"Decisions in changing conditions: the urgency-gating model","volume":"29","author":"P Cisek","year":"2009","journal-title":"J Neurosci"},{"key":"pcbi.1011950.ref294","article-title":"Choice anticipation as gated accumulation of sensory expectations.","author":"B Caie","year":"2023","journal-title":"bioRxiv"},{"issue":"2","key":"pcbi.1011950.ref295","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1080\/17470216108416478","article-title":"Sequential redundancy and speed in a serial two-choice responding task.","volume":"13","author":"P. Bertelson","year":"1961","journal-title":"Q J Exp Psychol"},{"issue":"4","key":"pcbi.1011950.ref296","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1037\/0096-1523.10.4.581","article-title":"Automatic aftereffects in two-choice reaction time: a mathematical representation of some concepts.","volume":"10","author":"E Soetens","year":"1984","journal-title":"J Exp Psychol Hum Percept Perform"},{"issue":"5","key":"pcbi.1011950.ref297","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1037\/0096-1523.11.5.598","article-title":"Expectancy or automatic facilitation? Separating sequential effects in two-choice reaction time.","volume":"11","author":"E Soetens","year":"1985","journal-title":"J Exp Psychol Hum Percept Perform"},{"issue":"3","key":"pcbi.1011950.ref298","doi-asserted-by":"crossref","first-page":"1382","DOI":"10.1152\/jn.00184.2015","article-title":"A neuro-computational model of economic decisions","volume":"114","author":"A Rustichini","year":"2015","journal-title":"J Neurophysiol"},{"key":"pcbi.1011950.ref299","doi-asserted-by":"crossref","first-page":"e20047","DOI":"10.7554\/eLife.20047","article-title":"Response repetition biases in human perceptual decisions are explained by activity decay in competitive attractor models.","volume":"5","author":"JJ Bonaiuto","year":"2016","journal-title":"eLife"},{"key":"pcbi.1011950.ref300","doi-asserted-by":"crossref","first-page":"102876","DOI":"10.1016\/j.actpsy.2019.102876","article-title":"Choice perseveration in value-based decision making: the impact of inter-trial interval and mood.","volume":"198","author":"U Senftleben","year":"2019","journal-title":"Acta Psychol"},{"issue":"1","key":"pcbi.1011950.ref301","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1177\/1747021820964330","article-title":"To stay or not to stay: the stability of choice perseveration in value-based decision making.","volume":"74","author":"U Senftleben","year":"2021","journal-title":"Q J Exp Psychol"},{"key":"pcbi.1011950.ref302","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.jmp.2016.03.007","article-title":"How hierarchical models improve point estimates of model parameters at the individual level.","volume":"73","author":"K. Katahira","year":"2016","journal-title":"J Math Psychol"},{"key":"pcbi.1011950.ref303","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1162\/CPSY_a_00002","article-title":"Revealing neurocomputational mechanisms of reinforcement learning and decision-making with the hBayesDM package.","volume":"1","author":"WY Ahn","year":"2017","journal-title":"Comput Psychiatr"},{"issue":"6","key":"pcbi.1011950.ref304","doi-asserted-by":"crossref","first-page":"e1007043","DOI":"10.1371\/journal.pcbi.1007043","article-title":"Hierarchical Bayesian inference for concurrent model fitting and comparison for group studies.","volume":"15","author":"P Piray","year":"2019","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref305","doi-asserted-by":"crossref","first-page":"102602","DOI":"10.1016\/j.jmp.2021.102602","article-title":"Hierarchical Bayesian models of reinforcement learning: introduction and comparison to alternative methods.","volume":"105","author":"C van Geen","year":"2021","journal-title":"J Math Psychol"},{"issue":"12","key":"pcbi.1011950.ref306","doi-asserted-by":"crossref","first-page":"e1006679","DOI":"10.1371\/journal.pcbi.1006679","article-title":"Neuroscience in Psychiatry Network Research Consortium. Change, stability, and instability in the Pavlovian guidance of behaviour from adolescence to young adulthood.","volume":"14","author":"M Moutoussis","year":"2018","journal-title":"PLOS Comput Biol."},{"issue":"12","key":"pcbi.1011950.ref307","doi-asserted-by":"crossref","first-page":"5472","DOI":"10.1073\/pnas.1818430116","article-title":"Large-scale analysis of test-retest reliabilities of self-regulation measures","volume":"116","author":"AZ Enkavi","year":"2019","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"2","key":"pcbi.1011950.ref308","doi-asserted-by":"crossref","first-page":"e1006803","DOI":"10.1371\/journal.pcbi.1006803","article-title":"Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling.","volume":"15","author":"NSPN Consortium","year":"2019","journal-title":"PLOS Comput Biol."},{"key":"pcbi.1011950.ref309","first-page":"513","article-title":"Test-retest reliability of canonical reinforcement learning models.","volume":"14","author":"L Weidinger","year":"2019","journal-title":"Conf Cogn Comput Neurosci"},{"issue":"6","key":"pcbi.1011950.ref310","first-page":"601","article-title":"Improving the reliability of computational analyses: model-based planning and its relationship with compulsivity","volume":"5","author":"VM Brown","year":"2020","journal-title":"Biol Psychiatry Cogn Neurosci Neuroimaging"},{"key":"pcbi.1011950.ref311","article-title":"Theoretically informed generative models can advance the psychological and brain sciences: lessons from the reliability paradox.","author":"N Haines","year":"2020","journal-title":"PsyArXiv"},{"key":"pcbi.1011950.ref312","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cobeha.2021.06.004","article-title":"What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience","volume":"41","author":"MK Eckstein","year":"2021","journal-title":"Curr Opin Behav Sci"},{"key":"pcbi.1011950.ref313","article-title":"Test-retest reliability of affective bias tasks.","author":"AC Pike","year":"2022","journal-title":"PsyArXiv"},{"issue":"1","key":"pcbi.1011950.ref314","doi-asserted-by":"crossref","first-page":"189","DOI":"10.5334\/cpsy.89","article-title":"Enhancing the psychometric properties of the Iowa gambling task using full generative modeling.","volume":"6","author":"H Sullivan-Toole","year":"2022","journal-title":"Comput Psychiatr."},{"issue":"6","key":"pcbi.1011950.ref315","doi-asserted-by":"crossref","first-page":"2993","DOI":"10.3758\/s13428-021-01739-7","article-title":"Sufficient reliability of the behavioral and computational readouts of a probabilistic reversal learning task.","volume":"54","author":"M Waltmann","year":"2022","journal-title":"Behav Res Methods"},{"key":"pcbi.1011950.ref316","doi-asserted-by":"crossref","first-page":"105137","DOI":"10.1016\/j.neubiorev.2023.105137","article-title":"Individual differences in computational psychiatry: a review of current challenges.","volume":"148","author":"P Karvelis","year":"2023","journal-title":"Neurosci Biobehav Rev"},{"issue":"1","key":"pcbi.1011950.ref317","doi-asserted-by":"crossref","first-page":"30","DOI":"10.5334\/cpsy.86","article-title":"Reliability of decision-making and reinforcement learning computational parameters.","volume":"7","author":"A Mkrtchian","year":"2023","journal-title":"Comput Psychiatr."},{"key":"pcbi.1011950.ref318","article-title":"Test-retest reliability of reinforcement learning parameters.","author":"JV Schaaf","year":"2023","journal-title":"PsyArXiv"},{"key":"pcbi.1011950.ref319","article-title":"Dynamic computational phenotyping of human cognition.","author":"R Schurr","year":"2023","journal-title":"PsyArXiv"},{"key":"pcbi.1011950.ref320","article-title":"Comparing the test-retest reliability of behavioral, computational and self-reported individual measures of reward and punishment sensitivity in relation to mental health symptoms.","author":"S Vrizzi","year":"2023","journal-title":"PsyArXiv"},{"issue":"5","key":"pcbi.1011950.ref321","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1038\/scientificamerican0577-119","article-title":"Stein\u2019s paradox in statistics","volume":"236","author":"B Efron","year":"1977","journal-title":"Sci Am"},{"issue":"434","key":"pcbi.1011950.ref322","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1080\/01621459.1996.10476919","article-title":"Empirical Bayes methods for combining likelihoods","volume":"91","author":"B. Efron","year":"1996","journal-title":"J Am Stat Assoc"},{"issue":"6","key":"pcbi.1011950.ref323","doi-asserted-by":"crossref","first-page":"544","DOI":"10.1016\/j.neunet.2011.03.001","article-title":"Are computational models of any use to psychiatry?","volume":"24","author":"QJ Huys","year":"2011","journal-title":"Neural Netw"},{"issue":"2","key":"pcbi.1011950.ref324","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1038\/nn.2723","article-title":"From reinforcement learning models to psychiatric and neurological disorders","volume":"14","author":"TV Maia","year":"2011","journal-title":"Nat Neurosci"},{"issue":"1","key":"pcbi.1011950.ref325","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.tics.2011.11.018","article-title":"Computational psychiatry.","volume":"16","author":"PR Montague","year":"2012","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011950.ref326","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.neuroimage.2016.06.038","article-title":"Computational neuroimaging strategies for single patient predictions.","volume":"145","author":"KE Stephan","year":"2017","journal-title":"NeuroImage"},{"key":"pcbi.1011950.ref327","doi-asserted-by":"crossref","first-page":"E18","DOI":"10.1017\/pen.2018.14","article-title":"Computational phenotyping: using models to understand individual differences in personality, development, and mental illness.","volume":"1","author":"EH Patzelt","year":"2018","journal-title":"Personal Neurosci."},{"issue":"8","key":"pcbi.1011950.ref328","first-page":"822","article-title":"From classical methods to generative models: tackling the unreliability of neuroscientific measures in mental health research","volume":"8","author":"N Haines","year":"2023","journal-title":"Biol Psychiatry Cogn Neurosci Neuroimaging"},{"issue":"1","key":"pcbi.1011950.ref329","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1196\/annals.1390.022","article-title":"Model-based fMRI and its application to reward learning and decision making","volume":"1104","author":"JP O\u2019Doherty","year":"2007","journal-title":"Ann N Y Acad Sci"},{"issue":"9","key":"pcbi.1011950.ref330","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1038\/s41562-019-0681-8","article-title":"Assessing inter-individual differences with task-related functional neuroimaging.","volume":"3","author":"M Lebreton","year":"2019","journal-title":"Nat Hum Behav"},{"issue":"2","key":"pcbi.1011950.ref331","doi-asserted-by":"crossref","first-page":"e1008738","DOI":"10.1371\/journal.pcbi.1008738","article-title":"Revisiting the importance of model fitting for model-based fMRI: it does matter in computational psychiatry.","volume":"17","author":"K Katahira","year":"2021","journal-title":"PLOS Comput Biol"},{"issue":"4","key":"pcbi.1011950.ref332","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1038\/npp.2008.175","article-title":"Response perseveration and ventral prefrontal sensitivity to reward and punishment in male problem gamblers and smokers","volume":"34","author":"MB de Ruiter","year":"2009","journal-title":"Neuropsychopharmacology"},{"issue":"5","key":"pcbi.1011950.ref333","doi-asserted-by":"crossref","first-page":"2653","DOI":"10.1152\/jn.90629.2008","article-title":"The relative influences of priors and sensory evidence on an oculomotor decision variable during perceptual learning","volume":"100","author":"JI Gold","year":"2008","journal-title":"J Neurophysiol"},{"issue":"5","key":"pcbi.1011950.ref334","doi-asserted-by":"crossref","first-page":"1456","DOI":"10.1037\/xlm0000111","article-title":"The role of response bias in perceptual learning.","volume":"41","author":"PR Jones","year":"2015","journal-title":"J Exp Psychol Learn Mem Cogn"},{"issue":"1","key":"pcbi.1011950.ref335","first-page":"1","article-title":"Pupil-linked arousal is driven by decision uncertainty and alters serial choice bias.","volume":"8","author":"AE Urai","year":"2017","journal-title":"Nat Commun"},{"key":"pcbi.1011950.ref336","doi-asserted-by":"crossref","first-page":"e46331","DOI":"10.7554\/eLife.46331","article-title":"Choice history biases subsequent evidence accumulation.","volume":"8","author":"AE Urai","year":"2019","journal-title":"eLife"},{"issue":"2","key":"pcbi.1011950.ref337","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1037\/0033-295X.85.2.59","article-title":"A theory of memory retrieval.","volume":"85","author":"R. Ratcliff","year":"1978","journal-title":"Psychol Rev"},{"issue":"3","key":"pcbi.1011950.ref338","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1037\/0033-295X.100.3.432","article-title":"Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment.","volume":"100","author":"JR Busemeyer","year":"1993","journal-title":"Psychol Rev"},{"issue":"3","key":"pcbi.1011950.ref339","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1037\/0033-295X.108.3.550","article-title":"The time course of perceptual choice: the leaky, competing accumulator model.","volume":"108","author":"M Usher","year":"2001","journal-title":"Psychol Rev."},{"issue":"10","key":"pcbi.1011950.ref340","doi-asserted-by":"crossref","first-page":"e0186822","DOI":"10.1371\/journal.pone.0186822","article-title":"Value-based decision making via sequential sampling with hierarchical competition and attentional modulation.","volume":"12","author":"JT Colas","year":"2017","journal-title":"PLOS ONE"},{"key":"pcbi.1011950.ref341","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/10469.001.0001","volume-title":"Cognitive choice modeling","author":"ZJ Wang","year":"2021"},{"key":"pcbi.1011950.ref342","article-title":"A study of the relation of accuracy and speed.","volume":"56","author":"HE Garrett","year":"1922","journal-title":"Arch Psychol."},{"key":"pcbi.1011950.ref343","article-title":"Confidence and speed in the two-category judgment.","volume":"241","author":"DM Johnson","year":"1939","journal-title":"Arch Psychol."},{"key":"pcbi.1011950.ref344","volume-title":"Principles of behavior: an introduction to behavior theory.","author":"CL Hull","year":"1943"},{"issue":"4","key":"pcbi.1011950.ref345","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1037\/a0020198","article-title":"Decision making and the avoidance of cognitive demand","volume":"139","author":"W Kool","year":"2010","journal-title":"J Exp Psychol Gen"},{"issue":"12","key":"pcbi.1011950.ref346","doi-asserted-by":"crossref","first-page":"e51637","DOI":"10.1371\/journal.pone.0051637","article-title":"The decision to engage cognitive control is driven by expected reward-value: neural and behavioral evidence.","volume":"7","author":"ML Dixon","year":"2012","journal-title":"PLOS ONE"},{"issue":"2","key":"pcbi.1011950.ref347","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: an integrative theory of anterior cingulate cortex function","volume":"79","author":"A Shenhav","year":"2013","journal-title":"Neuron"},{"issue":"7","key":"pcbi.1011950.ref348","doi-asserted-by":"crossref","first-page":"e68210","DOI":"10.1371\/journal.pone.0068210","article-title":"What is the subjective cost of cognitive effort? Load, trait, and aging effects revealed by economic preference.","volume":"8","author":"A Westbrook","year":"2013","journal-title":"PLOS ONE."},{"issue":"1","key":"pcbi.1011950.ref349","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1037\/a0031048","article-title":"A labor\/leisure tradeoff in cognitive control","volume":"43","author":"W Kool","year":"2014","journal-title":"J Exp Psychol Gen"},{"key":"pcbi.1011950.ref350","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1146\/annurev-psych-010814-015044","article-title":"Motivation and cognitive control: from behavior to neural mechanism.","volume":"66","author":"M Botvinick","year":"2015","journal-title":"Annu Rev Psychol"},{"key":"pcbi.1011950.ref351","doi-asserted-by":"crossref","first-page":"395","DOI":"10.3758\/s13415-015-0334-y","article-title":"Cognitive effort: A neuroeconomic approach.","volume":"15","author":"A Westbrook","year":"2015","journal-title":"Cogn Affect Behav Neurosci"},{"issue":"10","key":"pcbi.1011950.ref352","doi-asserted-by":"crossref","first-page":"1286","DOI":"10.1038\/nn.4384","article-title":"Dorsal anterior cingulate cortex and the value of control","volume":"19","author":"A Shenhav","year":"2016","journal-title":"Nat Neurosci"},{"issue":"12","key":"pcbi.1011950.ref353","doi-asserted-by":"crossref","first-page":"899","DOI":"10.1038\/s41562-018-0401-9","article-title":"Mental labour.","volume":"2","author":"W Kool","year":"2018","journal-title":"Nat Hum Behav"},{"issue":"4","key":"pcbi.1011950.ref354","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.tics.2018.01.009","article-title":"Hierarchical active inference: a theory of motivated control.","volume":"22","author":"G Pezzulo","year":"2018","journal-title":"Trends Cogn Sci"},{"issue":"9","key":"pcbi.1011950.ref355","doi-asserted-by":"crossref","first-page":"e1007326","DOI":"10.1371\/journal.pcbi.1007326","article-title":"Cost-benefit trade-offs in decision-making and learning.","volume":"15","author":"N Sidarus","year":"2019","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref356","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.neuropsychologia.2018.09.013","article-title":"An information-theoretic perspective on the costs of cognition.","volume":"123","author":"A Z\u00e9non","year":"2019","journal-title":"Neuropsychologia."},{"key":"pcbi.1011950.ref357","doi-asserted-by":"crossref","first-page":"104394","DOI":"10.1016\/j.cognition.2020.104394","article-title":"Origin of perseveration in the trade-off between reward and complexity.","volume":"204","author":"SJ Gershman","year":"2020","journal-title":"Cognition."},{"key":"pcbi.1011950.ref358","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.cobeha.2021.02.015","article-title":"Resource-rational decision making.","volume":"41","author":"R Bhui","year":"2021","journal-title":"Curr Opin Behav Sci"},{"key":"pcbi.1011950.ref359","article-title":"Human decision making balances reward maximization and policy compression.","author":"L Lai","year":"2023","journal-title":"PsyArXiv"},{"issue":"2","key":"pcbi.1011950.ref360","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1037\/h0042769","article-title":"Rational choice and the structure of the environment","volume":"63","author":"HA Simon","year":"1956","journal-title":"Psychol Rev"},{"issue":"1","key":"pcbi.1011950.ref361","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1111\/j.1756-8765.2008.01006.x","article-title":"Homo heuristicus: why biased minds make better inferences.","volume":"1","author":"G Gigerenzer","year":"2009","journal-title":"Top Cogn Sci"},{"key":"pcbi.1011950.ref362","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1146\/annurev-psych-120709-145346","article-title":"Heuristic decision making.","volume":"62","author":"G Gigerenzer","year":"2011","journal-title":"Annu Rev Psychol"},{"issue":"9","key":"pcbi.1011950.ref363","doi-asserted-by":"crossref","first-page":"2407","DOI":"10.1162\/neco.2009.09-08-866","article-title":"Sequential effects in two-choice reaction time tasks: decomposition and synthesis of mechanisms.","volume":"21","author":"J Gao","year":"2009","journal-title":"Neural Comput"},{"key":"pcbi.1011950.ref364","article-title":"Confirmation bias optimizes reward learning.","author":"T Tarantola","year":"2021","journal-title":"bioRxiv"},{"issue":"2","key":"pcbi.1011950.ref365","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1162\/neco_a_01455","article-title":"A normative account of confirmation bias during reinforcement learning.","volume":"34","author":"G Lefebvre","year":"2022","journal-title":"Neural Comput"},{"issue":"5","key":"pcbi.1011950.ref366","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1038\/nn.3689","article-title":"Serial dependence in visual perception","volume":"17","author":"J Fischer","year":"2014","journal-title":"Nat Neurosci"},{"issue":"13","key":"pcbi.1011950.ref367","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1167\/jov.21.13.6","article-title":"Recurrent processing improves occluded object recognition and gives rise to perceptual hysteresis.","volume":"21","author":"MR Ernst","year":"2021","journal-title":"J Vis"},{"issue":"4157","key":"pcbi.1011950.ref368","doi-asserted-by":"crossref","first-page":"1124","DOI":"10.1126\/science.185.4157.1124","article-title":"Judgment under uncertainty: heuristics and biases","volume":"185","author":"A Tversky","year":"1974","journal-title":"Science"},{"issue":"1","key":"pcbi.1011950.ref369","doi-asserted-by":"crossref","first-page":"322","DOI":"10.3758\/s13423-017-1286-8","article-title":"The anchoring bias reflects rational use of cognitive resources.","volume":"25","author":"F Lieder","year":"2018","journal-title":"Psychon Bull Rev"},{"key":"pcbi.1011950.ref370","volume-title":"A dynamic theory of personality","author":"K. Lewin","year":"1935"},{"key":"pcbi.1011950.ref371","doi-asserted-by":"crossref","DOI":"10.1037\/10019-000","volume-title":"Principles of topological psychology","author":"K. Lewin","year":"1936"},{"issue":"4","key":"pcbi.1011950.ref372","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men.","volume":"55","author":"EC Tolman","year":"1948","journal-title":"Psychol Rev."},{"issue":"2","key":"pcbi.1011950.ref373","doi-asserted-by":"crossref","first-page":"490","DOI":"10.1016\/j.neuron.2018.10.002","article-title":"What is a cognitive map? Organizing knowledge for flexible behavior","volume":"100","author":"TE Behrens","year":"2018","journal-title":"Neuron"},{"issue":"4","key":"pcbi.1011950.ref374","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1016\/S0893-6080(02)00047-3","article-title":"Actor-critic models of the basal ganglia: new anatomical and computational perspectives.","volume":"15","author":"D Joel","year":"2002","journal-title":"Neural Netw"},{"key":"pcbi.1011950.ref375","first-page":"91","volume-title":"Recent breakthroughs in basal ganglia research.Nova Science","author":"ND Daw","year":"2006"},{"issue":"43","key":"pcbi.1011950.ref376","doi-asserted-by":"crossref","first-page":"13465","DOI":"10.1523\/JNEUROSCI.1500-09.2009","article-title":"Brain hemispheres selectively track the expected value of contralateral options","volume":"29","author":"S Palminteri","year":"2009","journal-title":"J Neurosci"},{"issue":"40","key":"pcbi.1011950.ref377","doi-asserted-by":"crossref","first-page":"17199","DOI":"10.1073\/pnas.0901077106","article-title":"Neural computations underlying action-based decision making in the human brain","volume":"106","author":"K Wunderlich","year":"2009","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1011950.ref378","article-title":"Motor system-dependent effects of amygdala and ventral striatum lesions on explore-exploit behaviors","author":"F Giarrocco","year":"2023","journal-title":"J Neurosci"},{"issue":"8","key":"pcbi.1011950.ref379","doi-asserted-by":"crossref","first-page":"e0136084","DOI":"10.1371\/journal.pone.0136084","article-title":"Undesirable choice biases with small differences in the spatial structure of chance stimulus sequences","volume":"10","author":"D Herrera","year":"2015","journal-title":"PLOS ONE"},{"issue":"1","key":"pcbi.1011950.ref380","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/S1389-0417(01)00039-0","article-title":"A modular neural-network model of the basal ganglia\u2019s role in learning and selecting motor behaviours.","volume":"3","author":"G. Baldassarre","year":"2002","journal-title":"Cogn Syst Res"},{"issue":"2","key":"pcbi.1011950.ref381","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1177\/105971230501300205","article-title":"Actor-critic models of reinforcement learning in the basal ganglia: from natural to artificial rats.","volume":"13","author":"M Khamassi","year":"2005","journal-title":"Adapt Behav"},{"issue":"3","key":"pcbi.1011950.ref382","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"SW Lee","year":"2014","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011950.ref383","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1162\/neco.1994.6.2.181","article-title":"Hierarchical mixtures of experts and the EM algorithm.","volume":"6","author":"MI Jordan","year":"1994","journal-title":"Neural Comput."},{"key":"pcbi.1011950.ref384","doi-asserted-by":"crossref","first-page":"287","DOI":"10.7551\/mitpress\/3122.003.0037","volume-title":"From Animals to Animats 8: Proceedings of the Eighth International Conference on the Simulation of Adaptive Behavior.","author":"E Uchibe","year":"2004"},{"key":"pcbi.1011950.ref385","first-page":"1","volume-title":"International Conference on Statistical Language and Speech Processing (SLSP 2013).","author":"Y. Bengio"},{"key":"pcbi.1011950.ref386","article-title":"Estimating or propagating gradients through stochastic neurons for conditional computation.","author":"Y Bengio","year":"2013","journal-title":"arXiv."},{"key":"pcbi.1011950.ref387","article-title":"Learning factored representations in a deep mixture of experts.","author":"D Eigen","year":"2013","journal-title":"arXiv"},{"key":"pcbi.1011950.ref388","article-title":"Conditional computation in neural networks for faster models.","author":"E Bengio","year":"2015","journal-title":"arXiv"},{"key":"pcbi.1011950.ref389","article-title":"Adaptive computation time for recurrent neural networks.","author":"A. Graves","year":"2016","journal-title":"arXiv"},{"issue":"4","key":"pcbi.1011950.ref390","first-page":"81","article-title":"Terrain-adaptive locomotion skills using deep reinforcement learning.","volume":"35","author":"XB Peng","year":"2016","journal-title":"ACM Trans Graph"},{"key":"pcbi.1011950.ref391","article-title":"MCP: Learning composable hierarchical control with multiplicative compositional policies.","volume":"32","author":"XB Peng","year":"2019","journal-title":"Adv Neural Inf Process Syst."},{"key":"pcbi.1011950.ref392","article-title":"Probabilistic mixture-of-experts for efficient deep reinforcement learning.","author":"J Ren","year":"2021","journal-title":"arXiv"},{"key":"pcbi.1011950.ref393","article-title":"Towards applicable reinforcement learning: improving the generalization and sample efficiency with policy ensemble.","author":"Z Yang","year":"2022","journal-title":"arXiv"},{"issue":"6","key":"pcbi.1011950.ref394","doi-asserted-by":"crossref","first-page":"3812","DOI":"10.1109\/LRA.2023.3271445","article-title":"Multi-task reinforcement learning with attention-based mixture of experts.","volume":"8","author":"G Cheng","year":"2023","journal-title":"IEEE Robot Autom Lett"},{"key":"pcbi.1011950.ref395","article-title":"Multi-task reinforcement learning with mixture of orthogonal experts.","author":"A Hendawy","year":"2023","journal-title":"arXiv"},{"key":"pcbi.1011950.ref396","article-title":"From Google Gemini to OpenAI Q* (Q-star): a survey of reshaping the generative artificial intelligence (AI) research landscape.","author":"TR McIntosh","year":"2023","journal-title":"arXiv"},{"issue":"5025","key":"pcbi.1011950.ref397","doi-asserted-by":"crossref","first-page":"1227","DOI":"10.1126\/science.253.5025.1227","article-title":"New approaches to robotics","volume":"253","author":"RA Brooks","year":"1991","journal-title":"Science"},{"key":"pcbi.1011950.ref398","volume-title":"The artificial life route to artificial intelligence: building embodied, situated agents.","author":"L Steels","year":"1995"},{"key":"pcbi.1011950.ref399","doi-asserted-by":"crossref","first-page":"5","DOI":"10.3389\/fpsyg.2011.00005","article-title":"The mechanics of embodiment: a dialog on embodiment and computational modeling.","volume":"2","author":"G Pezzulo","year":"2011","journal-title":"Front Psychol"},{"issue":"11","key":"pcbi.1011950.ref400","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: a survey.","volume":"32","author":"J Kober","year":"2013","journal-title":"Int J Rob Res"},{"issue":"3","key":"pcbi.1011950.ref401","doi-asserted-by":"crossref","first-page":"122","DOI":"10.3390\/robotics2030122","article-title":"Reinforcement learning in robotics: applications and real-world challenges.","volume":"2","author":"P Kormushev","year":"2013","journal-title":"Robotics."},{"key":"pcbi.1011950.ref402","doi-asserted-by":"crossref","first-page":"612","DOI":"10.3389\/fpsyg.2012.00612","article-title":"Computational grounded cognition: a new alliance between grounded cognition and computational modeling.","volume":"3","author":"G Pezzulo","year":"2013","journal-title":"Front Psychol"},{"key":"pcbi.1011950.ref403","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.cobeha.2018.12.012","article-title":"Decision-making in brains and robots\u2014the case for an interdisciplinary approach.","volume":"26","author":"SW Lee","year":"2019","journal-title":"Curr Opin Behav Sci"},{"issue":"3","key":"pcbi.1011950.ref404","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1038\/s42256-019-0025-4","article-title":"Reinforcement learning in artificial and biological systems","volume":"1","author":"EO Neftci","year":"2019","journal-title":"Nature Machine Intelligence"},{"issue":"4","key":"pcbi.1011950.ref405","doi-asserted-by":"crossref","first-page":"625","DOI":"10.3758\/BF03196322","article-title":"Six views of embodied cognition.","volume":"9","author":"M. Wilson","year":"2002","journal-title":"Psychon Bull Rev"},{"key":"pcbi.1011950.ref406","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1146\/annurev.psych.59.103006.093639","article-title":"Grounded cognition.","volume":"59","author":"LW Barsalou","year":"2008","journal-title":"Annu Rev Psychol"},{"issue":"1","key":"pcbi.1011950.ref407","doi-asserted-by":"crossref","first-page":"178","DOI":"10.3758\/s13428-015-0561-8","article-title":"The DalHouses: 100 new photographs of houses with ratings of typicality, familiarity, and degree of similarity to faces.","volume":"48","author":"JH Filliter","year":"2016","journal-title":"Behav Res Methods"},{"issue":"4","key":"pcbi.1011950.ref408","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1016\/S0019-9958(77)90354-0","article-title":"An adaptive optimal controller for discrete-time Markov environments.","volume":"34","author":"IH Witten","year":"1977","journal-title":"Inf Control"},{"issue":"5","key":"pcbi.1011950.ref409","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TSMC.1983.6313077","article-title":"Neuronlike adaptive elements that can solve difficult learning control problems","volume":"13","author":"AG Barto","year":"1983","journal-title":"IEEE Trans Syst Man Cybern"},{"key":"pcbi.1011950.ref410","unstructured":"Sutton RS. Temporal credit assignment in reinforcement learning [Doctoral dissertation]. Amherst (MA): University of Massachusetts, Amherst; 1984."},{"issue":"1","key":"pcbi.1011950.ref411","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences.","volume":"3","author":"RS Sutton","year":"1988","journal-title":"Mach Learn"},{"issue":"3\u20134","key":"pcbi.1011950.ref412","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/BF00992701","article-title":"The convergence of TD(\u03bb) for general \u03bb.","volume":"8","author":"P. Dayan","year":"1992","journal-title":"Mach Learn"},{"issue":"3","key":"pcbi.1011950.ref413","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1007\/BF00993978","article-title":"TD(\u03bb) converges with probability 1.","volume":"14","author":"P Dayan","year":"1994","journal-title":"Mach Learn"},{"key":"pcbi.1011950.ref414","unstructured":"Watkins CJ. Learning from delayed rewards [Doctoral dissertation]. Cambridge, United Kingdom: University of Cambridge; 1989."},{"issue":"3\u20134","key":"pcbi.1011950.ref415","first-page":"279","volume":"8","author":"CJ Watkins","year":"1992","journal-title":"Q-learning. Mach Learn"},{"key":"pcbi.1011950.ref416","unstructured":"Rummery GA, Niranjan M. On-line Q-learning using connectionist systems. Cambridge, United Kingdom: Department of Engineering, University of Cambridge; 1994. Technical Report No.: CUED\/F-INFENG\/TR 166."},{"issue":"10","key":"pcbi.1011950.ref417","doi-asserted-by":"crossref","first-page":"1250","DOI":"10.1038\/nn.2904","article-title":"Differential roles of human striatum and amygdala in associative learning","volume":"14","author":"J Li","year":"2011","journal-title":"Nat Neurosci"},{"issue":"2","key":"pcbi.1011950.ref418","doi-asserted-by":"crossref","first-page":"263","DOI":"10.2307\/1914185","article-title":"Prospect theory: an analysis of decision under risk.","volume":"47","author":"D Kahneman","year":"1979","journal-title":"Econometrica."},{"issue":"1","key":"pcbi.1011950.ref419","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1038\/nrn3136","article-title":"Normalization as a canonical neural computation","volume":"13","author":"M. Carandini","year":"2012","journal-title":"Nat Rev Neurosci. 2012"},{"issue":"6","key":"pcbi.1011950.ref420","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1016\/j.conb.2012.07.011","article-title":"Value normalization in decision making: theory and evidence","volume":"22","author":"A Rangel","year":"2012","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011950.ref421","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cobeha.2021.06.006","article-title":"Context-dependent outcome encoding in human reinforcement learning.","volume":"41","author":"S Palminteri","year":"2021","journal-title":"Curr Opin Behav Sci"},{"issue":"4","key":"pcbi.1011950.ref422","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1038\/nn1209","article-title":"Prefrontal cortex and decision making in a mixed-strategy game","volume":"7","author":"DJ Barraclough","year":"2004","journal-title":"Nat Neurosci"},{"key":"pcbi.1011950.ref423","first-page":"36","article-title":"Striatal dopamine ramping may indicate flexible reinforcement learning with forgetting in the cortico-basal ganglia circuits.","volume":"8","author":"K Morita","year":"2014","journal-title":"Front Neural Circuits"},{"issue":"10","key":"pcbi.1011950.ref424","doi-asserted-by":"crossref","first-page":"e1005145","DOI":"10.1371\/journal.pcbi.1005145","article-title":"Forgetting in reinforcement learning links sustained dopamine signals to motivation.","volume":"12","author":"A Kato","year":"2016","journal-title":"PLOS Comput Biol"},{"key":"pcbi.1011950.ref425","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.jmp.2017.05.005","article-title":"Model-based estimation of subjective values using choice tasks with probabilistic feedback.","volume":"79","author":"K Katahira","year":"2017","journal-title":"J Math Psychol"},{"issue":"4","key":"pcbi.1011950.ref426","doi-asserted-by":"crossref","first-page":"764","DOI":"10.3758\/s13415-017-0511-2","article-title":"A simple computational algorithm of model-based choice preference.","volume":"17","author":"A Toyama","year":"2017","journal-title":"Cogn Affect Behav Neurosci"},{"key":"pcbi.1011950.ref427","unstructured":"Klopf AH. Brain function and adaptive systems\u2014a heterostatic theory. Bedford (MA): Air Force Cambridge Research Laboratories; 1972. Technical Report No.: AFCRL-72-0164."},{"issue":"2","key":"pcbi.1011950.ref428","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1037\/0033-295X.88.2.135","article-title":"Toward a modern theory of adaptive networks: expectation and prediction.","volume":"88","author":"RS Sutton","year":"1981","journal-title":"Psychol Rev."},{"issue":"3\/4","key":"pcbi.1011950.ref429","doi-asserted-by":"crossref","first-page":"285","DOI":"10.2307\/2332286","article-title":"On the likelihood that one unknown probability exceeds another in view of the evidence of two samples","volume":"25","author":"WR Thompson","year":"1933","journal-title":"Biometrika"},{"issue":"7095","key":"pcbi.1011950.ref430","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1038\/nature04766","article-title":",Cortical substrates for exploratory decisions in humans","volume":"441","author":"Daw ND","year":"2006","journal-title":"Nature"},{"issue":"1481","key":"pcbi.1011950.ref431","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1098\/rstb.2007.2098","article-title":"Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration","volume":"362","author":"JD Cohen","year":"2007","journal-title":"Philos Trans R Soc Lond B Biol Sci"},{"issue":"6","key":"pcbi.1011950.ref432","doi-asserted-by":"crossref","first-page":"2074","DOI":"10.1037\/a0038199","article-title":"Humans use directed and random exploration to solve the explore-exploit dilemma","volume":"143","author":"RC Wilson","year":"2014","journal-title":"J Exp Psychol Gen"},{"issue":"2","key":"pcbi.1011950.ref433","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1111\/tops.12145","article-title":"Uncertainty and exploration in a restless bandit problem.","volume":"7","author":"M Speekenbrink","year":"2015","journal-title":"Top Cogn Sci"},{"key":"pcbi.1011950.ref434","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.cognition.2017.12.014","article-title":"Deconstructing the human algorithms for exploration.","volume":"173","author":"SJ Gershman","year":"2018","journal-title":"Cognition"},{"key":"pcbi.1011950.ref435","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.conb.2018.11.003","article-title":"The algorithmic architecture of exploration in the human brain","volume":"55","author":"E Schulz","year":"2019","journal-title":"Curr Opin Neurobiol"},{"issue":"4","key":"pcbi.1011950.ref436","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","article-title":"A simplex method for function minimization.","volume":"7","author":"JA Nelder","year":"1965","journal-title":"Comput J"}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011950","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,29]],"date-time":"2024-03-29T20:04:33Z","timestamp":1711742673000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011950"}},"subtitle":[],"editor":[{"given":"Stefano","family":"Palminteri","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,3,29]]},"references-count":436,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011950","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,29]]}}}