{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T07:22:26Z","timestamp":1772004146890,"version":"3.50.1"},"reference-count":140,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["2019-0-01371"],"award-info":[{"award-number":["2019-0-01371"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100014364","name":"Samsung Science and Technology Foundation","doi-asserted-by":"publisher","award":["SRFCTC1603-52"],"award-info":[{"award-number":["SRFCTC1603-52"]}],"id":[{"id":"10.13039\/501100014364","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2019M3E5D2A01066267"],"award-info":[{"award-number":["2019M3E5D2A01066267"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2020R1G1A1102683"],"award-info":[{"award-number":["2020R1G1A1102683"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2021M3E5D2A0102249311"],"award-info":[{"award-number":["2021M3E5D2A0102249311"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Recent investigation on reinforcement learning (RL) has demonstrated considerable flexibility in dealing with various problems. However, such models often experience difficulty learning seemingly easy tasks for humans. To reconcile the discrepancy, our paper is focused on the computational benefits of the brain's RL. We examine the brain's ability to combine complementary learning strategies to resolve the trade-off between prediction performance, computational costs, and time constraints. The complex need for task performance created by a volatile and\/or multi-agent environment motivates the brain to continually explore an ideal combination of multiple strategies, called meta-control. Understanding these functions would allow us to build human-aligned RL models.<\/jats:p>","DOI":"10.3389\/fncom.2022.1060101","type":"journal-article","created":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T09:14:05Z","timestamp":1671614045000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Importance of prefrontal meta control in human-like reinforcement learning"],"prefix":"10.3389","volume":"16","author":[{"given":"Jee Hang","family":"Lee","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joel Z.","family":"Leibo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Su Jin","family":"An","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sang Wan","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2022,12,21]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1126\/science.7466396","article-title":"The evolution of cooperation","volume":"211","author":"Axelrod","year":"1981","journal-title":"Science"},{"key":"B2","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1016\/j.neuron.2011.12.025","article-title":"Rostrolateral prefrontal cortex and individual differences in uncertainty-driven exploration","volume":"73","author":"Badre","year":"2012","journal-title":"Neuron"},{"key":"B3","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1038\/npp.2009.131","article-title":"Human and rodent homologies in action control: corticostriatal determinants of goal-directed and habitual action","volume":"35","author":"Balleine","year":"2010","journal-title":"Neuropsychopharmacology"},{"key":"B4","first-page":"687","article-title":"\u201cMonte Carlo matrix inversion and reinforcement learning,\u201d","author":"Barto","year":"1994","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B5","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1109\/TSMC.1983.6313077","article-title":"Neuronlike adaptive elements that can solve difficult learning control problems","author":"Barto","year":"1983","journal-title":"IEEE Trans. Syst. Man Cybernet."},{"key":"B6","first-page":"679","article-title":"A Markovian decision process","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"B7","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1109\/CDC.1995.478953","article-title":"\u201cNeuro-dynamic programming: an overview.\u201d","volume-title":"Proceedings of the 34th IEEE Conference on Decision and Control, 1995","author":"Bertsekas","year":"1995"},{"key":"B8","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1016\/j.neuron.2009.05.014","article-title":"How green is the grass on the other side? frontopolar cortex and the evidence in favor of alternative courses of action","volume":"62","author":"Boorman","year":"2009","journal-title":"Neuron"},{"key":"B9","first-page":"213","article-title":"R-max-a general polynomial time algorithm for near-optimal reinforcement learning","volume":"3","author":"Brafman","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"B10","author":"Busoniu","year":"2010","journal-title":"Reinforcement Learning and Dynamic Programming Using Function Approximators"},{"key":"B11","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1257\/0022051053737843","article-title":"Neuroeconomics: how neuroscience can inform economics","volume":"43","author":"Camerer","year":"2005","journal-title":"J. Econ. Lit."},{"key":"B12","article-title":"\u201cUncertainty and dependency in causal inference,\u201d","author":"Carroll","year":"2011","journal-title":"Proceedings of the Cognitive Science Society"},{"key":"B13","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1523\/JNEUROSCI.06-03-00613.1986","article-title":"Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat","volume":"6","author":"Christoph","year":"1986","journal-title":"J. Neurosci."},{"key":"B14","first-page":"746","article-title":"The dynamics of reinforcement learning in cooperative multiagent systems","volume":"1998","author":"Claus","year":"1998","journal-title":"AAAI\/IAAI"},{"key":"B15","article-title":"Humans decompose tasks by trading off utility and computational cost","author":"Correa","year":"2022","journal-title":"arXiv preprint arXiv:2211.03890"},{"key":"B16","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1145\/2959100.2959190","article-title":"\u201cDeep neural networks for youtube recommendations,\u201d","author":"Covington","year":"2016","journal-title":"Proceedings of the 10th ACM Conference on Recommender Systems"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/j.neuron.2011.02.027","article-title":"Model-based influences on humans' choices and striatal prediction errors","volume":"69","author":"Daw","year":"2011","journal-title":"Neuron"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nat. Neurosci."},{"key":"B19","doi-asserted-by":"publisher","first-page":"876","DOI":"10.1038\/nature04766","article-title":"Cortical substrates for exploratory decisions in humans","volume":"441","author":"Daw","year":"2006","journal-title":"Nature"},{"key":"B20","doi-asserted-by":"publisher","first-page":"473","DOI":"10.3758\/s13415-014-0277-8","article-title":"Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation","volume":"14","author":"Dayan","year":"2014","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"B21","doi-asserted-by":"publisher","first-page":"429","DOI":"10.3758\/CABN.8.4.429","article-title":"Decision theory, reinforcement learning, and the brain","volume":"8","author":"Dayan","year":"2008","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"B22","first-page":"863","article-title":"\u201cArtificial agents learning human fairness,\u201d","author":"De Jong","year":"2008","journal-title":"Proceedings of the 7th International Joint Conference on Autonomous Agents and Multiagent Systems"},{"key":"B23","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1038\/nn.3279","article-title":"Confidence in value-based choice","volume":"16","author":"De Martino","year":"2013","journal-title":"Nat. Neurosci."},{"key":"B24","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1080\/14640748308400912","article-title":"The effect of the instrumental training contingency on susceptibility to reinforcer devaluation","volume":"35","author":"Dickinson","year":"1983","journal-title":"Q. J. Exp. Psychol."},{"key":"B25","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.neuron.2013.09.007","article-title":"Goals and habits in the brain","volume":"80","author":"Dolan","year":"2013","journal-title":"Neuron"},{"key":"B26","doi-asserted-by":"publisher","first-page":"767","DOI":"10.1038\/nn.3981","article-title":"Model-based choices involve prospective neural activity","volume":"18","author":"Doll","year":"2015","journal-title":"Nat. Neurosci."},{"key":"B27","doi-asserted-by":"crossref","first-page":"961","DOI":"10.1016\/S0893-6080(99)00046-5","article-title":"What are the computations of the cerebellum, the basal ganglia and the cerebral cortex?","volume":"12","author":"Doya","year":"1999","journal-title":"Neural Netw."},{"key":"B28","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.1162\/089976602753712972","article-title":"Multiple model-based reinforcement learning","volume":"14","author":"Doya","year":"2002","journal-title":"Neural Comput."},{"key":"B29","doi-asserted-by":"publisher","first-page":"2419","DOI":"10.1007\/s10994-021-05961-4","article-title":"Challenges of real-world reinforcement learning: definitions, benchmarks and analysis","volume":"110","author":"Dulac-Arnold","year":"2021","journal-title":"Mach. Learn."},{"key":"B30","unstructured":"EvansR.\n            GaoJ.\n          Deepmind AI Reduces Google Data Centre Cooling Bill by 40%. DeepMind blog2016"},{"key":"B31","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1016\/j.tics.2007.09.002","article-title":"Social neuroeconomics: the neural circuitry of social preferences","volume":"11","author":"Fehr","year":"2007","journal-title":"Trends Cogn. Sci."},{"key":"B32","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1162\/003355399556151","article-title":"A theory of fairness, competition, and cooperation","volume":"114","author":"Fehr","year":"1999","journal-title":"Q. J. Econ."},{"key":"B33","doi-asserted-by":"publisher","first-page":"594","DOI":"10.1109\/TPAMI.2006.79","article-title":"One-shot learning of object categories","volume":"28","author":"Fei-Fei","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B34","doi-asserted-by":"publisher","first-page":"457","DOI":"10.3389\/fnhum.2014.00457","article-title":"Model averaging, optimal inference, and habit formation","volume":"8","author":"FitzGerald","year":"2014","journal-title":"Front. Hum. Neurosci."},{"key":"B35","doi-asserted-by":"publisher","first-page":"1541","DOI":"10.1126\/science.1191883","article-title":"Relating introspective accuracy to individual differences in brain structure","volume":"329","author":"Fleming","year":"2010","journal-title":"Science"},{"key":"B36","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1038\/nn1103","article-title":"Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission","volume":"6","author":"Floresco","year":"2003","journal-title":"Nat. Neurosci."},{"key":"B37","first-page":"122","article-title":"\u201cLearning with opponent-learning awareness,\u201d","author":"Foerster","year":"2018","journal-title":"Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems"},{"key":"B38","first-page":"2052","article-title":"\u201cOff-policy deep reinforcement learning without exploration,\u201d","volume-title":"International Conference on Machine Learning","author":"Fujimoto","year":"2019"},{"key":"B39","article-title":"\u201cFew-shot learning with graph neural networks,\u201d","author":"Garcia","year":"2018","journal-title":"International Conference on Learning Representations"},{"key":"B40","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1192\/apt.bp.109.007104","article-title":"Jumping to conclusions: the psychology of delusional reasoning","volume":"17","author":"Garety","year":"2011","journal-title":"Adv. Psychiatr. Treat."},{"key":"B41","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1002\/cne.20668","article-title":"Afferents of the ventral tegmental area in the rat-anatomical substratum for integrative functions","volume":"490","author":"Geisler","year":"2005","journal-title":"J. Comp. Neurol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"135","DOI":"10.3389\/fnbeh.2015.00135","article-title":"Goal-directed, habitual and Pavlovian prosocial behavior","volume":"9","author":"Gesiarz","year":"2015","journal-title":"Front. Behav. Neurosci."},{"key":"B43","volume-title":"Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Behavior","author":"Gintis","year":"2000"},{"key":"B44","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"Gl\u00e4scher","year":"2010","journal-title":"Neuron"},{"key":"B45","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1093\/cercor\/bhn098","article-title":"Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making","volume":"19","author":"Gl\u00e4scher","year":"2008","journal-title":"Cereb. Cortex"},{"key":"B46","volume-title":"Neuroeconomics: Decision Making and the Brain","author":"Glimcher","year":"2013"},{"key":"B47","doi-asserted-by":"publisher","first-page":"15647","DOI":"10.1073\/pnas.1014269108","article-title":"Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis","volume":"108","author":"Glimcher","year":"2011","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B48","doi-asserted-by":"publisher","first-page":"661","DOI":"10.1037\/a0017201","article-title":"Theory-based causal induction","volume":"116","author":"Griffiths","year":"2009","journal-title":"Psychol. Rev."},{"key":"B49","doi-asserted-by":"publisher","first-page":"8360","DOI":"10.1523\/JNEUROSCI.1010-06.2006","article-title":"The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans","volume":"26","author":"Hampton","year":"2006","journal-title":"J. Neurosci."},{"key":"B50","article-title":"\u201cMetacontrol for adaptive imagination-based optimization,\u201d","author":"Hamrick","year":"2017","journal-title":"International Conference on Learning Representations"},{"key":"B51","doi-asserted-by":"crossref","first-page":"1243","DOI":"10.1126\/science.162.3859.1243","article-title":"The tragedy of the commons","volume":"162","author":"Hardin","year":"1968","journal-title":"Science"},{"key":"B52","doi-asserted-by":"publisher","first-page":"646","DOI":"10.1126\/science.1168450","article-title":"Self-control in decision-making involves modulation of the vmPFC valuation system","volume":"324","author":"Hare","year":"2009","journal-title":"Science"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.2307\/j.ctvc77f0d","volume-title":"The Secret of Our Success: How Culture Is Driving Human Evolution, Domesticating Our Species, and Making Us Smarter","author":"Henrich","year":"2015"},{"key":"B54","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1257\/aer.91.2.73","article-title":"In search of homo economicus: behavioral experiments in 15 small-scale societies","volume":"91","author":"Henrich","year":"2001","journal-title":"Am. Econ. Rev."},{"key":"B55","first-page":"4214","article-title":"\u201cMuesli: combining improvements in policy optimization,\u201d","volume-title":"International Conference on Machine Learning","author":"Hessel","year":"2021"},{"key":"B56","doi-asserted-by":"publisher","first-page":"702","DOI":"10.1037\/a0020488","article-title":"Analogical and category-based inference: a theoretical integration with bayesian causal models","volume":"139","author":"Holyoak","year":"2010","journal-title":"J. Exp. Psychol. Gen."},{"key":"B57","article-title":"Inequity aversion resolves intertemporal social dilemmas","author":"Hughes","year":"2018","journal-title":"arXiv preprint arXiv:1803.08884"},{"key":"B58","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1037\/h0093874","article-title":"Judgment of contingency between responses and outcomes","volume":"79","author":"Jenkins","year":"1965","journal-title":"Psychol. Monogr. Gen. Appl."},{"key":"B59","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1016\/j.neuron.2016.12.038","article-title":"Ventromedial prefrontal cortex encodes a latent estimate of cumulative reward","volume":"93","author":"Juechems","year":"2017","journal-title":"Neuron"},{"key":"B60","first-page":"651","article-title":"\u201cScalable deep reinforcement learning for vision-based robotic manipulation,\u201d","volume-title":"Conference on Robot Learning","author":"Kalashnikov","year":"2018"},{"key":"B61","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1023\/A:1017984413808","article-title":"Near-optimal reinforcement learning in polynomial time","volume":"49","author":"Kearns","year":"2002","journal-title":"Mach. Learn."},{"key":"B62","first-page":"331","article-title":"Evaluating SR-based reinforcement learning algorithm under the highly uncertain decision task","volume":"11","author":"Kim","year":"2022","journal-title":"KIPS Trans. Softw. Data Eng."},{"key":"B63","article-title":"\u201cCoordinate to cooperate or compete: abstract goals and joint intentions in social interaction,\u201d","author":"Kleiman-Weiner","year":"2016","journal-title":"COGSCI"},{"key":"B64","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1146\/annurev.soc.24.1.183","article-title":"Social dilemmas: the anatomy of cooperation","volume":"24","author":"Kollock","year":"1998","journal-title":"Annu. Rev. Sociol."},{"key":"B65","article-title":"\u201cHierarchical deep reinforcement learning: integrating temporal abstraction and intrinsic motivation,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Kulkarni","year":"2016"},{"key":"B66","first-page":"101","article-title":"\u201cModel-based reinforcement learning with an approximate, learned model,\u201d","author":"Kuvayev","year":"1996","journal-title":"Proc. Yale Workshop Adapt. Learn. Syst"},{"key":"B67","doi-asserted-by":"publisher","first-page":"e253","DOI":"10.1017\/S0140525X16001837","article-title":"Building machines that learn and think like people","volume":"40","author":"Lake","year":"2017","journal-title":"Behav. Brain Sci."},{"key":"B68","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1146\/annurev-neuro-062111-150512","article-title":"Neural basis of reinforcement learning and decision making","volume":"35","author":"Lee","year":"2012","journal-title":"Annu. Rev. Neurosci."},{"key":"B69","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1109\/TKDE.2009.123","article-title":"A nonsupervised learning framework of human behavior patterns based on sequential actions","volume":"22","author":"Lee","year":"2009","journal-title":"IEEE Trans. Knowledge Data Eng."},{"key":"B70","doi-asserted-by":"publisher","first-page":"e1002137","DOI":"10.1371\/journal.pbio.1002137","article-title":"Neural computations mediating one-shot learning in the human brain","volume":"13","author":"Lee","year":"2015","journal-title":"PLoS Biol."},{"key":"B71","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"Lee","year":"2014","journal-title":"Neuron"},{"key":"B72","first-page":"464","article-title":"\u201cMulti-agent reinforcement learning in sequential social dilemmas,\u201d","author":"Leibo","year":"2017","journal-title":"Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems"},{"key":"B73","article-title":"Maintaining cooperation in complex social dilemmas using deep reinforcement learning","author":"Lerer","year":"2017","journal-title":"arXiv preprint arXiv:1707.01068"},{"key":"B74","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1145\/1390156.1390228","article-title":"\u201cKnows what it knows: a framework for self-aware learning,\u201d","volume-title":"Proceedings of the 25th International Conference on Machine Learning","author":"Li","year":"2008"},{"key":"B75","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2015","journal-title":"arXiv preprint arXiv:1509.02971"},{"key":"B76","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/B978-1-55860-335-6.50027-1","article-title":"\u201cMarkov games as a framework for multi-agent reinforcement learning,\u201d","volume-title":"Machine Learning Proceedings 1994","author":"Littman","year":"1994"},{"key":"B77","volume-title":"Algorithms for sequential decision making","author":"Littman","year":"1996"},{"key":"B78","doi-asserted-by":"publisher","first-page":"1111","DOI":"10.1038\/nature05860","article-title":"Lateral habenula as a source of negative reward signals in dopamine neurons","volume":"447","author":"Matsumoto","year":"2007","journal-title":"Nature"},{"key":"B79","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1038\/s41562-016-0042","article-title":"The developmental foundations of human fairness","volume":"1","author":"McAuliffe","year":"2017","journal-title":"Nat. Hum. Behav."},{"key":"B80","doi-asserted-by":"publisher","first-page":"211","DOI":"10.3758\/BF03200258","article-title":"An elemental model of associative learning: I. latent inhibition and perceptual learning","volume":"28","author":"McLaren","year":"2000","journal-title":"Anim. Learn. Behav."},{"key":"B81","doi-asserted-by":"publisher","first-page":"e1004305","DOI":"10.1371\/journal.pcbi.1004305","article-title":"The sense of confidence during probabilistic learning: a normative account","volume":"11","author":"Meyniel","year":"2015","journal-title":"PLoS Comput. Biol."},{"key":"B82","first-page":"1928","article-title":"\u201cAsynchronous methods for deep reinforcement learning,\u201d","author":"Mnih","year":"2016","journal-title":"International Conference on Machine Learning"},{"key":"B83","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"B84","article-title":"Model-based reinforcement learning: A survey","author":"Moerland","year":"2020","journal-title":"arXiv preprint arXiv:2006.16712"},{"key":"B85","doi-asserted-by":"crossref","first-page":"1936","DOI":"10.1523\/JNEUROSCI.16-05-01936.1996","article-title":"A framework for mesencephalic dopamine systems based on predictive hebbian learning","volume":"16","author":"Montague","year":"1996","journal-title":"J. Neurosci."},{"key":"B86","doi-asserted-by":"publisher","first-page":"261","DOI":"10.3758\/CABN.6.4.261","article-title":"Jumping to conclusions: a network model predicts schizophrenic patients' performance on a probabilistic reasoning task","volume":"6","author":"Moore","year":"2006","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"B87","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1080\/13546805.2010.548678","article-title":"Bayesian modelling of jumping-to-conclusions bias in delusional patients","volume":"16","author":"Moutoussis","year":"2011","journal-title":"Cogn. Neuropsychiatry"},{"key":"B88","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1016\/S0896-6273(03)00869-9","article-title":"Dopamine neurons can represent context-dependent prediction error","volume":"41","author":"Nakahara","year":"2004","journal-title":"Neuron"},{"key":"B89","doi-asserted-by":"publisher","first-page":"289","DOI":"10.3389\/fnbeh.2015.00289","article-title":"Individual variability in behavioral flexibility predicts sign-tracking tendency","volume":"9","author":"Nasser","year":"2015","journal-title":"Front. Behav. Neurosci."},{"key":"B90","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.jmp.2008.12.005","article-title":"Reinforcement learning in the brain","volume":"53","author":"Niv","year":"2009","journal-title":"J. Math. Psychol."},{"key":"B91","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1146\/annurev-psych-010416-044216","article-title":"Learning, reward, and decision making","volume":"68","author":"O'Doherty","year":"2017","journal-title":"Annu. Rev. Psychol."},{"key":"B92","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.cobeha.2014.10.004","article-title":"The structure of reinforcement-learning mechanisms in the human brain","volume":"1","author":"O'Doherty","year":"2015","journal-title":"Curr. Opin. Behav. Sci."},{"key":"B93","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.neubiorev.2020.10.022","article-title":"Why and how the brain weights contributions from a mixture of experts","volume":"123","author":"O'Doherty","year":"2021","journal-title":"Neurosci. Biobehav. Rev."},{"key":"B94","unstructured":"OpenAI Five2018"},{"key":"B95","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1038\/nature04676","article-title":"Neurons in the orbitofrontal cortex encode economic value","volume":"441","author":"Padoa-Schioppa","year":"2006","journal-title":"Nature"},{"key":"B96","doi-asserted-by":"publisher","first-page":"387","DOI":"10.1007\/s10458-005-2631-2","article-title":"Cooperative multi-agent learning: the state of the art","volume":"11","author":"Panait","year":"2005","journal-title":"Auton. Agents Multiagent Syst."},{"key":"B97","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1037\/0033-295X.87.6.532","article-title":"A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli","volume":"87","author":"Pearce","year":"1980","journal-title":"Psychol. Rev."},{"key":"B98","first-page":"3643","article-title":"\u201cA multi-agent reinforcement learning model of common-pool resource appropriation,\u201d","author":"Perolat","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B99","doi-asserted-by":"publisher","first-page":"92","DOI":"10.3389\/fpsyg.2013.00092","article-title":"The mixed instrumental controller: using value of information to combine habitual choice and mental simulation","volume":"4","author":"Pezzulo","year":"2013","journal-title":"Front. Psychol."},{"key":"B100","doi-asserted-by":"publisher","first-page":"545","DOI":"10.1038\/nrn2357","article-title":"A framework for studying the neurobiology of value-based decision making","volume":"9","author":"Rangel","year":"2008","journal-title":"Nat. Rev. Neurosci."},{"key":"B101","doi-asserted-by":"crossref","DOI":"10.3998\/mpub.20269","volume-title":"Prisoner's Dilemma: A Study in Conflict and Cooperation","author":"Rapoport","year":"1965"},{"key":"B102","first-page":"64","article-title":"\u201cA theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement,\u201d","volume-title":"Classical Conditioning II: Current Research and Theory","author":"Recorla","year":"1972"},{"key":"B103","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1613\/jair.3987","article-title":"A survey of multi-objective sequential decision-making","volume":"48","author":"Roijers","year":"2013","journal-title":"J. Artif. Intell. Res"},{"key":"B104","author":"Rummery","year":"1994","journal-title":"On-Line Q-Learning Using Connectionist Systems"},{"key":"B105","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1016\/j.conb.2012.04.011","article-title":"Valuation and decision-making in frontal cortex: one or many serial or parallel systems?","volume":"22","author":"Rushworth","year":"2012","journal-title":"Curr. Opin. Neurobiol."},{"key":"B106","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1016\/j.neuron.2015.07.024","article-title":"Abstract context representations in primate amygdala and prefrontal cortex","volume":"87","author":"Saez","year":"2015","journal-title":"Neuron"},{"key":"B107","doi-asserted-by":"publisher","first-page":"2532","DOI":"10.1111\/j.0021-9029.2006.00116.x","article-title":"The psychological benefits of superstitious rituals in top sport: a study among top sportspersons 1","volume":"36","author":"Schippers","year":"2006","journal-title":"J. Appl. Soc. Psychol."},{"key":"B108","doi-asserted-by":"publisher","first-page":"604","DOI":"10.1038\/s41586-020-03051-4","article-title":"Mastering Atari, Go, chess and shogi by planning with a learned model","volume":"588","author":"Schrittwieser","year":"2020","journal-title":"Nature"},{"key":"B109","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1152\/jn.1998.80.1.1","article-title":"Predictive reward signal of dopamine neurons","volume":"80","author":"Schultz","year":"1998","journal-title":"J. Neurophysiol."},{"key":"B110","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"Schultz","year":"1997","journal-title":"Science"},{"key":"B111","doi-asserted-by":"publisher","first-page":"15871","DOI":"10.1073\/pnas.1821647116","article-title":"Credit assignment to state-independent task representations and its relationship with model-based decision making","volume":"116","author":"Shahar","year":"2019","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B112","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/j.neuron.2013.07.007","article-title":"The expected value of control: an integrative theory of anterior cingulate cortex function","volume":"79","author":"Shenhav","year":"2013","journal-title":"Neuron"},{"key":"B113","doi-asserted-by":"crossref","DOI":"10.1109\/9780470544785","author":"Si","year":"2004","journal-title":"Handbook of Learning and Approximate Dynamic Programming"},{"key":"B114","doi-asserted-by":"crossref","DOI":"10.1002\/9781118557426","author":"Sigaud","year":"2013","journal-title":"Markov Decision Processes in Artificial Intelligence"},{"key":"B115","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"B116","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"B117","first-page":"387","article-title":"\u201cDeterministic policy gradient algorithms,\u201d","author":"Silver","year":"2014","journal-title":"Proceedings of the 31st International Conference on Machine Learning (ICML-14)"},{"key":"B118","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"","journal-title":"Nature"},{"key":"B119","article-title":"Mastering chess and shogi by self-play with a general reinforcement learning algorithm","author":"Silver","year":"","journal-title":"arXiv preprint arXiv:1712.01815"},{"key":"B120","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00114726","article-title":"Reinforcement learning with replacing eligibility traces","volume":"22","author":"Singh","year":"1996","journal-title":"Mach. Learn."},{"key":"B121","article-title":"Open-ended learning leads to generally capable agents","author":"Stooke","year":"2021","journal-title":"arXiv preprint arXiv:2107.12808"},{"key":"B122","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn."},{"key":"B123","article-title":"\u201cGeneralization in reinforcement learning: Successful examples using sparse coarse coding,\u201d","author":"Sutton","year":"1995","journal-title":"Advances in Neural Information Processing Systems."},{"key":"B124","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"1998"},{"key":"B125","first-page":"1","article-title":"\u201cSynthesis lectures on artificial intelligence and machine learning,\u201d","author":"Szepesv\u00e1ri","year":"2010","journal-title":"Algorithms for Reinforcement Learning, Vol. 4"},{"key":"B126","volume-title":"Structure & Function of the Body","author":"Thibodeau","year":"1992"},{"key":"B127","doi-asserted-by":"crossref","DOI":"10.1037\/h0067373","article-title":"\u201cAnimal intelligence: An experimental study of the associative processes in animals,\u201d","author":"Thorndike","year":"1898","journal-title":"The Psychological Review: Monograph Supplements"},{"key":"B128","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychol. Rev."},{"key":"B129","doi-asserted-by":"publisher","first-page":"2225","DOI":"10.1111\/j.1460-9568.2009.06796.x","article-title":"A specific role for posterior dorsolateral striatum in human habit learning","volume":"29","author":"Tricomi","year":"2009","journal-title":"Eur. J. Neurosci."},{"key":"B130","doi-asserted-by":"publisher","first-page":"1089","DOI":"10.1038\/nature08785","article-title":"Neural evidence for inequality-averse social preferences","volume":"463","author":"Tricomi","year":"2010","journal-title":"Nature"},{"key":"B131","doi-asserted-by":"publisher","first-page":"4019","DOI":"10.1523\/JNEUROSCI.0564-07.2007","article-title":"Determining the neural substrates of goal-directed learning in the human brain","volume":"27","author":"Valentin","year":"2007","journal-title":"J. Neurosci."},{"key":"B132","first-page":"2094","article-title":"\u201cDeep reinforcement learning with double q-learning,\u201d","author":"Van Hasselt","year":"2016","journal-title":"AAAI"},{"key":"B133","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-27645-3_1","volume-title":"Reinforcement Learning and Markov Decision Processes","author":"van Otterlo","year":"2012"},{"key":"B134","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1109\/ICRA.2019.8794074","article-title":"\u201cA practical approach to insertion with variable socket position using deep reinforcement learning,\u201d","volume-title":"2019 International Conference on Robotics and Automation (ICRA)","author":"Vecerik","year":"2019"},{"key":"B135","first-page":"22536","article-title":"\u201cTowards evaluating adaptivity of model-based reinforcement learning methods,\u201d","author":"Wan","year":"2022","journal-title":"Proceedings of the 39th International Conference on Machine Learning"},{"key":"B136","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1038\/s41593-018-0147-8","article-title":"Prefrontal cortex as a meta-reinforcement learning system","volume":"21","author":"Wang","year":"2018","journal-title":"Nat. Neurosci."},{"key":"B137","article-title":"Learning to reinforcement learn","author":"Wang","year":"2016","journal-title":"arXiv preprint arXiv:1611.05763"},{"key":"B138","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"B139","volume-title":"Learning from delayed rewards","author":"Watkins","year":"1989"},{"key":"B140","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1038\/nn.3068","article-title":"Mapping value based planning and extensively trained choice in the human brain","volume":"15","author":"Wunderlich","year":"2012","journal-title":"Nat. Neurosci."}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2022.1060101\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T09:15:16Z","timestamp":1671614116000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2022.1060101\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,21]]},"references-count":140,"alternative-id":["10.3389\/fncom.2022.1060101"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2022.1060101","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,21]]},"article-number":"1060101"}}