{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T04:18:50Z","timestamp":1784348330409,"version":"3.55.0"},"reference-count":51,"publisher":"eLife Sciences Publications, Ltd","license":[{"start":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T00:00:00Z","timestamp":1569974400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T00:00:00Z","timestamp":1569974400000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T00:00:00Z","timestamp":1569974400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01 AG037564- 01A1"],"award-info":[{"award-number":["R01 AG037564- 01A1"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>In competitive situations, winning depends on selecting actions that surprise the opponent. Such unpredictable action can be generated based on representations of the opponent\u2019s strategy and choice history (model-based counter-prediction) or by choosing actions in a memory-free, stochastic manner. Across five different experiments using a variant of a matching-pennies game with simulated and human opponents we found that people toggle between these two strategies, using model-based selection when recent wins signal the appropriateness of the current model, but reverting to stochastic selection following losses. Also, after wins, feedback-related, mid-frontal EEG activity reflected information about the opponent\u2019s global and local strategy, and predicted upcoming choices. After losses, this activity was nearly absent\u2014indicating that the internal model is suppressed after negative feedback. We suggest that the mixed-strategy approach allows negotiating two conflicting goals: 1) exploiting the opponent\u2019s deviations from randomness while 2) remaining unpredictable for the opponent.<\/jats:p>","DOI":"10.7554\/elife.48810","type":"journal-article","created":{"date-parts":[[2019,10,2]],"date-time":"2019-10-02T08:00:20Z","timestamp":1570003220000},"source":"Crossref","is-referenced-by-count":6,"title":["Balancing model-based and memory-free action selection under competitive pressure"],"prefix":"10.7554","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2179-2700","authenticated-orcid":true,"given":"Atsushi","family":"Kikumoto","sequence":"first","affiliation":[{"name":"Department of Psychology, University of Oregon, Eugene, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7512-4556","authenticated-orcid":true,"given":"Ulrich","family":"Mayr","sequence":"additional","affiliation":[{"name":"Department of Psychology, University of Oregon, Eugene, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"4374","published-online":{"date-parts":[[2019,10,2]]},"reference":[{"key":"bib1","doi-asserted-by":"publisher","first-page":"E3548","DOI":"10.1073\/pnas.1518786113","article-title":"Adaptable history biases in human perceptual decisions","volume":"113","author":"Abrahamyan","year":"2016","journal-title":"PNAS"},{"key":"bib2","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1037\/0021-843X.87.1.49","article-title":"Learned helplessness in humans: critique and reformulation","volume":"87","author":"Abramson","year":"1978","journal-title":"Journal of Abnormal Psychology"},{"key":"bib3","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1111\/j.0956-7976.2004.00728.x","article-title":"The cost of a voluntary task switch","volume":"15","author":"Arrington","year":"2004","journal-title":"Psychological Science"},{"key":"bib4","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1080\/14640746608400019","article-title":"The capacity for generating information by randomization","volume":"18","author":"Baddeley","year":"1966","journal-title":"Quarterly Journal of Experimental Psychology"},{"key":"bib5","doi-asserted-by":"publisher","first-page":"819","DOI":"10.1080\/713755788","article-title":"Random generation and the executive control of working memory","volume":"51","author":"Baddeley","year":"1998","journal-title":"The Quarterly Journal of Experimental Psychology Section A"},{"key":"bib6","doi-asserted-by":"publisher","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"Behrens","year":"2007","journal-title":"Nature Neuroscience"},{"key":"bib7","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1163\/156856897X00357","article-title":"The psychophysics toolbox","volume":"10","author":"Brainard","year":"1997","journal-title":"Spatial Vision"},{"key":"bib8","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1016\/j.cobeha.2015.04.005","article-title":"A psychological approach to strategic thinking in games","volume":"3","author":"Camerer","year":"2015","journal-title":"Current Opinion in Behavioral Sciences"},{"key":"bib9","doi-asserted-by":"publisher","first-page":"2575","DOI":"10.1093\/cercor\/bhr332","article-title":"Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation","volume":"22","author":"Cavanagh","year":"2012","journal-title":"Cerebral Cortex"},{"key":"bib10","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1016\/j.tics.2014.04.012","article-title":"Frontal theta as a mechanism for cognitive control","volume":"18","author":"Cavanagh","year":"2014","journal-title":"Trends in Cognitive Sciences"},{"key":"bib11","doi-asserted-by":"publisher","first-page":"1138","DOI":"10.1257\/00028280260344678","article-title":"Testing Mixed-Strategy equilibria when players are heterogeneous: the case of penalty kicks in soccer","volume":"92","author":"Chiappori","year":"2002","journal-title":"American Economic Review"},{"key":"bib12","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1016\/j.neuroimage.2006.11.056","article-title":"Reward expectation modulates feedback-related negativity and EEG spectra","volume":"35","author":"Cohen","year":"2007","journal-title":"NeuroImage"},{"key":"bib13","doi-asserted-by":"publisher","first-page":"558","DOI":"10.1016\/j.tics.2011.10.004","article-title":"Cortical electrophysiological network dynamics of feedback learning","volume":"15","author":"Cohen","year":"2011","journal-title":"Trends in Cognitive Sciences"},{"key":"bib14","doi-asserted-by":"publisher","first-page":"371","DOI":"10.1523\/JNEUROSCI.4421-06.2007","article-title":"Reinforcement learning signals predict future decisions","volume":"27","author":"Cohen","year":"2007","journal-title":"Journal of Neuroscience"},{"key":"bib15","volume-title":"Analysis of Binary Data","author":"Cox","year":"2018"},{"key":"bib16","doi-asserted-by":"publisher","first-page":"1704","DOI":"10.1038\/nn1560","article-title":"Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control","volume":"8","author":"Daw","year":"2005","journal-title":"Nature Neuroscience"},{"key":"bib17","doi-asserted-by":"publisher","first-page":"876","DOI":"10.1038\/nature04766","article-title":"Cortical substrates for exploratory decisions in humans","volume":"441","author":"Daw","year":"2006","journal-title":"Nature"},{"key":"bib18","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.jneumeth.2003.10.009","article-title":"EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis","volume":"134","author":"Delorme","year":"2004","journal-title":"Journal of Neuroscience Methods"},{"key":"bib19","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1016\/j.neuron.2013.07.040","article-title":"Cortical signals for rewarded actions and strategic exploration","volume":"80","author":"Donahue","year":"2013","journal-title":"Neuron"},{"key":"bib20","first-page":"848","article-title":"Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria","volume":"88","author":"Erev","year":"1998","journal-title":"American Economic Review"},{"key":"bib21","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1006\/nimg.1997.0291","article-title":"Psychophysiological and modulatory interactions in neuroimaging","volume":"6","author":"Friston","year":"1997","journal-title":"NeuroImage"},{"key":"bib22","doi-asserted-by":"publisher","first-page":"416","DOI":"10.1016\/j.cognition.2008.09.007","article-title":"The smart potential behind probability matching","volume":"109","author":"Gaissmaier","year":"2008","journal-title":"Cognition"},{"key":"bib23","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1111\/j.1467-9280.1993.tb00586.x","article-title":"A neural system for error detection and compensation","volume":"4","author":"Gehring","year":"1993","journal-title":"Psychological Science"},{"key":"bib24","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1016\/j.neuron.2010.04.016","article-title":"States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning","volume":"66","author":"Gl\u00e4scher","year":"2010","journal-title":"Neuron"},{"key":"bib25","doi-asserted-by":"publisher","DOI":"10.1101\/433409","article-title":"Response outcomes gate the impact of expectations on perceptual decisions","volume-title":"bioRxiv","author":"Hermoso-Mendizabal","year":"2019"},{"key":"bib26","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1037\/0033-295X.109.4.679","article-title":"The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity","volume":"109","author":"Holroyd","year":"2002","journal-title":"Psychological Review"},{"key":"bib27","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1177\/0956797611407933","article-title":"Banking on a bad bet. Probability matching in risky choice is linked to expectation generation","volume":"22","author":"James","year":"2011","journal-title":"Psychological Science"},{"key":"bib28","doi-asserted-by":"publisher","first-page":"1073","DOI":"10.3758\/s13415-017-0531-y","article-title":"Increased locus coeruleus tonic activity causes disengagement from a patch-foraging task","volume":"17","author":"Kane","year":"2017","journal-title":"Cognitive, Affective, & Behavioral Neuroscience"},{"key":"bib29","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1016\/j.conb.2015.12.007","article-title":"Multiple signals in anterior cingulate cortex","volume":"37","author":"Kolling","year":"2016","journal-title":"Current Opinion in Neurobiology"},{"key":"bib30","doi-asserted-by":"publisher","first-page":"1280","DOI":"10.1038\/nn.4382","article-title":"Value, search, persistence and model updating in anterior cingulate cortex","volume":"19","author":"Kolling","year":"2016","journal-title":"Nature Neuroscience"},{"key":"bib31","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1038\/nn2065","article-title":"Game theory and neural basis of social decision making","volume":"11","author":"Lee","year":"2008","journal-title":"Nature Neuroscience"},{"key":"bib32","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1146\/annurev-neuro-062111-150512","article-title":"Neural basis of reinforcement learning and decision making","volume":"35","author":"Lee","year":"2012","journal-title":"Annual Review of Neuroscience"},{"key":"bib33","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.neuron.2013.11.028","article-title":"Neural computations underlying arbitration between model-based and model-free learning","volume":"81","author":"Lee","year":"2014","journal-title":"Neuron"},{"key":"bib34","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-017-0067","article-title":"Behavioural and neural characterization of optimistic reinforcement learning","volume":"1","author":"Lefebvre","year":"2017","journal-title":"Nature Human Behaviour"},{"key":"bib35","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1016\/j.bbr.2013.12.043","article-title":"Learning from feedback: the neural mechanisms of feedback processing facilitating better performance","volume":"261","author":"Luft","year":"2014","journal-title":"Behavioural Brain Research"},{"key":"bib36","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1037\/rev0000033","article-title":"Learned helplessness at fifty: insights from neuroscience","volume":"123","author":"Maier","year":"2016","journal-title":"Psychological Review"},{"key":"bib37","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1111\/j.1467-9280.2006.01781.x","article-title":"On how to be unpredictable: evidence from the voluntary task-switching paradigm","volume":"17","author":"Mayr","year":"2006","journal-title":"Psychological Science"},{"key":"bib38","volume-title":"Theory of Games and Economic Behavior","author":"Morgenstern","year":"1953"},{"key":"bib39","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.39404","article-title":"Control of entropy in neural models of environmental state","volume":"8","author":"Muller","year":"2019","journal-title":"eLife"},{"key":"bib40","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1073\/pnas.36.1.48","article-title":"Equilibrium points in N-Person games","volume":"36","author":"Nash","year":"1950","journal-title":"PNAS"},{"key":"bib41","doi-asserted-by":"publisher","first-page":"1040","DOI":"10.1038\/nn.3130","article-title":"Rational regulation of learning dynamics by pupil-linked arousal systems","volume":"15","author":"Nassar","year":"2012","journal-title":"Nature Neuroscience"},{"key":"bib42","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1177\/0956797612463080","article-title":"The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive","volume":"24","author":"Otto","year":"2013","journal-title":"Psychological Science"},{"key":"bib43","doi-asserted-by":"publisher","first-page":"20941","DOI":"10.1073\/pnas.1312011110","article-title":"Working-memory capacity protects model-based learning from stress","volume":"110","author":"Otto","year":"2013","journal-title":"PNAS"},{"key":"bib44","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1037\/0096-3445.121.3.352","article-title":"Generation of random series in two-person strictly competitive games","volume":"121","author":"Rapoport","year":"1992","journal-title":"Journal of Experimental Psychology: General"},{"key":"bib45","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1037\/0033-295X.104.3.603","article-title":"Randomization in individual choice behavior","volume":"104","author":"Rapoport","year":"1997","journal-title":"Psychological Review"},{"key":"bib46","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.tics.2015.11.002","article-title":"Forming beliefs: why valence matters","volume":"20","author":"Sharot","year":"2016","journal-title":"Trends in Cognitive Sciences"},{"key":"bib47","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.cell.2014.08.037","article-title":"Behavioral variability through stochastic choice and its gating by anterior cingulate cortex","volume":"159","author":"Tervo","year":"2014","journal-title":"Cell"},{"key":"bib48","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1037\/h0032060","article-title":"Generation of random sequences by human subjects: a critical survey of literature","volume":"77","author":"Wagenaar","year":"1972","journal-title":"Psychological Bulletin"},{"key":"bib49","doi-asserted-by":"publisher","first-page":"1521","DOI":"10.1257\/aer.91.5.1521","article-title":"Minimax play at Wimbledon","volume":"91","author":"Walker","year":"2001","journal-title":"American Economic Review"},{"key":"bib50","doi-asserted-by":"publisher","first-page":"2074","DOI":"10.1037\/a0038199","article-title":"Humans use directed and random exploration to solve the explore\u2013exploit dilemma","volume":"143","author":"Wilson","year":"2014","journal-title":"Journal of Experimental Psychology: General"},{"key":"bib51","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1016\/j.neuron.2009.01.012","article-title":"Trial outcome and associative learning signals in the monkey Hippocampus","volume":"61","author":"Wirth","year":"2009","journal-title":"Neuron"}],"container-title":["eLife"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/cdn.elifesciences.org\/articles\/48810\/elife-48810-v2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/cdn.elifesciences.org\/articles\/48810\/elife-48810-v2.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/elifesciences.org\/articles\/48810","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,11]],"date-time":"2023-10-11T21:01:45Z","timestamp":1697058105000},"score":1,"resource":{"primary":{"URL":"https:\/\/elifesciences.org\/articles\/48810"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,2]]},"references-count":51,"alternative-id":["10.7554\/eLife.48810"],"URL":"https:\/\/doi.org\/10.7554\/elife.48810","archive":["CLOCKSS"],"relation":{"is-supplemented-by":[{"id-type":"accession","id":"j6beq","asserted-by":"subject"}],"has-review":[{"id-type":"doi","id":"10.7554\/eLife.48810.029","asserted-by":"object"},{"id-type":"doi","id":"10.7554\/eLife.48810.028","asserted-by":"object"}],"has-preprint":[{"id-type":"doi","id":"10.1101\/662700","asserted-by":"object"}]},"ISSN":["2050-084X"],"issn-type":[{"value":"2050-084X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,2]]},"article-number":"e48810"}}