{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T00:44:22Z","timestamp":1776127462198,"version":"3.50.1"},"reference-count":66,"publisher":"Public Library of Science (PLoS)","issue":"6","license":[{"start":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T00:00:00Z","timestamp":1655164800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["863818"],"award-info":[{"award-number":["863818"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["850529"],"award-info":[{"award-number":["850529"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>In repeated interactions, players can use strategies that respond to the outcome of previous rounds. Much of the existing literature on direct reciprocity assumes that all competing individuals use the same strategy space. Here, we study both learning and evolutionary dynamics of players that differ in the strategy space they explore. We focus on the infinitely repeated donation game and compare three natural strategy spaces: memory-1 strategies, which consider the last moves of both players, reactive strategies, which respond to the last move of the co-player, and unconditional strategies. These three strategy spaces differ in the memory capacity that is needed. We compute the long term average payoff that is achieved in a pairwise learning process. We find that smaller strategy spaces can dominate larger ones. For weak selection, unconditional players dominate both reactive and memory-1 players. For intermediate selection, reactive players dominate memory-1 players. Only for strong selection and low cost-to-benefit ratio, memory-1 players dominate the others. We observe that the supergame between strategy spaces can be a social dilemma: maximum payoff is achieved if both players explore a larger strategy space, but smaller strategy spaces dominate.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1010149","type":"journal-article","created":{"date-parts":[[2022,6,14]],"date-time":"2022-06-14T17:33:51Z","timestamp":1655228031000},"page":"e1010149","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":24,"title":["Direct reciprocity between individuals that use different strategy spaces"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6978-7329","authenticated-orcid":true,"given":"Laura","family":"Schmid","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5116-955X","authenticated-orcid":true,"given":"Christian","family":"Hilbe","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4561-241X","authenticated-orcid":true,"given":"Krishnendu","family":"Chatterjee","sequence":"additional","affiliation":[]},{"given":"Martin A.","family":"Nowak","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,6,14]]},"reference":[{"issue":"5805","key":"pcbi.1010149.ref001","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1126\/science.1133755","article-title":"Five rules for the evolution of cooperation","volume":"314","author":"MA Nowak","year":"2006","journal-title":"Science"},{"key":"pcbi.1010149.ref002","doi-asserted-by":"crossref","DOI":"10.1515\/9781400832255","volume-title":"The Calculus of Selfishness","author":"K Sigmund","year":"2010"},{"issue":"7","key":"pcbi.1010149.ref003","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1038\/s41562-018-0320-9","article-title":"Partners and rivals in direct reciprocity","volume":"2","author":"C Hilbe","year":"2018","journal-title":"Nature Human Behaviour"},{"issue":"1","key":"pcbi.1010149.ref004","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1086\/406755","article-title":"The evolution of reciprocal altruism","volume":"46","author":"RL Trivers","year":"1971","journal-title":"The Quarterly review of biology"},{"issue":"4489","key":"pcbi.1010149.ref005","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1126\/science.7466396","article-title":"The evolution of cooperation","volume":"211","author":"R Axelrod","year":"1981","journal-title":"Science"},{"key":"pcbi.1010149.ref006","doi-asserted-by":"crossref","DOI":"10.3998\/mpub.20269","volume-title":"Prisoner\u2019s dilemma: A study in conflict and cooperation","author":"A Rapoport","year":"1965"},{"key":"pcbi.1010149.ref007","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195300796.001.0001","volume-title":"Repeated games and reputations: long-run relationships","author":"GJ Mailath","year":"2006"},{"issue":"7713","key":"pcbi.1010149.ref008","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1038\/s41586-018-0277-x","article-title":"Evolution of cooperation in stochastic games","volume":"559","author":"C Hilbe","year":"2018","journal-title":"Nature"},{"issue":"25","key":"pcbi.1010149.ref009","doi-asserted-by":"crossref","first-page":"9929","DOI":"10.1073\/pnas.1206694109","article-title":"Direct reciprocity in structured populations","volume":"109","author":"M van Veelen","year":"2012","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"4","key":"pcbi.1010149.ref010","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1086\/342070","article-title":"The Continuous Prisoner\u2019s Dilemma and the Evolution of Cooperation through Reciprocal Altruism with Variable Investment","volume":"160","author":"T Killingback","year":"2002","journal-title":"The American Naturalist"},{"issue":"6103","key":"pcbi.1010149.ref011","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1038\/325433a0","article-title":"Tit for tat in sticklebacks and the evolution of cooperation","volume":"325","author":"M Milinski","year":"1987","journal-title":"nature"},{"key":"pcbi.1010149.ref012","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1038\/308181a0","article-title":"Reciprocal food sharing in the vampire bat","volume":"308","author":"GS Wilkinson","year":"1984","journal-title":"Nature"},{"key":"pcbi.1010149.ref013","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/BF00049570","article-title":"The evolution of stochastic strategies in the prisoner\u2019s dilemma","volume":"20","author":"MA Nowak","year":"1990","journal-title":"Acta Applicandae Mathematicae"},{"key":"pcbi.1010149.ref014","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1038\/355250a0","article-title":"Tit for tat in heterogeneous populations","volume":"355","author":"MA Nowak","year":"1992","journal-title":"Nature"},{"key":"pcbi.1010149.ref015","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1177\/0022002785029004004","article-title":"The optimal level of generosity in a selfish, uncertain environment","volume":"29","author":"P Molander","year":"1985","journal-title":"Journal of Conflict Resolution"},{"issue":"6432","key":"pcbi.1010149.ref016","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1038\/364056a0","article-title":"A strategy of win-stay, lose-shift that outperforms tit-for-tat in the Prisoner\u2019s Dilemma game","volume":"364","author":"M Nowak","year":"1993","journal-title":"Nature"},{"key":"pcbi.1010149.ref017","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/BF01074955","article-title":"Learning to cooperate with Pavlov an adaptive strategy for the iterated prisoner\u2019s dilemma with noise","volume":"35","author":"DP Kraines","year":"1993","journal-title":"Theory and Decision"},{"issue":"1381","key":"pcbi.1010149.ref018","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1098\/rspb.1997.0073","article-title":"Effects of increasing the number of players and memory size in the iterated Prisoner\u2019s Dilemma: a numerical approach","volume":"264","author":"C Hauert","year":"1997","journal-title":"Proceedings of the Royal Society of London B: Biological Sciences"},{"key":"pcbi.1010149.ref019","first-page":"337","volume-title":"The Economy as an Evolving Complex System II","author":"K Lindgren","year":"1997"},{"issue":"18","key":"pcbi.1010149.ref020","doi-asserted-by":"crossref","first-page":"4715","DOI":"10.1073\/pnas.1621239114","article-title":"Memory-n strategies of direct reciprocity","volume":"114","author":"C Hilbe","year":"2017","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1010149.ref021","doi-asserted-by":"crossref","first-page":"25676","DOI":"10.1038\/srep25676","article-title":"Comparing reactive and memory-one strategies of direct reciprocity","volume":"6","author":"SK Baek","year":"2016","journal-title":"Scientific reports"},{"issue":"1","key":"pcbi.1010149.ref022","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-73855-x","article-title":"Five rules for friendly rivalry in direct reciprocity","volume":"10","author":"Y Murase","year":"2020","journal-title":"Scientific reports"},{"issue":"3","key":"pcbi.1010149.ref023","doi-asserted-by":"crossref","first-page":"e0120625","DOI":"10.1371\/journal.pone.0120625","article-title":"The art of war: Beyond memory-one strategies in population games","volume":"10","author":"C Lee","year":"2015","journal-title":"PLoS One"},{"issue":"12","key":"pcbi.1010149.ref024","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0188046","article-title":"Reinforcement learning produces dominant strategies for the Iterated Prisoner\u2019s Dilemma","volume":"12","author":"M Harper","year":"2017","journal-title":"PLOS ONE"},{"issue":"1","key":"pcbi.1010149.ref025","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-74181-y","article-title":"Using a theory of mind to find best responses to memory-one strategies","volume":"10","author":"NE Glynatsi","year":"2020","journal-title":"Scientific Reports"},{"issue":"26","key":"pcbi.1010149.ref026","doi-asserted-by":"crossref","first-page":"10409","DOI":"10.1073\/pnas.1206569109","article-title":"Iterated Prisoner\u2019s Dilemma contains strategies that dominate any evolutionary opponent","volume":"109","author":"WH Press","year":"2012","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1010149.ref027","doi-asserted-by":"crossref","first-page":"022804","DOI":"10.1103\/PhysRevE.89.022804","article-title":"Evolution of extortion in structured populations","volume":"89","author":"A Szolnoki","year":"2014","journal-title":"Physical Review E"},{"issue":"5","key":"pcbi.1010149.ref028","doi-asserted-by":"crossref","first-page":"052803","DOI":"10.1103\/PhysRevE.91.052803","article-title":"Extortion under uncertainty: Zero-determinant strategies in noisy games","volume":"91","author":"D Hao","year":"2015","journal-title":"Physical Review E"},{"issue":"13","key":"pcbi.1010149.ref029","doi-asserted-by":"crossref","first-page":"3573","DOI":"10.1073\/pnas.1520163113","article-title":"Autocratic strategies for iterated games with arbitrary action spaces","volume":"113","author":"A McAvoy","year":"2016","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1010149.ref030","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.jtbi.2017.11.002","article-title":"Zero-determinant strategies in finitely repeated games","volume":"438","author":"G Ichinose","year":"2018","journal-title":"Journal of Theoretical Biology"},{"issue":"49","key":"pcbi.1010149.ref031","doi-asserted-by":"crossref","first-page":"17558","DOI":"10.1073\/pnas.1408618111","article-title":"Collapse of cooperation in evolving games","volume":"111","author":"AJ Stewart","year":"2014","journal-title":"Proceedings of the National Academy of Sciences USA"},{"issue":"38","key":"pcbi.1010149.ref032","doi-asserted-by":"crossref","first-page":"15348","DOI":"10.1073\/pnas.1306246110","article-title":"From extortion to generosity, evolution in the Iterated Prisoner\u2019s Dilemma","volume":"110","author":"AJ Stewart","year":"2013","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"3","key":"pcbi.1010149.ref033","doi-asserted-by":"crossref","first-page":"175","DOI":"10.3390\/g6030175","article-title":"What you gotta know to play good in the iterated prisoner\u2019s dilemma","volume":"6","author":"E Akin","year":"2015","journal-title":"Games"},{"key":"pcbi.1010149.ref034","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1515\/9783110461510-004","volume-title":"Ergodic Theory, Advances in Dynamics","author":"E Akin","year":"2016"},{"key":"pcbi.1010149.ref035","doi-asserted-by":"crossref","first-page":"26889","DOI":"10.1038\/srep26889","article-title":"Small groups and long memories promote cooperation","volume":"6","author":"AJ Stewart","year":"2016","journal-title":"Scientific Reports"},{"issue":"23","key":"pcbi.1010149.ref036","doi-asserted-by":"crossref","first-page":"13755","DOI":"10.1073\/pnas.95.23.13755","article-title":"Working memory constrains human cooperation in the Prisoner\u2019s Dilemma","volume":"95","author":"M Milinski","year":"1998","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1010149.ref037","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.socec.2014.01.006","article-title":"Cognitive load in the multi-player prisoner\u2019s dilemma game: Are there brains in games?","volume":"51","author":"S Duffy","year":"2014","journal-title":"Journal of Behavioral and Experimental Economics"},{"key":"pcbi.1010149.ref038","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/0025-5564(78)90077-9","article-title":"Evolutionarily stable strategies and game dynamics","volume":"40","author":"PD Taylor","year":"1978","journal-title":"Mathematical Biosciences"},{"issue":"45","key":"pcbi.1010149.ref039","doi-asserted-by":"crossref","first-page":"19061","DOI":"10.1073\/pnas.0904357106","article-title":"A theory for the evolution of other-regard integrating proximate and ultimate perspectives","volume":"106","author":"E Ak\u00e7ay","year":"2009","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"10","key":"pcbi.1010149.ref040","doi-asserted-by":"crossref","first-page":"1305","DOI":"10.1163\/156853903771980602","article-title":"The Evolution of Social Dominance I: Two-player Models","volume":"140","author":"G Hengeveld","year":"2003","journal-title":"Behaviour"},{"key":"pcbi.1010149.ref041","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.jtbi.2003.11.008","article-title":"Reactive strategies in indirect reciprocity","volume":"227","author":"H Ohtsuki","year":"2004","journal-title":"Journal of Theoretical Biology"},{"issue":"1680","key":"pcbi.1010149.ref042","first-page":"463","article-title":"Stochastic evolutionary dynamics of direct reciprocity","volume":"277","author":"LA Imhof","year":"2010","journal-title":"Proceedings of the Royal Society of London B: Biological Sciences"},{"key":"pcbi.1010149.ref043","doi-asserted-by":"crossref","first-page":"e35287","DOI":"10.1371\/journal.pone.0035287","article-title":"The Structure of Mutations and the Evolution of Cooperation","volume":"7","author":"J Garc\u00eda","year":"2012","journal-title":"PLoS One"},{"issue":"1","key":"pcbi.1010149.ref044","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1038\/s41467-017-02721-8","article-title":"Crosstalk in concurrent repeated games impedes direct reciprocity and requires stronger levels of forgiveness","volume":"9","author":"JG Reiter","year":"2018","journal-title":"Nature Communications"},{"key":"pcbi.1010149.ref045","article-title":"A unified framework of direct and indirect reciprocity","author":"L Schmid","year":"2021","journal-title":"Nature Human Behaviour"},{"issue":"11","key":"pcbi.1010149.ref046","doi-asserted-by":"crossref","first-page":"3929","DOI":"10.1257\/aer.20181480","article-title":"Strategy choice in the infinitely repeated prisoner\u2019s dilemma","volume":"109","author":"P Dal B\u00f3","year":"2019","journal-title":"American Economic Review"},{"key":"pcbi.1010149.ref047","doi-asserted-by":"crossref","first-page":"284","DOI":"10.3758\/s13420-019-00394-5","article-title":"Reciprocity: Different behavioural strategies, cognitive mechanisms and psychological processes","volume":"47","author":"MK Schweinfurth","year":"2019","journal-title":"Learning and Behavior"},{"issue":"7770","key":"pcbi.1010149.ref048","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1038\/s41586-019-1488-5","article-title":"Social dilemmas among unequals","volume":"572","author":"OP Hauser","year":"2019","journal-title":"Nature"},{"key":"pcbi.1010149.ref049","doi-asserted-by":"crossref","unstructured":"McAvoy A, Kates-Harbeck J, Chatterjee K, Hilbe C. Evolutionary (in)stability of selifsh learning in repeated games. arXiv. 2021;2105.06199.","DOI":"10.1093\/pnasnexus\/pgac141"},{"key":"pcbi.1010149.ref050","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139173179","volume-title":"Evolutionary Games and Population Dynamics","author":"J Hofbauer","year":"1998"},{"issue":"3","key":"pcbi.1010149.ref051","doi-asserted-by":"crossref","first-page":"341","DOI":"10.3982\/TE771","article-title":"Survival of dominated strategies under evolutionary dynamics","volume":"6","author":"J Hofbauer","year":"2011","journal-title":"Theoretical Economics"},{"key":"pcbi.1010149.ref052","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/BF00318088","article-title":"Lotka-Volterra equation and replicator dynamics: A two-dimensional classification","volume":"48","author":"I Bomze","year":"1983","journal-title":"Biological Cybernetics"},{"issue":"737","key":"pcbi.1010149.ref053","article-title":"Cooperation in alternating interactions with memory constraints","volume":"13","author":"PS Park","year":"2022","journal-title":"Nature Communications"},{"issue":"7","key":"pcbi.1010149.ref054","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1007\/BF00184645","article-title":"Automata, repeated games and noise","volume":"33","author":"MA Nowak","year":"1995","journal-title":"Journal of Mathematical Biology"},{"key":"pcbi.1010149.ref055","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.geb.2015.05.005","article-title":"Partners or rivals? Strategies for the iterated prisoner\u2019s dilemma","volume":"92","author":"C Hilbe","year":"2015","journal-title":"Games and economic behavior"},{"key":"pcbi.1010149.ref056","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pone.0045150","article-title":"The Pace of Cultural Evolution","volume":"7","author":"C Perreault","year":"2012","journal-title":"PLOS ONE"},{"issue":"Supplement 2","key":"pcbi.1010149.ref057","doi-asserted-by":"crossref","first-page":"8985","DOI":"10.1073\/pnas.0914631107","article-title":"Gene-culture coevolution in the age of genomics","volume":"107","author":"PJ Richerson","year":"2010","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"S1","key":"pcbi.1010149.ref058","doi-asserted-by":"crossref","first-page":"S149","DOI":"10.1086\/650295","article-title":"Working Memory Capacity and the Evolution of Modern Cognitive Potential: Implications from Animal and Early Human Tool Use","volume":"51","author":"MN Haidle","year":"2010","journal-title":"Current Anthropology"},{"key":"pcbi.1010149.ref059","volume-title":"Not by genes alone: How culture transformed human evolution","author":"PJ Richerson","year":"2008"},{"key":"pcbi.1010149.ref060","doi-asserted-by":"crossref","unstructured":"Fujimoto Y, Kaneko K. Exploitation by asymmetry of information reference in coevolutionary learning in prisoner\u2019s dilemma game; 2021.","DOI":"10.1088\/2632-072X\/ac301a"},{"key":"pcbi.1010149.ref061","doi-asserted-by":"crossref","unstructured":"Hertwig R, Engel C. Deliberate ignorance: Choosing not to know. Strungmann Forum Reports; 2021.","DOI":"10.7551\/mitpress\/13757.001.0001"},{"issue":"3","key":"pcbi.1010149.ref062","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1177\/1745691616635594","article-title":"Homo ignorans: Deliberately choosing not to know","volume":"11","author":"R Hertwig","year":"2016","journal-title":"Perspectives on Psychological Science"},{"key":"pcbi.1010149.ref063","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/BF02067272","article-title":"Payoffs and strategies in territorial contests: ESS analyses of two ecotypes of the spiderAgelenopsis aperta","volume":"2","author":"P Hammerstein","year":"1988","journal-title":"Evolutionary Ecology"},{"key":"pcbi.1010149.ref064","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.geb.2019.02.013","article-title":"In and out of equilibrium II: Evolution in repeated games with discounting and complexity costs","volume":"115","author":"M van Veelen","year":"2019","journal-title":"Games and Economic Behavior"},{"issue":"2","key":"pcbi.1010149.ref065","first-page":"274","article-title":"Evolution and Cooperation in Noisy Repeated Games","volume":"80","author":"D Fudenberg","year":"1990","journal-title":"The American Economic Review"},{"key":"pcbi.1010149.ref066","unstructured":"Sandholm WH, Dokumaci E, Franchetti F. Dynamo: Diagrams for Evolutionary Game Dynamics."}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010149","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T06:14:52Z","timestamp":1675836892000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1010149"}},"subtitle":[],"editor":[{"given":"Lusha","family":"Zhu","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,6,14]]},"references-count":66,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,6,14]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1010149","relation":{},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,14]]}}}