{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T00:52:34Z","timestamp":1791852754688,"version":"4.3.4"},"reference-count":42,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,4,10]],"date-time":"2019-04-10T00:00:00Z","timestamp":1554854400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Games"],"abstract":"<jats:p>We make use of data from a Facebook application where hundreds of thousands of people played a simultaneous move, zero-sum game\u2014rock-paper-scissors\u2014with varying information to analyze whether play in strategic settings is consistent with extant theories. We report three main insights. First, we observe that most people employ strategies consistent with Nash, at least some of the time. Second, however, players strategically use information on previous play of their opponents, a non-Nash equilibrium behavior; they are more likely to do so when the expected payoffs for such actions increase. Third, experience matters: players with more experience use information on their opponents more effectively than less experienced players, and are more likely to win as a result. We also explore the degree to which the deviations from Nash predictions are consistent with various non-equilibrium models. We analyze both a level-k framework and an adapted quantal response model. The naive version of each these strategies\u2014where players maximize the probability of winning without considering the probability of losing\u2014does better than the standard formulation. While one set of people use strategies that resemble quantal response, there is another group of people who employ strategies that are close to     k 1; for naive strategies the latter group is much larger.<\/jats:p>","DOI":"10.3390\/g10020018","type":"journal-article","created":{"date-parts":[[2019,4,10]],"date-time":"2019-04-10T11:25:08Z","timestamp":1554895508000},"page":"18","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Behavior in Strategic Settings: Evidence from a Million Rock-Paper-Scissors Games"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8580-6537","authenticated-orcid":false,"given":"Dimitris","family":"Batzilis","sequence":"first","affiliation":[{"name":"Department of Economics, American College of Greece\u2014Deree, Agia Paraskevi 15342, Greece"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8924-0294","authenticated-orcid":false,"given":"Sonia","family":"Jaffe","sequence":"additional","affiliation":[{"name":"Office of the Chief Economist, Microsoft, Redmond, WA 98052, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven","family":"Levitt","sequence":"additional","affiliation":[{"name":"Department of Economics, University of Chicago, Chicago, IL 60637, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John A.","family":"List","sequence":"additional","affiliation":[{"name":"Department of Economics, University of Chicago, Chicago, IL 60637, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffrey","family":"Picel","sequence":"additional","affiliation":[{"name":"Department of Economics, Harvard University, Cambridge, MA 02138, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,10]]},"reference":[{"key":"ref_1","unstructured":"Camerer, C. (2003). Behavioral Game Theory: Experiments in Strategic Interaction, Princeton University Press."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kagel, J.H., and Roth, A.E. (1995). The Handbook of Experimental Economics, Princeton University Press.","DOI":"10.1515\/9780691213255"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1138","DOI":"10.1257\/00028280260344678","article-title":"Testing mixed-strategy equilibria when players are heterogeneous: The case of penalty kicks in soccer","volume":"92","author":"Chiappori","year":"2002","journal-title":"Am. Econ. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1717","DOI":"10.1257\/0002828043052187","article-title":"Social comparisons and pro-social behavior: Testing \u201cconditional cooperation\u201d in a field experiment","volume":"94","author":"Frey","year":"2004","journal-title":"Am. Econ. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1688","DOI":"10.1257\/000282805775014407","article-title":"Using experimental economics to measure social capital and predict financial decisions","volume":"95","author":"Karlan","year":"2005","journal-title":"Am. Econ. Rev."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1257\/aer.91.5.1521","article-title":"Minimax play at Wimbledon","volume":"91","author":"Walker","year":"2001","journal-title":"Am. Econ. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"8817","DOI":"10.1038\/srep08817","article-title":"An experimental investigation of evolutionary dynamics in the Rock-Paper-Scissors game","volume":"5","author":"Hoffman","year":"2015","journal-title":"Sci. Rep."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1093\/restud\/rdt023","article-title":"Cycles and instability in a rock\u2013paper\u2013scissors population game: A continuous time experiment","volume":"81","author":"Cason","year":"2013","journal-title":"Rev. Econ. Stud."},{"key":"ref_9","unstructured":"Koopmans, T. (1951). Iterative Solutions of Games by Fictitious Play. Activity Analysis of Production and Allocation, Wiley."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/j.jet.2004.02.003","article-title":"Fictitious Play in 2xN Games","volume":"120","author":"Berger","year":"2005","journal-title":"J. Econ. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"572","DOI":"10.1016\/j.jet.2005.12.010","article-title":"Brown\u2019s original fictitious play","volume":"135","author":"Berger","year":"2007","journal-title":"J. Econ. Theory"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"57","DOI":"10.2307\/2951778","article-title":"The Evolution of Conventions","volume":"61","author":"Young","year":"1993","journal-title":"Econometrica"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1006\/game.1994.1037","article-title":"Learning Behavior in an Experimental Matching Pennies Game","volume":"7","author":"Mookherjee","year":"1994","journal-title":"Games Econ. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1006\/game.1997.0540","article-title":"Learning and Decision Costs in Experimental Constant Sum Games","volume":"19","author":"Mookherjee","year":"1997","journal-title":"Games Econ. Behav."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1006\/game.1993.1033","article-title":"Evolution of Smartn Players","volume":"5","author":"Stahl","year":"1993","journal-title":"Games Econ. Behav."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/0167-2681(94)90103-1","article-title":"Experimental evidence on players\u2019 models of other players","volume":"25","author":"Stahl","year":"1994","journal-title":"J. Econ. Behav. Organ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1006\/game.1995.1031","article-title":"On Players\u2019 Models of Other Players: Theory and Experimental Evidence","volume":"10","author":"Stahl","year":"1995","journal-title":"Games Econ. Behav."},{"key":"ref_18","first-page":"1313","article-title":"Unraveling in guessing games: An experimental study","volume":"85","author":"Nagel","year":"1995","journal-title":"Am. Econ. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1737","DOI":"10.1257\/aer.96.5.1737","article-title":"Cognition and behavior in two-person guessing games: An experimental study","volume":"96","author":"Crawford","year":"2006","journal-title":"Am. Econ. Rev."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1731","DOI":"10.1257\/aer.97.5.1731","article-title":"Fatal attraction: Salience, naivete, and sophistication in experimental \u201cHide-and-Seek\u201d games","volume":"97","author":"Crawford","year":"2007","journal-title":"Am. Econ. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"3561","DOI":"10.1257\/aer.102.7.3561","article-title":"The 11\u201320 Money Request Game: A Level-k Reasoning Study","volume":"102","author":"Arad","year":"2012","journal-title":"Am. Econ. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1111\/1468-0262.00239","article-title":"Cognition and Behavior in Normal-Form Games: An Experimental Study","volume":"69","author":"Crawford","year":"2001","journal-title":"Econometrica"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0010-0277(02)00054-9","article-title":"What do you think I think you think?: Strategic reasoning in matrix games","volume":"85","author":"Hedden","year":"2002","journal-title":"Cognition"},{"key":"ref_24","first-page":"947","article-title":"Iterated dominance and iterated best response in experimental \u201cp-beauty contests\u201d","volume":"88","author":"Ho","year":"1998","journal-title":"Am. Econ. Rev."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1687","DOI":"10.1257\/000282802762024737","article-title":"One, two, (three), infinity,...: Newspaper and lab beauty-contest experiments","volume":"92","author":"Montalvo","year":"2002","journal-title":"Am. Econ. Rev."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1257\/mic.3.3.1","article-title":"Testing game theory in the field: Swedish LUPI lottery games","volume":"3","author":"Ostling","year":"2011","journal-title":"Am. Econ. J. Microecon."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Gillen, B. (2009). Identification and estimation of level-k auctions. SSRN Electron. J.","DOI":"10.2139\/ssrn.1337843"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"3130","DOI":"10.1257\/aer.101.7.3130","article-title":"Who thinks about the competition? Managerial ability and strategic entry in US local telephone markets","volume":"101","author":"Goldfarb","year":"2011","journal-title":"Am. Econ. Rev."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1257\/mic.4.2.1","article-title":"To review or not to review? Limited strategic thinking at the movie box office","volume":"4","author":"Brown","year":"2012","journal-title":"Am. Econ. J. Microecon."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1006\/game.1995.1023","article-title":"Quantal response equilibria for normal form games","volume":"10","author":"McKelvey","year":"1995","journal-title":"Games Econ. Behav."},{"key":"ref_31","unstructured":"Vogel, C. (2013, December 12). Rock, Paper, Payoff: Child\u2019s Play Wins Auction House an Art Sale. Available online: https:\/\/www.nytimes.com\/2005\/04\/29\/arts\/design\/rock-paper-payoff-childs-play-wins-auction-house-an-art-sale.html."},{"key":"ref_32","unstructured":"Facebook (2013, December 12). Developer Partner Case Study: Red Bull Application. Available online: https:\/\/allfacebook.de\/wp-content\/uploads\/2010\/04\/81162-RedBull-Case-Study-FINAL.pdf."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/S0899-8256(03)00002-2","article-title":"\u2018Learning\u2019 with no feedback in a competitive guessing game","volume":"44","author":"Weber","year":"2003","journal-title":"Games Econ. Behav."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"258","DOI":"10.1006\/jeth.1996.0014","article-title":"Fictitious play property for games with identical interests","volume":"68","author":"Monderer","year":"1996","journal-title":"J. Econ. Theory"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Georganas, S., Healy, P.J., and Weber, R.A. (2014, May 15). On the Persistence of Strategic Sophistication. Ohio State Working Paper Series No. 4653. Available online: https:\/\/ssrn.com\/abstract=2407895.","DOI":"10.2139\/ssrn.2407895"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"141","DOI":"10.3982\/QE647","article-title":"An Empirical Model of Non-Equilibrium Behavior in Games","volume":"9","author":"Kline","year":"2018","journal-title":"Quant. Econ."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hahn, P.R., Goswami, I., and Mela, C. (arXiv, 2014). A Bayesian hierarchical model for inferring player strategy types in a number guessing game, arXiv.","DOI":"10.1214\/15-AOAS830"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1111\/j.1468-0262.2004.00512.x","article-title":"Behavior in a dynamic decision problem: An analysis of experimental evidence using a Bayesian type classification algorithm","volume":"72","author":"Houser","year":"2004","journal-title":"Econometrica"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1162\/0033553041502225","article-title":"A cognitive hierarchy model of games","volume":"119","author":"Camerer","year":"2004","journal-title":"Q. J. Econ."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.geb.2017.09.009","article-title":"Predicting human behavior in unrepeated, simultaneous-move games","volume":"106","author":"Wright","year":"2017","journal-title":"Games Econ. Behav."},{"key":"ref_41","unstructured":"(1996). Daniel McFadden uantal Choice Analysis: A survey. Ann. Econ. Soc. Meas., 5, 390\u2013393."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1006\/game.2000.0808","article-title":"On the irrelevance of risk attitudes in repeated two-outcome games","volume":"34","author":"Wooders","year":"2001","journal-title":"Games Econ. Behav."}],"container-title":["Games"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-4336\/10\/2\/18\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:44:24Z","timestamp":1760186664000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-4336\/10\/2\/18"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,10]]},"references-count":42,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["g10020018"],"URL":"https:\/\/doi.org\/10.3390\/g10020018","relation":{},"ISSN":["2073-4336"],"issn-type":[{"value":"2073-4336","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,10]]}}}