{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,6]],"date-time":"2025-11-06T15:49:33Z","timestamp":1762444173186},"reference-count":44,"publisher":"MIT Press - Journals","issue":"10","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2009,10]]},"abstract":"<jats:p> Over the past several decades, economists, psychologists, and neuroscientists have conducted experiments in which a subject, human or animal, repeatedly chooses between alternative actions and is rewarded based on choice history. While individual choices are unpredictable, aggregate behavior typically follows Herrnstein's matching law: the average reward per choice is equal for all chosen alternatives. In general, matching behavior does not maximize the overall reward delivered to the subject, and therefore matching appears inconsistent with the principle of utility maximization. Here we show that matching can be made consistent with maximization by regarding the choices of a single subject as being made by a sequence of multiple selves\u2014one for each instant of time. If each self is blind to the state of the world and discounts future rewards completely, then the resulting game has at least one Nash equilibrium that satisfies both Herrnstein's matching law and the unpredictability of individual choices. This equilibrium is, in general, Pareto suboptimal, and can be understood as a mutual defection of the multiple selves in an intertemporal prisoner's dilemma. <\/jats:p><jats:p> The mathematical assumptions about the multiple selves should not be interpreted literally as psychological assumptions. Human and animals do remember past choices and care about future rewards. However, they may be unable to comprehend or take into account the relationship between past and future. This can be made more explicit when a mechanism that converges on the equilibrium, such as reinforcement learning, is considered. <\/jats:p><jats:p> Using specific examples, we show that there exist behaviors that satisfy the matching law but are not Nash equilibria. We expect that these behaviors will not be observed experimentally in animals and humans. If this is the case, the Nash equilibrium formulation can be regarded as a refinement of Herrnstein's matching law. <\/jats:p>","DOI":"10.1162\/neco.2009.09-08-854","type":"journal-article","created":{"date-parts":[[2009,7,27]],"date-time":"2009-07-27T20:17:41Z","timestamp":1248725861000},"page":"2755-2773","source":"Crossref","is-referenced-by-count":17,"title":["Operant Matching as a Nash Equilibrium of an Intertemporal Game"],"prefix":"10.1162","volume":"21","author":[{"given":"Yonatan","family":"Loewenstein","sequence":"first","affiliation":[{"name":"Departments of Neurobiology and Cognitive Sciences and the Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem, 91904, Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Drazen","family":"Prelec","sequence":"additional","affiliation":[{"name":"Sloan School of Management, Massachusetts Institute of Technology, Cambridge MA 02139, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"H. Sebastian","family":"Seung","sequence":"additional","affiliation":[{"name":"Howard Hughes Medical Institute and Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA 02139, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","volume-title":"Picoeconomics: The strategic interaction of successive motivational states within the person","author":"Ainslie G.","year":"1992"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1006\/game.1997.0577"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1613\/jair.806"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1037\/14496-000"},{"key":"B5","volume-title":"Proceedings of the AAMAS-04 Workshop on Game-Theoretic and Decision-Theoretic Agents","author":"Cheng S. F.","year":"2004"},{"key":"B6","volume-title":"The matching law: A research review","author":"Davison M.","year":"1988"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1162\/089892998563022"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1257\/002205102320161311"},{"key":"B9","volume-title":"The theory of learning in games","author":"Fudenberg D.","year":"1998"},{"key":"B10","volume-title":"Game theory","author":"Fudenberg D.","year":"1991"},{"key":"B11","volume-title":"The organization of learning","author":"Gallistel C. R.","year":"1990"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1037\/0097-7403.27.4.354"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1006\/lmot.2001.1099"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.3758\/BF03210960"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1006\/game.1997.0508"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/2302.001.0001"},{"issue":"3","key":"B17","first-page":"247","volume":"105","author":"Hammond P. J.","year":"1997","journal-title":"Rivista internazionale di scienzesociali"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1006\/game.1996.0018"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.3960060302"},{"key":"B20","first-page":"107","volume":"1975","author":"Herrnstein R. J.","year":"2001","journal-title":"J. Exp. Anal. Behav."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1257\/jep.5.3.137"},{"key":"B22","volume-title":"IX Symposium on the Quantitative Analysis of Behavior","author":"Herrnstein R. J.","year":"1986"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1016\/S0376-8716(02)00071-6"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.3758\/BF03209261"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1090\/S0273-0979-03-00988-1"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1162\/003355397555253"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1000007"},{"key":"B28","volume-title":"Abstract viewer Itinerary planner","author":"Loewenstein Y.","year":"2007"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0505220103"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1126\/science.7292017"},{"key":"B32","volume-title":"Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence","author":"McAllester D.","year":"1999"},{"key":"B33","first-page":"79","volume":"2007","author":"Neiman T.","year":"2007","journal-title":"Neural Plasticity"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1006\/game.1997.0536"},{"key":"B35","unstructured":"Rachlin, H. & Laibson, D. I. (Eds.). (1997): The matching law: Papers in psychology and economics. Cambridge, MA: Harvard University Press."},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2008.20.1.227"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(03)00761-X"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.413"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5159-05.2006"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.2307\/2295722"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1126\/science.1094765"},{"key":"B42","volume-title":"Reinforcement learning: An introduction","author":"Sutton R. S.","year":"1998"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1901\/jeab.1981.36-141"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.1111\/1467-6419.00106"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1901\/jeab.1992.58-325"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco.2009.09-08-854","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:37:40Z","timestamp":1615585060000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/21\/10\/2755-2773\/7418"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2009,10]]},"references-count":44,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2009,10]]}},"alternative-id":["10.1162\/neco.2009.09-08-854"],"URL":"https:\/\/doi.org\/10.1162\/neco.2009.09-08-854","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2009,10]]}}}