{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T22:03:39Z","timestamp":1766441019123,"version":"3.37.3"},"reference-count":72,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,10,12]],"date-time":"2024-10-12T00:00:00Z","timestamp":1728691200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,10,12]],"date-time":"2024-10-12T00:00:00Z","timestamp":1728691200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100014013","name":"UK Research and Innovation","doi-asserted-by":"publisher","award":["EP\/S023356\/1"],"award-info":[{"award-number":["EP\/S023356\/1"]}],"id":[{"id":"10.13039\/100014013","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","award":["EP\/T517380\/1"],"award-info":[{"award-number":["EP\/T517380\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Agent Multi-Agent Syst"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Social dilemmas present a significant challenge in multi-agent cooperation because individuals are incentivised to behave in ways that undermine socially optimal outcomes. Consequently, self-interested agents often avoid collective behaviour. In response, we formalise social dilemmas and introduce a novel metric, the <jats:italic>general self-interest level<\/jats:italic>, to quantify the disparity between individual and group rationality in such scenarios. This metric represents the maximum proportion of their individual rewards that agents can retain while ensuring that a social welfare optimum becomes a dominant strategy. Our approach diverges from traditional concepts of altruism, instead focusing on strategic reward redistribution. By transferring rewards among agents in a manner that aligns individual and group incentives, rational agents will maximise collective welfare while pursuing their own interests. We provide an algorithm to compute efficient transfer structures for an arbitrary number of agents, and introduce novel multi-player social dilemma games to illustrate the effectiveness of our method. This work provides both a descriptive tool for analysing social dilemmas and a prescriptive solution for resolving them via efficient reward transfer contracts. Applications include mechanism design, where we can assess the impact on collaborative behaviour of modifications to models of environments.<\/jats:p>","DOI":"10.1007\/s10458-024-09675-4","type":"journal-article","created":{"date-parts":[[2024,10,12]],"date-time":"2024-10-12T03:16:38Z","timestamp":1728702998000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Resolving social dilemmas with minimal reward transfer"],"prefix":"10.1007","volume":"38","author":[{"given":"Richard","family":"Willis","sequence":"first","affiliation":[]},{"given":"Yali","family":"Du","sequence":"additional","affiliation":[]},{"given":"Joel Z.","family":"Leibo","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Luck","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,10,12]]},"reference":[{"issue":"2","key":"9675_CR1","doi-asserted-by":"publisher","first-page":"404","DOI":"10.2307\/1964229","volume":"86","author":"E Ostrom","year":"1992","unstructured":"Ostrom, E., Walker, J., & Gardner, R. (1992). Covenants with and without a Sword: Self-Governance Is Possible. American Political Science Review, 86(2), 404\u2013417. https:\/\/doi.org\/10.2307\/1964229","journal-title":"American Political Science Review"},{"issue":"5652","key":"9675_CR2","doi-asserted-by":"publisher","first-page":"1907","DOI":"10.1126\/science.1091015","volume":"302","author":"T Dietz","year":"2003","unstructured":"Dietz, T., Ostrom, E., & Stern, P. C. (2003). The Struggle to Govern the Commons. Science, 302(5652), 1907\u20131912. https:\/\/doi.org\/10.1126\/science.1091015","journal-title":"Science"},{"key":"9675_CR3","doi-asserted-by":"publisher","first-page":"207","DOI":"10.1613\/jair.4164","volume":"49","author":"KR Apt","year":"2014","unstructured":"Apt, K. R., & Schaefer, G. (2014). Selfishness Level of Strategic Games. Journal of Artificial Intelligence Research, 49, 207\u2013240. https:\/\/doi.org\/10.1613\/jair.4164","journal-title":"Journal of Artificial Intelligence Research"},{"key":"9675_CR4","doi-asserted-by":"publisher","unstructured":"Deng, Y. & Conitzer, V. Disarmament Games with Resources. In: Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, pp. 981\u2013988. AAAI Press, New Orleans, Louisiana, USA (2018-02-02\/2018-02-07). https:\/\/doi.org\/10.1609\/aaai.v32i1.11443.","DOI":"10.1609\/aaai.v32i1.11443"},{"issue":"5805","key":"9675_CR5","doi-asserted-by":"publisher","first-page":"1560","DOI":"10.1126\/science.1133755","volume":"314","author":"MA Nowak","year":"2006","unstructured":"Nowak, M. A. (2006). Five Rules for the Evolution of Cooperation. Science, 314(5805), 1560\u20131563. https:\/\/doi.org\/10.1126\/science.1133755","journal-title":"Science"},{"key":"9675_CR6","unstructured":"Hughes, E., Leibo, J.Z., Phillips, M., Tuyls, K., Due\u00f1ez-Guzman, E., Casta\u00f1eda, A.G., Dunning, I., Zhu, T., McKee, K., Koster, R., Roff, H., & Graepel, T. (2018). Inequity aversion improves cooperation in intertemporal social dilemmas. In: 32nd Conference on Neural Information Processing Systems, pp. 3330\u20133340. Curran Associates, Inc., Montr\u00e9al, Canada."},{"key":"9675_CR7","doi-asserted-by":"publisher","unstructured":"Hughes, E., Anthony, T.W., Eccles, T., Leibo, J.Z., Balduzzi, D., & Bachrach, Y. Learning to Resolve Alliance Dilemmas in Many-Player Zero-Sum Games. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, pp. 538\u2013547. International Foundation for Autonomous Agents and Multiagent Systems, Auckland, New Zealand (2020-05-09\/2020-05-13). https:\/\/doi.org\/10.5555\/3398761.3398827.","DOI":"10.5555\/3398761.3398827"},{"key":"9675_CR8","doi-asserted-by":"publisher","unstructured":"Schmid, K., K\u00f6lle, M., & Matheis, T. Learning to Participate through Trading of Reward Shares. In: Proceedings of the 15th International Conference on Agents and Artificial Intelligence, vol. 1, pp. 355\u2013362. SCITEPRESS, Lisbon, Portugal (2023-02-22\/2023-02-24). https:\/\/doi.org\/10.5220\/0011781600003393.","DOI":"10.5220\/0011781600003393"},{"key":"9675_CR9","unstructured":"Deng, Y., Tang, P., & Zheng, S. Complexity and Algorithms of K-implementation. In: Proceedings of the 15th International Conference on Autonomous Agents and Multiagent Systems, Singapore (2016-05-09\/2016-05-13)."},{"key":"9675_CR10","doi-asserted-by":"publisher","unstructured":"Elias, J., Martignon, F., Avrachenkov, K., & Neglia, G. Socially-Aware Network Design Games. In: 2010 Proceedings IEEE INFOCOM, pp. 1\u20135. IEEE, San Diego, CA, USA (2010). https:\/\/doi.org\/10.1109\/INFCOM.2010.5462275.","DOI":"10.1109\/INFCOM.2010.5462275"},{"key":"9675_CR11","doi-asserted-by":"publisher","unstructured":"Chen, P.-A., & Kempe, D. (2008). Altruism, selfishness, and spite in traffic routing. In: Proceedings of the 9th ACM Conference on Electronic Commerce, pp. 140\u2013149. ACM, Chicago Il USA. https:\/\/doi.org\/10.1145\/1386790.1386816.","DOI":"10.1145\/1386790.1386816"},{"key":"9675_CR12","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/978-3-642-25510-6_33","volume-title":"Internet and Network Economics","author":"P-A Chen","year":"2011","unstructured":"Chen, P.-A., De Keijzer, B., Kempe, D., & Sch\u00e4fer, G. (2011). The Robust Price of Anarchy of Altruistic Games. In N. Chen, E. Elkind, & E. Koutsoupias (Eds.), Internet and Network Economics (Vol. 7090, pp. 383\u2013390). Springer. https:\/\/doi.org\/10.1007\/978-3-642-25510-6_33"},{"key":"9675_CR13","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1007\/978-3-642-15640-3_12","volume-title":"Trustworthly Global Computing","author":"I Caragiannis","year":"2010","unstructured":"Caragiannis, I., Kaklamanis, C., Kanellopoulos, P., Kyropoulou, M., & Papaioannou, E. (2010). The Impact of Altruism on the Efficiency of Atomic Congestion Games. In M. Wirsing, M. Hofmann, & A. Rauschmayer (Eds.), Trustworthly Global Computing (Vol. 6084, pp. 172\u2013188). Springer. https:\/\/doi.org\/10.1007\/978-3-642-15640-3_12"},{"issue":"2","key":"9675_CR14","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/j.cosrev.2009.04.003","volume":"3","author":"E Koutsoupias","year":"2009","unstructured":"Koutsoupias, E., & Papadimitriou, C. (2009). Worst-case equilibria. Computer Science Review, 3(2), 65\u201369. https:\/\/doi.org\/10.1016\/j.cosrev.2009.04.003","journal-title":"Computer Science Review"},{"key":"9675_CR15","doi-asserted-by":"publisher","unstructured":"Anshelevich, E., Dasgupta, A., Kleinberg, J., Tardos, E., Wexler, T., & Roughgarden, T. (2004). The Price of Stability for Network Design with Fair Cost Allocation. In: 45th Annual IEEE Symposium on Foundations Of Computer Science, pp. 295\u2013304. IEEE, Rome, Italy. https:\/\/doi.org\/10.1109\/FOCS.2004.68.","DOI":"10.1109\/FOCS.2004.68"},{"key":"9675_CR16","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2020.103357","volume":"288","author":"E Elkind","year":"2020","unstructured":"Elkind, E., Fanelli, A., & Flammini, M. (2020). Price of Pareto Optimality in hedonic games. Artificial Intelligence, 288, 103357. https:\/\/doi.org\/10.1016\/j.artint.2020.103357","journal-title":"Artificial Intelligence"},{"key":"9675_CR17","doi-asserted-by":"crossref","unstructured":"Axelrod, R. (1986). An Evolutionary Approach to Norms. The American Political Science Review 80, 18.","DOI":"10.1017\/S0003055400185016"},{"key":"9675_CR18","unstructured":"Mahmoud, S., Griffiths, N., Keppens, J., & Luck, M. (2010). An Analysis of Norm Emergence in Axelrod\u2019s Model. 8th European Workshop on Multi-Agent Systems, 15."},{"key":"9675_CR19","unstructured":"Montes, N., & Sierra, C. (2021). Value-Guided Synthesis of Parametric Normative Systems. Autonomous Agents and Multi-Agent Systems, 9"},{"key":"9675_CR20","unstructured":"Sierra, C., Osman, N., Noriega, P., & Sabater-Mir, J. (2021) . Value alignment: A formal approach. arXiv preprint arXiv:2110.09240, 15"},{"issue":"188","key":"9675_CR21","doi-asserted-by":"publisher","first-page":"20220036","DOI":"10.1098\/rsif.2022.0036","volume":"19","author":"TA Han","year":"2022","unstructured":"Han, T. A. (2022). Institutional incentives for the evolution of committed cooperation: Ensuring participation is as important as enhancing compliance. Journal of The Royal Society Interface, 19(188), 20220036. https:\/\/doi.org\/10.1098\/rsif.2022.0036","journal-title":"Journal of The Royal Society Interface"},{"issue":"4","key":"9675_CR22","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1016\/j.jtbi.2005.08.008","volume":"239","author":"H Ohtsuki","year":"2006","unstructured":"Ohtsuki, H., & Iwasa, Y. (2006). The leading eight: Social norms that can maintain cooperation by indirect reciprocity. Journal of Theoretical Biology, 239(4), 435\u2013444. https:\/\/doi.org\/10.1016\/j.jtbi.2005.08.008","journal-title":"Journal of Theoretical Biology"},{"key":"9675_CR23","unstructured":"Pereira, L.M., Lenaerts, T., & Martinez-Vaquero, L.A. (2017). Social Manifestation of Guilt Leads to Stable Cooperation in Multi-Agent Systems. In: Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems, S\u00e3o Paulo, Brazil."},{"issue":"1\u20133","key":"9675_CR24","doi-asserted-by":"publisher","first-page":"121","DOI":"10.3233\/FI-2018-1644","volume":"158","author":"E Lorini","year":"2018","unstructured":"Lorini, E., & M\u00fchlenbernd, R. (2018). The Long-Term Benefits of Following Fairness Norms under Dynamics of Learning and Evolution. Fundamenta Informaticae, 158(1\u20133), 121\u2013148. https:\/\/doi.org\/10.3233\/FI-2018-1644","journal-title":"Fundamenta Informaticae"},{"key":"9675_CR25","unstructured":"Jacq, A., Perolat, J., Geist, M., & Pietquin, O. Foolproof Cooperative Learning. In: Proceedings of The 12th Asian Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 129, pp. 401\u2013416. PMLR, Bangkok, Thailand (2020-11-18\/2020-11-20)"},{"issue":"1","key":"9675_CR26","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/002200278002400101","volume":"24","author":"R Axelrod","year":"1980","unstructured":"Axelrod, R. (1980). Effective Choice in the Prisoner\u2019s Dilemma. Journal of Conflict Resolution, 24(1), 3\u201325. https:\/\/doi.org\/10.1177\/002200278002400101","journal-title":"Journal of Conflict Resolution"},{"issue":"26","key":"9675_CR27","doi-asserted-by":"publisher","first-page":"10409","DOI":"10.1073\/pnas.1206569109","volume":"109","author":"WH Press","year":"2012","unstructured":"Press, W. H., & Dyson, F. J. (2012). Iterated Prisoner\u2019s Dilemma contains strategies that dominate any evolutionary opponent. Proceedings of the National Academy of Sciences, 109(26), 10409\u201310413. https:\/\/doi.org\/10.1073\/pnas.1206569109","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"17","key":"9675_CR28","doi-asserted-by":"publisher","first-page":"6913","DOI":"10.1073\/pnas.1214834110","volume":"110","author":"C Hilbe","year":"2013","unstructured":"Hilbe, C., Nowak, M. A., & Sigmund, K. (2013). Evolution of extortion in Iterated Prisoner\u2019s Dilemma games. Proceedings of the National Academy of Sciences, 110(17), 6913\u20136918. https:\/\/doi.org\/10.1073\/pnas.1214834110","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"38","key":"9675_CR29","doi-asserted-by":"publisher","first-page":"15348","DOI":"10.1073\/pnas.1306246110","volume":"110","author":"AJ Stewart","year":"2013","unstructured":"Stewart, A. J., & Plotkin, J. B. (2013). From extortion to generosity, evolution in the Iterated Prisoner\u2019s Dilemma. Proceedings of the National Academy of Sciences, 110(38), 15348\u201315353. https:\/\/doi.org\/10.1073\/pnas.1306246110","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"4","key":"9675_CR30","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1090\/S0273-0979-03-00988-1","volume":"40","author":"J Hofbauer","year":"2003","unstructured":"Hofbauer, J., & Sigmund, K. (2003). Evolutionary game dynamics. Bulletin of the American mathematical society, 40(4), 479\u2013519. https:\/\/doi.org\/10.1090\/S0273-0979-03-00988-1","journal-title":"Bulletin of the American mathematical society"},{"issue":"5659","key":"9675_CR31","doi-asserted-by":"publisher","first-page":"793","DOI":"10.1126\/science.1093411","volume":"303","author":"MA Nowak","year":"2004","unstructured":"Nowak, M. A., & Sigmund, K. (2004). Evolutionary Dynamics of Biological Games. Science, 303(5659), 793\u2013799. https:\/\/doi.org\/10.1126\/science.1093411","journal-title":"Science"},{"issue":"1","key":"9675_CR32","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1038\/ncomms1442","volume":"2","author":"DG Rand","year":"2011","unstructured":"Rand, D. G., & Nowak, M. A. (2011). The evolution of antisocial punishment in optional public goods games. Nature Communications, 2(1), 434. https:\/\/doi.org\/10.1038\/ncomms1442","journal-title":"Nature Communications"},{"key":"9675_CR33","doi-asserted-by":"crossref","unstructured":"Leyton-Brown, K., & Shoham, Y. (2008). Essentials of Game Theory: A Concise, Multidisciplinary Introduction. Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 3. Morgan & Claypool, San Rafael, Calif.","DOI":"10.1007\/978-3-031-01545-8"},{"key":"9675_CR34","unstructured":"Peysakhovich, A., & Lerer, A. (2017). Prosocial Learning Agents Solve Generalized Stag Hunts Better than Selfish Ones. arXiv."},{"key":"9675_CR35","doi-asserted-by":"publisher","unstructured":"McKee, K.R., Gemp, I., McWilliams, B., Du\u00e9\u00f1ez-Guzm\u00e1n, E.A., Hughes, E., & Leibo, J.Z. (2020). Social Diversity and Social Preferences in Mixed-Motive Reinforcement Learning. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, pp. 869\u2013877. International Foundation for Autonomous Agents and Multiagent Systems, Auckland, New Zealand. https:\/\/doi.org\/10.5555\/3398761.3398863.","DOI":"10.5555\/3398761.3398863"},{"key":"9675_CR36","doi-asserted-by":"publisher","unstructured":"Haeri, H. Reward-Sharing Relational Networks in Multi-Agent Reinforcement Learning as a Framework for Emergent Behavior. In: Proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems, pp. 1808\u20131810. International Foundation for Autonomous Agents and Multiagent Systems, Online (2021-05-03\/2021-05-07). https:\/\/doi.org\/10.5555\/3463952.3464246.","DOI":"10.5555\/3463952.3464246"},{"key":"9675_CR37","unstructured":"Wang, J.X., Hughes, E., Fernando, C., Czarnecki, W.M., Du\u00e9\u00f1ez-Guzm\u00e1n, E.A., & Leibo, J.Z. (2019) Evolving Intrinsic Motivations for Altruistic Behavior. In: Proceedings of the 18th International Conference on Autonomous Agents and Multiagent Systems, pp. 683\u2013692. International Foundation for Autonomous Agents and Multiagent Systems, Montreal QC, Canada."},{"issue":"3","key":"9675_CR38","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1007\/s10458-016-9338-4","volume":"31","author":"TA Han","year":"2017","unstructured":"Han, T. A., Pereira, L. M., & Lenaerts, T. (2017). Evolution of commitment and level of participation in public goods games. Autonomous Agents and Multi-Agent Systems, 31(3), 561\u2013583. https:\/\/doi.org\/10.1007\/s10458-016-9338-4","journal-title":"Autonomous Agents and Multi-Agent Systems"},{"issue":"3","key":"9675_CR39","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1177\/1059712321993166","volume":"30","author":"NB Ogbo","year":"2022","unstructured":"Ogbo, N. B., Elragig, A., & Han, T. A. (2022). Evolution of coordination in pairwise and multi-player interactions via prior commitments. Adaptive Behavior, 30(3), 257\u2013277. https:\/\/doi.org\/10.1177\/1059712321993166","journal-title":"Adaptive Behavior"},{"key":"9675_CR40","unstructured":"Christoffersen, P.J.K., Haupt, A.A., & Hadfield-Menell, D. (2022). Get It in Writing: Formal Contracts Mitigate Social Dilemmas in Multi-Agent RL. arXiv."},{"key":"9675_CR41","unstructured":"Sodomka, E., Hilliard, E.M., Littman, M.L., & Greenwald, A. Coco-Q: Learning in Stochastic Games with Side Payments. In: Proceedings of the 30th International Conference on Machine Learning. 3, vol. 28, pp. 1471\u20131479. JMLR.org, Atlanta, Georgia, USA (2013-06-17\/2013-06-19)"},{"key":"9675_CR42","doi-asserted-by":"publisher","unstructured":"Deng, Y., & Conitzer, V. Disarmament Games. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pp. 473\u2013479. AAAI Press, San Francisco, California USA (2017-02-04\/2017-02-09). https:\/\/doi.org\/10.1609\/aaai.v31i1.10573","DOI":"10.1609\/aaai.v31i1.10573"},{"issue":"1","key":"9675_CR43","doi-asserted-by":"publisher","first-page":"77","DOI":"10.2307\/2555629","volume":"17","author":"RA Lambert","year":"1986","unstructured":"Lambert, R. A. (1986). Executive Effort and Selection of Risky Projects. The RAND Journal of Economics, 17(1), 77. https:\/\/doi.org\/10.2307\/2555629","journal-title":"The RAND Journal of Economics"},{"key":"9675_CR44","doi-asserted-by":"publisher","unstructured":"Demski, J. S., & Sappington, D. E. M. (1987). Delegated Expertise. Journal of Accounting Research, 25(1), 68. https:\/\/doi.org\/10.2307\/2491259.","DOI":"10.2307\/2491259"},{"key":"9675_CR45","doi-asserted-by":"publisher","DOI":"10.2202\/1935-1704.1528","author":"JM Malcomson","year":"2009","unstructured":"Malcomson, J. M. (2009). Principal and Expert Agent. The B.E. Journal of Theoretical Economics. https:\/\/doi.org\/10.2202\/1935-1704.1528","journal-title":"The B.E. Journal of Theoretical Economics"},{"key":"9675_CR46","doi-asserted-by":"crossref","unstructured":"Lupu, A., & Precup, D. (2020). Gifting in Multi-Agent Reinforcement Learning. In: New Zealand, p. 9.","DOI":"10.1609\/aaai.v34i10.7208"},{"key":"9675_CR47","doi-asserted-by":"publisher","unstructured":"Wang, W.Z., Beliaev, M., B\u0131y\u0131k, E., Lazar, D.A., Pedarsani, R., & Sadigh, D. (2021). Emergent Prosociality in Multi-Agent Games Through Gifting. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, pp. 434\u2013442. ijcai.org, Montreal, . https:\/\/doi.org\/10.24963\/ijcai.2021\/61.","DOI":"10.24963\/ijcai.2021\/61"},{"key":"9675_CR48","unstructured":"Yang, J., Li, A., Farajtabar, M., Sunehag, P., Hughes, E., & Zha, H. (2020). Learning to Incentivize Other Learning Agents. In: Proceedings of the 34th Conference on Neural Information Processing Systems, vol. 33, pp. 15208\u201315219. Curran Associates, Inc., Vancouver, Canada."},{"key":"9675_CR49","unstructured":"Baker, B. Emergent Reciprocity and Team Formation from Randomized Uncertain Social Preferences. In: Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, virtual (2020-12-06\/2020-12-12)."},{"key":"9675_CR50","unstructured":"Yi, Y., Li, G., Wang, Y., & Lu, Z. (2022). Learning to Share in Multi-Agent Reinforcement Learning. In: Proceedings of the 36th Conference on Neural Information Processing Systems"},{"key":"9675_CR51","doi-asserted-by":"publisher","unstructured":"Gemp, I., McKee, K.R., Everett, R., Du\u00e9\u00f1ez-Guzm\u00e1n, E.A., Bachrach, Y., Balduzzi, D., & Tacchetti, A. (2022). D3C: Reducing the Price of Anarchy in Multi-Agent Learning. In: Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pp. 498\u2013506. International Foundation for Autonomous Agents and Multiagent Systems, Online. https:\/\/doi.org\/10.5555\/3535850.3535907","DOI":"10.5555\/3535850.3535907"},{"issue":"2","key":"9675_CR52","doi-asserted-by":"publisher","first-page":"117","DOI":"10.3390\/info14020117","volume":"14","author":"H Taherdoost","year":"2023","unstructured":"Taherdoost, H. (2023). Smart Contracts in Blockchain Technology: A Critical Review. Information, 14(2), 117. https:\/\/doi.org\/10.3390\/info14020117","journal-title":"Information"},{"key":"9675_CR53","doi-asserted-by":"publisher","unstructured":"Conitzer, V., & Oesterheld, C. Foundations of Cooperative AI. In: Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence. AAAI Press, Washington, DC, USA (2023-02-07\/2023-02-14). https:\/\/doi.org\/10.1609\/AAAI.V37I13.26791.","DOI":"10.1609\/AAAI.V37I13.26791"},{"issue":"2","key":"9675_CR54","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1016\/j.geb.2004.02.002","volume":"49","author":"M Tennenholtz","year":"2004","unstructured":"Tennenholtz, M. (2004). Program equilibrium. Games and Economic Behavior, 49(2), 363\u2013373. https:\/\/doi.org\/10.1016\/j.geb.2004.02.002","journal-title":"Games and Economic Behavior"},{"issue":"1","key":"9675_CR55","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1007\/s11238-018-9679-3","volume":"86","author":"C Oesterheld","year":"2019","unstructured":"Oesterheld, C. (2019). Robust program equilibrium. Theory and Decision, 86(1), 143\u2013159. https:\/\/doi.org\/10.1007\/s11238-018-9679-3","journal-title":"Theory and Decision"},{"key":"9675_CR56","doi-asserted-by":"publisher","unstructured":"Kovarik, V., Oesterheld, C., & Conitzer, V. Game Theory with Simulation of Other Players. In: IJCAI 2023. ijcai.org, Macao, SAR, China (2023-08-19\/2023-08-25). https:\/\/doi.org\/10.24963\/IJCAI.2023\/312.","DOI":"10.24963\/IJCAI.2023\/312"},{"key":"9675_CR57","doi-asserted-by":"publisher","first-page":"7229","DOI":"10.1073\/pnas.092080099","volume":"99","author":"MW Macy","year":"2002","unstructured":"Macy, M. W., & Flache, A. (2002). Learning dynamics in social dilemmas. Proceedings of the National Academy of Sciences of the National Academy of Sciences, 99, 7229\u20137236.","journal-title":"Proceedings of the National Academy of Sciences of the National Academy of Sciences"},{"key":"9675_CR58","unstructured":"Leibo, J.Z., Zambaldi, V., & Lanctot, M. (2017). Multi-agent Reinforcement Learning in Sequential Social Dilemmas. In: Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems, pp. 464\u2013473. ACM, S\u00e3o Paulo, Brazil."},{"key":"9675_CR59","unstructured":"Rawls, J. (1971). A Theory of Justice. The Belknap Press of Harvard University Press"},{"issue":"3","key":"9675_CR60","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1177\/002200277301700302","volume":"17","author":"TC Schelling","year":"1973","unstructured":"Schelling, T. C. (1973). Hockey Helmets, Concealed Weapons, and Daylight Saving: A Study of Binary Choices With Externalities. Journal of Conflict Resolution, 17(3), 381\u2013428. https:\/\/doi.org\/10.1177\/002200277301700302","journal-title":"Journal of Conflict Resolution"},{"issue":"3","key":"9675_CR61","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1287\/moor.1.3.273","volume":"1","author":"H Moulin","year":"1976","unstructured":"Moulin, H. (1976). Cooperation in Mixed Equilibrium. Mathematics of Operations Research, 1(3), 273\u2013286. https:\/\/doi.org\/10.1287\/moor.1.3.273","journal-title":"Mathematics of Operations Research"},{"issue":"1\u20132","key":"9675_CR62","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1016\/S0377-0427(00)00433-7","volume":"124","author":"FA Potra","year":"2000","unstructured":"Potra, F. A., & Wright, S. J. (2000). Interior-point methods. Journal of Computational and Applied Mathematics, 124(1\u20132), 281\u2013302. https:\/\/doi.org\/10.1016\/S0377-0427(00)00433-7","journal-title":"Journal of Computational and Applied Mathematics"},{"key":"9675_CR63","doi-asserted-by":"publisher","unstructured":"van\u00a0den Brand, J. A Deterministic Linear Program Solver in Current Matrix Multiplication Time. In: Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 259\u2013278. Society for Industrial and Applied Mathematics, Salt Lake City, UT, USA (2020-01-05\/2020-01-08). https:\/\/doi.org\/10.1137\/1.9781611975994.16","DOI":"10.1137\/1.9781611975994.16"},{"key":"9675_CR64","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1017\/CCOL0521340446.006","volume-title":"Advances in Economic Theory","author":"N Megiddo","year":"1987","unstructured":"Megiddo, N. (1987). On the complexity of linear programming. In T. F. Bewley (Ed.), Advances in Economic Theory (1st ed., pp. 225\u2013268). Cambridge University Press. https:\/\/doi.org\/10.1017\/CCOL0521340446.006","edition":"1"},{"issue":"1","key":"9675_CR65","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1146\/annurev.soc.24.1.183","volume":"24","author":"P Kollock","year":"1998","unstructured":"Kollock, P. (1998). Social Dilemmas: The Anatomy of Cooperation. Annual Review of Sociology, 24(1), 183\u2013214. https:\/\/doi.org\/10.1146\/annurev.soc.24.1.183","journal-title":"Annual Review of Sociology"},{"key":"9675_CR66","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1017\/CBO9780511528446.003","volume-title":"The Shapley Value","author":"LS Shapley","year":"1988","unstructured":"Shapley, L. S. (1988). A value for n -person games. In A. E. Roth (Ed.), The Shapley Value (1st ed., pp. 31\u201340). Cambridge University Press. https:\/\/doi.org\/10.1017\/CBO9780511528446.003","edition":"1"},{"key":"9675_CR67","doi-asserted-by":"publisher","first-page":"432","DOI":"10.1007\/978-3-642-17572-5_36","volume-title":"Internet and Network Economics","author":"Y Bachrach","year":"2010","unstructured":"Bachrach, Y., Polukarov, M., & Jennings, N. R. (2010). The Good, The Bad and The Cautious: Safety Level Cooperative Games. In A. Saberi (Ed.), Internet and Network Economics (Vol. 6484, pp. 432\u2013443). Springer. https:\/\/doi.org\/10.1007\/978-3-642-17572-5_36"},{"key":"9675_CR68","doi-asserted-by":"publisher","unstructured":"Harsanyi, J.C. (2004). Games with Incomplete Information Played by \u201cBayesian\u201d Players, I\u2013III: Part I. The Basic Model. Management Science 50(12_supplement), 1804\u20131817. https:\/\/doi.org\/10.1287\/mnsc.1040.0270.","DOI":"10.1287\/mnsc.1040.0270"},{"key":"9675_CR69","doi-asserted-by":"publisher","DOI":"10.7249\/R366","volume-title":"Linear Programming and Extensions","author":"G Dantzig","year":"1963","unstructured":"Dantzig, G. (1963). Linear Programming and Extensions. Princeton University Press."},{"key":"9675_CR70","unstructured":"K\u00f6ster, R., McKee, K.R., Everett, R., Weidinger, L., Isaac, W.S., Hughes, E., Du\u00e9\u00f1ez-Guzm\u00e1n, E.A., Graepel, T., Botvinick, M. & Leibo, J.Z. (2020). Model-Free Conventions in Multi-Agent Reinforcement Learning with Heterogeneous Preferences. arXiv."},{"key":"9675_CR71","unstructured":"Willis, R., & Luck, M. Resolving social dilemmas through reward transfer commitments. In: Proceedings of the Adaptive and Learning Agents Workshop, London (2023-05-09\/2023-05-10)"},{"key":"9675_CR72","unstructured":"Dafoe, A., Hughes, E., Bachrach, Y., Collins, T., McKee, K.R., Leibo, J.Z., Larson, K., & Graepel, T. (2020). Open Problems in Cooperative AI"}],"container-title":["Autonomous Agents and Multi-Agent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-024-09675-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10458-024-09675-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-024-09675-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,13]],"date-time":"2024-11-13T15:25:20Z","timestamp":1731511520000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10458-024-09675-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,12]]},"references-count":72,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["9675"],"URL":"https:\/\/doi.org\/10.1007\/s10458-024-09675-4","relation":{},"ISSN":["1387-2532","1573-7454"],"issn-type":[{"type":"print","value":"1387-2532"},{"type":"electronic","value":"1573-7454"}],"subject":[],"published":{"date-parts":[[2024,10,12]]},"assertion":[{"value":"2 September 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 October 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose beyond the grant information detailed on the title page.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"49"}}