{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T03:51:17Z","timestamp":1760586677354,"version":"3.41.0"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2013,9,1]],"date-time":"2013-09-01T00:00:00Z","timestamp":1377993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2013,9]]},"abstract":"<jats:p>\n            In multiagent systems, social optimality is a desirable goal to achieve in terms of maximizing the global efficiency of the system. We study the problem of coordinating on socially optimal outcomes among a population of agents, in which each agent randomly interacts with another agent from the population each round. Previous work [Hales and Edmonds 2003; Matlock and Sen 2007, 2009] mainly resorts to modifying the interaction protocol from random interaction to tag-based interactions and only focus on the case of symmetric games. Besides, in previous work the agents\u2019 decision making processes are usually based on evolutionary learning, which usually results in high communication cost and high deviation on the coordination rate. To solve these problems, we propose an alternative social learning framework with two major contributions as follows. First, we introduce the\n            <jats:italic>observation<\/jats:italic>\n            mechanism to reduce the amount of communication required among agents. Second, we propose that the agents\u2019 learning strategies should be based on reinforcement learning technique instead of evolutionary learning. Each agent explicitly keeps the record of its current state in its learning strategy, and learn its optimal policy for each state independently. In this way, the learning performance is much more stable and also it is suitable for both symmetric and asymmetric games. The performance of this social learning framework is extensively evaluated under the testbed of two-player general-sum games comparing with previous work [Hao and Leung 2011; Matlock and Sen 2007]. The influences of different factors on the learning performance of the social learning framework are investigated as well.\n          <\/jats:p>","DOI":"10.1145\/2517329","type":"journal-article","created":{"date-parts":[[2013,10,1]],"date-time":"2013-10-01T18:14:28Z","timestamp":1380651268000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Achieving Socially Optimal Outcomes in Multiagent Systems with Reinforcement Social Learning"],"prefix":"10.1145","volume":"8","author":[{"given":"Jianye","family":"Hao","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ho-Fung","family":"Leung","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2013,9]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Allison P. D. 1992. The cultural evolution of beneficent norms. Social Forces. 279--301.  Allison P. D. 1992. The cultural evolution of beneficent norms. Social Forces . 279--301.","DOI":"10.2307\/2580012"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(02)00121-2"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2004.04.013"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87654-0_14"},{"volume-title":"Proceedings of AAAI\u201998","author":"Claus C.","key":"e_1_2_1_5_1"},{"volume-title":"Proceedings of ICML\u201906","author":"Conitzer V.","key":"e_1_2_1_6_1"},{"volume-title":"Proceedings of the AAAI Workshop on Multiagent Learning.","author":"Crandall J. W.","key":"e_1_2_1_7_1"},{"volume-title":"Correlated Q-Learning. In Proceedings of ICML\u201903","author":"Greenwald A.","key":"e_1_2_1_8_1"},{"volume-title":"Multi-Agent-Based Simulation","series-title":"Lecture Notes in Artificial Intelligence.","author":"Hales D.","key":"e_1_2_1_9_1"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/860575.860655"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2010.75"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2011.31"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-32695-0_10"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/11691839_1"},{"volume-title":"Proceedings of the AAAI Fall Symposium on Socially Intelligent Agents. 61--63","author":"Hogg L. M.","key":"e_1_2_1_15_1"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/3468.952713"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/6677"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2005.1554801"},{"volume-title":"Proceedings of the ICML\u201998","author":"Hu J.","key":"e_1_2_1_19_1"},{"volume-title":"Proceedings of the AAAI\u201902","author":"Kapetanakis S.","key":"e_1_2_1_20_1"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-55860-335-6.50027-1"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/11691839_9"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1329125.1329429"},{"volume-title":"Proceedings of AAMAS\u201909","author":"Matlock M.","key":"e_1_2_1_24_1"},{"key":"e_1_2_1_25_1","unstructured":"Maynard J. S. 1982. Evolution and The Theory of Games. Cambridge University Press Cambridge UK.  Maynard J. S. 1982. Evolution and The Theory of Games . Cambridge University Press Cambridge UK."},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Nowak M. and Sigmund K. 1993. A strategy of winstay lose-shit that outperforms tit-for-tat in the prisoner\u2019s dilemma game. Nature 56--58.  Nowak M. and Sigmund K. 1993. A strategy of winstay lose-shit that outperforms tit-for-tat in the prisoner\u2019s dilemma game. Nature 56--58.","DOI":"10.1038\/364056a0"},{"key":"e_1_2_1_27_1","unstructured":"Osborne M. J. and Rubinstein A. 1994. A Course in Game Theory. MIT Press Cambridge MA.  Osborne M. J. and Rubinstein A. 1994. A Course in Game Theory . MIT Press Cambridge MA."},{"volume-title":"Proceedings of AAMAS. 387--434","author":"Panait L.","key":"e_1_2_1_28_1"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2012.31"},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Riol R. and Cohen M. D. 2001. Cooperation withour reciprocity. Nature 441--443.  Riol R. and Cohen M. D. 2001. Cooperation withour reciprocity. Nature 441--443.","DOI":"10.1038\/35106555"},{"volume-title":"Proceedings of IJCAI\u201907","author":"Sen S.","key":"e_1_2_1_31_1"},{"volume-title":"Proceedings of AAMAS\u201906","author":"Verbeeck K.","key":"e_1_2_1_32_1"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.5555\/2283396.2283465"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s101640170011"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2517329","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2517329","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:19:09Z","timestamp":1750234749000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2517329"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,9]]},"references-count":35,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2013,9]]}},"alternative-id":["10.1145\/2517329"],"URL":"https:\/\/doi.org\/10.1145\/2517329","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"type":"print","value":"1556-4665"},{"type":"electronic","value":"1556-4703"}],"subject":[],"published":{"date-parts":[[2013,9]]},"assertion":[{"value":"2013-01-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2013-09-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}