{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T20:53:04Z","timestamp":1773694384121,"version":"3.50.1"},"reference-count":81,"publisher":"Association for Computing Machinery (ACM)","issue":"1-2","license":[{"start":{"date-parts":[[2022,6,30]],"date-time":"2022-06-30T00:00:00Z","timestamp":1656547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Science of Security Lablet at North Carolina State University"},{"name":"NSF","award":["IIS-2116751"],"award-info":[{"award-number":["IIS-2116751"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Auton. Adapt. Syst."],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>Multi-agent systems provide a basis for developing systems of autonomous entities and thus find application in a variety of domains. We consider a setting where not only the member agents are adaptive but also the multi-agent system viewed as an entity in its own right is adaptive. Specifically, the social structure of a multi-agent system can be reflected in the social norms among its members. It is well recognized that the norms that arise in society are not always beneficial to its members. We focus on prosocial norms, which help achieve positive outcomes for society and often provide guidance to agents to act in a manner that takes into account the welfare of others.<\/jats:p>\n          <jats:p>Specifically, we propose Cha, a framework for the emergence of prosocial norms. Unlike previous norm emergence approaches, Cha supports continual change to a system (agents may enter and leave) and dynamism (norms may change when the environment changes). Importantly, Cha agents incorporate prosocial decision-making based on inequity aversion theory, reflecting an intuition of guilt arising from being antisocial. In this manner, Cha brings together two important themes in prosociality: decision-making by individuals and fairness of system-level outcomes. We demonstrate via simulation that Cha can improve aggregate societal gains and fairness of outcomes.<\/jats:p>","DOI":"10.1145\/3540202","type":"journal-article","created":{"date-parts":[[2022,6,6]],"date-time":"2022-06-06T09:58:19Z","timestamp":1654509499000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Prosocial Norm Emergence in Multi-agent Systems"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7986-7955","authenticated-orcid":false,"given":"Mehdi","family":"Mashayekhi","sequence":"first","affiliation":[{"name":"North Carolina State University, Raleigh"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3627-097X","authenticated-orcid":false,"given":"Nirav","family":"Ajmeri","sequence":"additional","affiliation":[{"name":"University of Bristol, Bristol, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7044-1042","authenticated-orcid":false,"given":"George F.","family":"List","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3599-3893","authenticated-orcid":false,"given":"Munindar P.","family":"Singh","sequence":"additional","affiliation":[{"name":"North Carolina State University, Raleigh"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,9,7]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1093\/oso\/9780190643027.001.0001"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2022\/2"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-013-9237-x"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/4"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/3398761.3398769"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.simpat.2018.03.006"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2882967"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1093\/logcom\/exx012"},{"key":"e_1_3_2_10_2","first-page":"135","volume-title":"Proceedings of the Normative Multi-Agent Systems","author":"Andrighetto Giulia","year":"2013","unstructured":"Giulia Andrighetto, Cristiano Castelfranchi, Eunate Mayor, John McBreen, Maite L\u00f3pez-S\u00e1nchez, and Simon Parsons. 2013. Social norm dynamics. In Proceedings of the Normative Multi-Agent Systems. Schloss Dagstuhl, Dagstuhl, Germany, 135\u2013170."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901744"},{"issue":"88","key":"e_1_3_2_12_2","first-page":"2635","article-title":"Multi-agent reinforcement learning in common interest and fixed sum stochastic games: An experimental study","volume":"9","author":"Bab Avraham","year":"2008","unstructured":"Avraham Bab and Ronen I. Brafman. 2008. Multi-agent reinforcement learning in common interest and fixed sum stochastic games: An experimental study. Journal of Machine Learning Research 9, 88 (2008), 2635\u20132675. Retrieved from http:\/\/jmlr.org\/papers\/v9\/bab08a.html.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2020.3018038"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9441"},{"key":"e_1_3_2_15_2","volume-title":"Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web","author":"Berners-Lee Tim","year":"1999","unstructured":"Tim Berners-Lee. 1999. Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web. Harper Business, New York."},{"key":"e_1_3_2_16_2","first-page":"255","volume-title":"Proceedings of the 9th International Conference on Principles of Knowledge Representation and Reasoning","author":"Boella Guido","year":"2004","unstructured":"Guido Boella and Leendert W. N. van der Torre. 2004. Regulative and constitutive norms in normative multiagent systems. In Proceedings of the 9th International Conference on Principles of Knowledge Representation and Reasoning. AAAI Press, Whistler, Canada, 255\u2013266."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3461702.3462515"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883018"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278740"},{"key":"e_1_3_2_20_2","volume-title":"Statistical Power Analysis for the Behavioral Sciences (2nd ed.)","author":"Cohen Jacob","year":"1988","unstructured":"Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Lawrence Erlbaum Associates, Hillsdale, New Jersey."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.18434\/M32189"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.5555\/3306127.3331881"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-020-09465-8"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315085241"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1002\/9780470665237"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2013.2255286"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11299-015-0183-y"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1162\/003355399556151"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038920"},{"key":"e_1_3_2_30_2","first-page":"211","volume-title":"Proceedings of the International Workshop on Coordination, Organizations, Institutions, and Norms in Agent Systems","author":"Frantz Christopher","year":"2013","unstructured":"Christopher Frantz, Martin K. Purvis, Mariusz Nowostawski, and Bastin Tony Savarimuthu. 2013. Modelling institutions using dynamic deontics. In Proceedings of the International Workshop on Coordination, Organizations, Institutions, and Norms in Agent Systems. Springer, St. Paul, 211\u2013233."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-008-9059-4"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203803233"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-49470-7"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3127498"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-009-8484-4"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-60960-100-3.ch304"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.18564\/jasss.1750"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/SASO.2011.26"},{"key":"e_1_3_2_39_2","first-page":"1123","volume-title":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","author":"Huang Xiaowei","year":"2016","unstructured":"Xiaowei Huang, Ji Ruan, Quingliang Chen, and Kaile Su. 2016. Normative multiagent systems: A dynamic generalization. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. IJCAI, New York, 1123\u20131129."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2016.89"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10706"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3365664"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-11558-0_46"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-25524-8_19"},{"key":"e_1_3_2_45_2","first-page":"373","volume-title":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","author":"Mashayekhi Mehdi","year":"2016","unstructured":"Mehdi Mashayekhi, Hongying Du, George F. List, and Munindar P. Singh. 2016. Silk: A simulation study of regulating open normative multiagent systems. In Proceedings of the 25th International Joint Conference on Artificial Intelligence. IJCAI, New York, 373\u2013379. Retrieved from https:\/\/www.ijcai.org\/Abstract\/16\/060."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-013-9240-2"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2517328"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2720024"},{"key":"e_1_3_2_49_2","first-page":"109","volume-title":"Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems","author":"Morales Javier","year":"2014","unstructured":"Javier Morales, Maite L\u00f3pez-S\u00e1nchez, Juan A. Rodr\u00edguez-Aguilar, Michael Wooldridge, and Wamberto Weber Vasconcelos. 2014. Minimality and simplicity in the on-line automated synthesis of normative systems. In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems. IFAAMAS, Paris, 109\u2013116."},{"key":"e_1_3_2_50_2","first-page":"433","volume-title":"Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems","author":"Morales Javier","year":"2015","unstructured":"Javier Morales, Maite L\u00f3pez-S\u00e1nchez, Juan A. Rodr\u00edguez-Aguilar, Michael Wooldridge, and Wamberto Weber Vasconcelos. 2015. Synthesising liberal normative systems. In Proceedings of the 14th International Conference on Autonomous Agents and Multiagent Systems. IFAAMAS, Istanbul, 433\u2013441."},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-018-9390-3"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-019-09422-0"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.5555\/3398761.3398958"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888916000023"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1186\/2194-3206-1-3"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12215"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.4159\/9780674042582"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33016146"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.5555\/2019196.2019199"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/1609991.1609992"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.18564\/jasss.1659"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14962-7_14"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.5555\/1625275.1625519"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.5555\/3237383.3237876"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.5555\/3237383.3237891"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jecp.2013.10.007"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-soc-073014-112242"},{"key":"e_1_3_2_68_2","first-page":"127","volume-title":"Proceedings of the Decentralized Artificial Intelligence, Volume 2","author":"Singh Munindar P.","year":"1991","unstructured":"Munindar P. Singh. 1991. Group ability and structure. In Proceedings of the Decentralized Artificial Intelligence, Volume 2\u2014Proceedings of the2nd European Workshop on Modeling Autonomous Agents in a Multi Agent World.Yves Demazeau and Jean-Pierre M\u00fcller (Eds.), Elsevier\/North-Holland, Amsterdam, 127\u2013145."},{"key":"e_1_3_2_69_2","first-page":"303","volume-title":"Proceedings of the 13th European Conference on Artificial Intelligence","author":"Singh Munindar P.","year":"1998","unstructured":"Munindar P. Singh. 1998. The intentions of teams: Team structure, endodeixis, and exodeixis. In Proceedings of the 13th European Conference on Artificial Intelligence. John Wiley, Brighton, United Kingdom, 303\u2013307."},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2542182.2542203"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.4230\/DFU.Vol4.12111.191"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1023\/A%3A1007678930559"},{"key":"e_1_3_2_73_2","first-page":"869","volume-title":"Proceedings of the 18th European Conference on Artificial Intelligence","author":"Staab Eugen","year":"2008","unstructured":"Eugen Staab, Volker Fusenig, and Thomas Engel. 2008. Trust-aided acquisition of unverifiable information. In Proceedings of the 18th European Conference on Artificial Intelligence. IOS Press, Patras, 869\u2013870."},{"key":"e_1_3_2_74_2","first-page":"1459","volume-title":"Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems","author":"Sugawara Toshiharu","year":"2014","unstructured":"Toshiharu Sugawara. 2014. Emergence of conventions in conflict situations in complex agent network environments. In Proceedings of the 13th International Conference on Autonomous Agents and Multiagent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Paris, 1459\u20131460."},{"key":"e_1_3_2_75_2","volume-title":"Reinforcement Learning: An Introduction (2nd ed.)","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction (2nd ed.). MIT Press, Cambridge, Massachusetts."},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.17226\/22097"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1177\/089443930101900305"},{"key":"e_1_3_2_78_2","first-page":"3","volume-title":"Proceedings of the Handbook of Normative Multiagent Systems","author":"Verhagen Harko","year":"2018","unstructured":"Harko Verhagen, Martin Neumann, and Munindar P. Singh. 2018. Normative multiagent systems: Foundations and history. In Proceedings of the Handbook of Normative Multiagent Systems, Amit Chopra, Leendert van der Torre, Harko Verhagen, and Serena Villata (Eds.). College Publications, London, Chapter 1, 3\u201325. Retrieved from http:\/\/www.collegepublications.co.uk\/downloads\/handbooks00004.pdf."},{"key":"e_1_3_2_79_2","first-page":"420","volume-title":"Proceedings of the 22nd International Joint Conference on Artificial Intelligence","author":"Villatoro Daniel","year":"2011","unstructured":"Daniel Villatoro, Jordi Sabater-Mir, and Sandip Sen. 2011. Social instruments for robust convention emergence. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. AAAI Press, Barcelona, Catalonia, Spain, 420\u2013425."},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2021.3045821"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.5555\/3535850.3536118"},{"key":"e_1_3_2_82_2","first-page":"475","volume-title":"Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems","author":"Yu Chao","year":"2013","unstructured":"Chao Yu, Minjie Zhang, Fenghui Ren, and Xudong Luo. 2013. Emergence of social norms through collective learning in networked agent societies. In Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems. IFAAMAS, Saint Paul, 475\u2013482."}],"container-title":["ACM Transactions on Autonomous and Adaptive Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3540202","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3540202","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3540202","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:01Z","timestamp":1750182661000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3540202"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,30]]},"references-count":81,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3540202"],"URL":"https:\/\/doi.org\/10.1145\/3540202","relation":{},"ISSN":["1556-4665","1556-4703"],"issn-type":[{"value":"1556-4665","type":"print"},{"value":"1556-4703","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,30]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-09-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}