{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T21:02:55Z","timestamp":1767992575388,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,1,27]],"date-time":"2019-01-27T00:00:00Z","timestamp":1548547200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ONR","award":["N00014-16-1-2278"],"award-info":[{"award-number":["N00014-16-1-2278"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,1,27]]},"DOI":"10.1145\/3306618.3314252","type":"proceedings-article","created":{"date-parts":[[2019,7,10]],"date-time":"2019-07-10T12:10:59Z","timestamp":1562760659000},"page":"21-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Requirements for an Artificial Agent with Norm Competence"],"prefix":"10.1145","author":[{"given":"Bertram F.","family":"Malle","sequence":"first","affiliation":[{"name":"Brown University, Providence, RI, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Bello","sequence":"additional","affiliation":[{"name":"U.S. Naval Research Laboratory, Washington, DC, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"Scheutz","sequence":"additional","affiliation":[{"name":"Tufts University, Medford, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,1,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.84.1.18"},{"key":"e_1_3_2_1_2_1","volume-title":"AAAI Workshop: AI, Ethics, and Society","volume":"02","author":"Abel D.","year":"2016","unstructured":"Abel , D. et al. 2016. Reinforcement learning as a framework for ethical decision making . AAAI Workshop: AI, Ethics, and Society , volume WS-16- 02 of 13th AAAI Workshops ( 2016 ). Abel, D. et al. 2016. Reinforcement learning as a framework for ethical decision making. AAAI Workshop: AI, Ethics, and Society, volume WS-16-02 of 13th AAAI Workshops (2016)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-006-0004-4"},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the IEEE.","author":"Anderson M.","year":"2018","unstructured":"Anderson , M. et al. 2018. A value-driven eldercare robot: Virtual and physical instantiations of a case-supported principle-based behavior paradigm . Proceedings of the IEEE. ( 2018 ), 1-- Anderson, M. et al. 2018. A value-driven eldercare robot: Virtual and physical instantiations of a case-supported principle-based behavior paradigm. Proceedings of the IEEE. (2018), 1--"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.2197\/ipsjjip.22.299"},{"key":"e_1_3_2_1_6_1","volume-title":"et al","author":"Arnold T.","year":"2017","unstructured":"Arnold , T. et al . 2017 . Value alignment or misalignment -- What will keep systems accountable? The Workshops of the Thirty-First AAAI Conference on Artificial Intelligence : Technical Reports, WS-17-02: AI, Ethics, and Society. The AAAI Press . 81--88. Arnold, T. et al. 2017. Value alignment or misalignment -- What will keep systems accountable? The Workshops of the Thirty-First AAAI Conference on Artificial Intelligence: Technical Reports, WS-17-02: AI, Ethics, and Society. The AAAI Press. 81--88."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the 40th Annual Meeting of the Cognitive Science Society","author":"Bello P.","year":"2018","unstructured":"Bello , P. et al. 2018. An attention-driven computational model of human causal reasoning . Proceedings of the 40th Annual Meeting of the Cognitive Science Society ( Austin, TX , 2018 ), 1353--1358. Bello, P. et al. 2018. An attention-driven computational model of human causal reasoning. Proceedings of the 40th Annual Meeting of the Cognitive Science Society (Austin, TX, 2018), 1353--1358."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1086\/321298"},{"key":"e_1_3_2_1_9_1","volume-title":"The grammar of society: The nature and dynamics of social norms","author":"Bicchieri C.","unstructured":"Bicchieri , C. 2006. The grammar of society: The nature and dynamics of social norms . Cambridge University Press . Bicchieri, C. 2006. The grammar of society: The nature and dynamics of social norms. Cambridge University Press."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1002\/ejsp.640","article-title":"Descriptive norms, prescriptive norms, and social control: An intercultural comparison of people's reactions to uncivil behaviors","volume":"40","author":"Brauer M.","year":"2010","unstructured":"Brauer , M. and Chaurand , N. 2010 . Descriptive norms, prescriptive norms, and social control: An intercultural comparison of people's reactions to uncivil behaviors . European Journal of Social Psychology. 40 , 3 (Apr. 2010), 490--499. Brauer, M. and Chaurand, N. 2010. Descriptive norms, prescriptive norms, and social control: An intercultural comparison of people's reactions to uncivil behaviors. European Journal of Social Psychology. 40, 3 (Apr. 2010), 490--499.","journal-title":"European Journal of Social Psychology."},{"key":"e_1_3_2_1_11_1","volume-title":"et al","author":"Brennan G.","year":"2013","unstructured":"Brennan , G. et al . 2013 . Explaining norms. Oxford University Press . Brennan, G. et al. 2013. Explaining norms. Oxford University Press."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2006.82"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1093\/analys\/24.2.33"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.58.6.1015"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-017-9372-x"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1086\/223933"},{"key":"e_1_3_2_1_17_1","volume-title":"-D","author":"Hechter M.","year":"2001","unstructured":"Hechter , M. and Opp , K . -D . 2001 . Social Norms. Russell Sage Foundation . Hechter, M. and Opp, K.-D. 2001. Social Norms. Russell Sage Foundation."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.evolhumbehav.2009.03.005"},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of the 56th IEEE Conference on Decision and Control (CDC","author":"Kasenberg D.","year":"2017","unstructured":"Kasenberg , D. and Scheutz , M . 2017. Interpretable apprenticeship learning with temporal logic specifications . Proceedings of the 56th IEEE Conference on Decision and Control (CDC 2017 ). IEEE Press. 4914--4921. Kasenberg, D. and Scheutz, M. 2017. Interpretable apprenticeship learning with temporal logic specifications. Proceedings of the 56th IEEE Conference on Decision and Control (CDC 2017). IEEE Press. 4914--4921."},{"key":"e_1_3_2_1_20_1","volume-title":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence","author":"Kasenberg D.","year":"2018","unstructured":"Kasenberg , D. and Scheutz , M . 2018. Norm conflict resolution in stochastic domains . Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence ( 2018 ). Kasenberg, D. and Scheutz, M. 2018. Norm conflict resolution in stochastic domains. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (2018)."},{"key":"e_1_3_2_1_21_1","volume-title":"et al","author":"Kim J.","year":"2018","unstructured":"Kim , J. et al . 2018 . Not-So-CLEVR: learning same--different relations strains feed-forward neural networks. Interface Focus . 8, 4 (2018). Kim, J. et al. 2018. Not-So-CLEVR: learning same--different relations strains feed-forward neural networks. Interface Focus. 8, 4 (2018)."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87654-0_2"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-42691-4_12"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1353\/sor.2018.0002","article-title":"Changing social norms","volume":"85","author":"Mack A.","year":"2018","unstructured":"Mack , A. ed. 2018 . Changing social norms . Social Research: An International Quarterly. 85 , 1 (2018), 1 -- 271 . Mack, A. ed. 2018. Changing social norms. Social Research: An International Quarterly. 85, 1 (2018), 1--271.","journal-title":"Social Research: An International Quarterly."},{"key":"e_1_3_2_1_25_1","volume-title":"et al","author":"Mahmoud M.A.","year":"2014","unstructured":"Mahmoud , M.A. et al . 2014 . A review of norms and normative multiagent systems. The Scientific World Journal . 2014, (2014), 1-- Mahmoud, M.A. et al. 2014. A review of norms and normative multiagent systems. The Scientific World Journal. 2014, (2014), 1--"},{"key":"e_1_3_2_1_26_1","unstructured":"Malle B.F. 2018. From binary deontics to deontic continua: The nature of human (and robot) norm systems. Malle B.F. 2018. From binary deontics to deontic continua: The nature of human (and robot) norm systems."},{"key":"e_1_3_2_1_27_1","volume-title":"et al","author":"Malle B.F.","year":"2017","unstructured":"Malle , B.F. et al . 2017 . Networks of social and moral norms in human and robot agents. A World with Robots : International Conference on Robot Ethics : ICRE 2015. M.I. Aldinhas Ferreira et al., eds. Springer International Publishing . 3--17. Malle, B.F. et al. 2017. Networks of social and moral norms in human and robot agents. A World with Robots: International Conference on Robot Ethics: ICRE 2015. M.I. Aldinhas Ferreira et al., eds. Springer International Publishing. 3--17."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1037\/h0028070"},{"key":"e_1_3_2_1_29_1","volume-title":"Normative Multi-agent Systems: Dagstuhl Seminar Proceedings","author":"Nickles M.","year":"2007","unstructured":"Nickles , M. 2007 . Towards a logic of graded normativity and norm adherence . Normative Multi-agent Systems: Dagstuhl Seminar Proceedings ( Dagstuhl, Germany , 2007). Nickles, M. 2007. Towards a logic of graded normativity and norm adherence. Normative Multi-agent Systems: Dagstuhl Seminar Proceedings (Dagstuhl, Germany, 2007)."},{"key":"e_1_3_2_1_30_1","volume-title":"2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE Press. 243--250","author":"Nyga D.","unstructured":"Nyga , D. and Beetz , M . 2012. Everything robots always wanted to know about housework (but were afraid to ask) . 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE Press. 243--250 . Nyga, D. and Beetz, M. 2012. Everything robots always wanted to know about housework (but were afraid to ask). 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE Press. 243--250."},{"key":"e_1_3_2_1_31_1","volume-title":"The social system","author":"Parsons T.","unstructured":"Parsons , T. 1951. The social system . Free Press . Parsons, T. 1951. The social system. Free Press."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Pereira L.M. and Saptawijaya A. 2007. Modelling morality with prospective logic. Progress in Artificial Intelligence. J. Neves et al. eds. Springer Berlin Heidelberg. 99--111. Pereira L.M. and Saptawijaya A. 2007. Modelling morality with prospective logic. Progress in Artificial Intelligence. J. Neves et al. eds. Springer Berlin Heidelberg. 99--111.","DOI":"10.1007\/978-3-540-77002-2_9"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Prentice D.A. and Miller D.T. 1996. Pluralistic ignorance and the perpetuation of social norms by unwitting actors. Advances in Experimental Social Psychology. M.P. Zanna ed. Academic Press. 161--209. Prentice D.A. and Miller D.T. 1996. Pluralistic ignorance and the perpetuation of social norms by unwitting actors. Advances in Experimental Social Psychology. M.P. Zanna ed. Academic Press. 161--209.","DOI":"10.1016\/S0065-2601(08)60238-5"},{"key":"e_1_3_2_1_34_1","first-page":"3","article-title":"The sources of normativity: Young children's awareness of the normative structure of games","volume":"44","author":"Rakoczy H.","year":"2008","unstructured":"Rakoczy , H. 2008 . The sources of normativity: Young children's awareness of the normative structure of games . Developmental Psychology. 44 , 3 (May 2008), 875--88 Rakoczy, H. et al. 2008. The sources of normativity: Young children's awareness of the normative structure of games. Developmental Psychology. 44, 3 (May 2008), 875--88","journal-title":"Developmental Psychology."},{"key":"e_1_3_2_1_35_1","volume-title":"et al","author":"Russell S.","year":"2016","unstructured":"Russell , S. et al . 2016 . Research priorities for robust and beneficial artificial intelligence. arXiv preprint arXiv:1602.03506. (2016). Russell, S. et al. 2016. Research priorities for robust and beneficial artificial intelligence. arXiv preprint arXiv:1602.03506. (2016)."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-012-9126-7"},{"key":"e_1_3_2_1_37_1","volume-title":"The strategy of conflict","author":"Schelling T.C.","unstructured":"Schelling , T.C. 1960. The strategy of conflict . Harvard University Press . Schelling, T.C. 1960. The strategy of conflict. Harvard University Press."},{"key":"e_1_3_2_1_38_1","volume-title":"Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems. 1378--1386","author":"Scheutz M.","unstructured":"Scheutz , M. et al. 2017. Spoken instruction-based one-shot object and action learning in a cognitive robotic architecture . Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems. 1378--1386 . Scheutz, M. et al. 2017. Spoken instruction-based one-shot object and action learning in a cognitive robotic architecture. Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems. 1378--1386."},{"key":"e_1_3_2_1_39_1","volume-title":"Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018)","author":"Serramia M.","year":"2018","unstructured":"Serramia , M. et al. 2018. Moral values in norm decision making . Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018) (Richland, SC, 2018 ), 1294--1302. Serramia, M. et al. 2018. Moral values in norm decision making. Proceedings of the 17th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2018) (Richland, SC, 2018), 1294--1302."},{"key":"e_1_3_2_1_40_1","volume-title":"Engineering Applications of Artificial Intelligence. 65","author":"Shams Z.","year":"2017","unstructured":"Shams , Z. et al. 2017. Practical reasoning with norms for autonomous software agents . Engineering Applications of Artificial Intelligence. 65 , ( Oct. 2017 ), 388--39 Shams, Z. et al. 2017. Practical reasoning with norms for autonomous software agents. Engineering Applications of Artificial Intelligence. 65, (Oct. 2017), 388--39"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15516709cog2901_3"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1089\/brain.2011.0038"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0277(84)90023-4"},{"key":"e_1_3_2_1_44_1","volume-title":"The emergence of norms","author":"Ullmann-Margalit E.","unstructured":"Ullmann-Margalit , E. 1977. The emergence of norms . Clarendon Press . Ullmann-Margalit, E. 1977. The emergence of norms. Clarendon Press."},{"key":"e_1_3_2_1_45_1","volume-title":"Darwin's cathedral: Evolution, religion, and the nature of society","author":"Wilson D.S.","unstructured":"Wilson , D.S. 2002. Darwin's cathedral: Evolution, religion, and the nature of society . University of Chicago Press. Wilson, D.S. 2002. Darwin's cathedral: Evolution, religion, and the nature of society. University of Chicago Press."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1353\/mpq.2008.0010"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0163141"}],"event":{"name":"AIES '19: AAAI\/ACM Conference on AI, Ethics, and Society","location":"Honolulu HI USA","acronym":"AIES '19","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","AAAI American Association for Artificial Intelligence"]},"container-title":["Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306618.3314252","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306618.3314252","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306618.3314252","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:02:04Z","timestamp":1750208524000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306618.3314252"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,27]]},"references-count":47,"alternative-id":["10.1145\/3306618.3314252","10.1145\/3306618"],"URL":"https:\/\/doi.org\/10.1145\/3306618.3314252","relation":{},"subject":[],"published":{"date-parts":[[2019,1,27]]},"assertion":[{"value":"2019-01-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}