{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T17:36:08Z","timestamp":1775756168683,"version":"3.50.1"},"reference-count":96,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2022,10,5]],"date-time":"2022-10-05T00:00:00Z","timestamp":1664928000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,5]],"date-time":"2022-10-05T00:00:00Z","timestamp":1664928000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100011898","name":"Marianne and Marcus Wallenberg Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011898","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100011898","name":"Marcus and Amalia Wallenberg Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011898","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003252","name":"Lund University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003252","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2024,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Although virtue ethics has repeatedly been proposed as a suitable framework for the development of artificial moral agents (AMAs), it has been proven difficult to approach from a computational perspective. In this work, we present the first technical implementation of artificial virtuous agents (AVAs) in moral simulations. First, we review previous conceptual and technical work in artificial virtue ethics and describe a functionalistic path to AVAs based on dispositional virtues, bottom-up learning, and top-down eudaimonic reward. We then provide the details of a technical implementation in a moral simulation based on a tragedy of the commons scenario. The experimental results show how the AVAs learn to tackle cooperation problems while exhibiting core features of their theoretical counterpart, including moral character, dispositional virtues, learning from experience, and the pursuit of eudaimonia. Ultimately, we argue that virtue ethics provides a compelling path toward morally excellent machines and that our work provides an important starting point for such endeavors.<\/jats:p>","DOI":"10.1007\/s00146-022-01569-x","type":"journal-article","created":{"date-parts":[[2022,10,5]],"date-time":"2022-10-05T10:03:02Z","timestamp":1664964182000},"page":"855-872","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Artificial virtuous agents in a multi-agent tragedy of the commons"],"prefix":"10.1007","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8579-3975","authenticated-orcid":false,"given":"Jakob","family":"Stenseke","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,10,5]]},"reference":[{"key":"1569_CR1","unstructured":"Abel D, MacGlashan J, Littman ML (2016) Reinforcement learning as a framework for ethical decision making. In: AAAI Workshop: AI, Ethics, and Society, vol\u00a016. Phoenix, AZ"},{"key":"1569_CR2","unstructured":"Amodei D, Olah C, Steinhardt J, et\u00a0al (2016) Concrete problems in ai safety. arXiv preprint arXiv:1606.06565"},{"key":"1569_CR3","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511978036","volume-title":"Machine ethics","author":"M Anderson","year":"2011","unstructured":"Anderson M, Anderson SL (2011) Machine ethics. Cambridge University Press"},{"key":"1569_CR4","unstructured":"Anderson M, Anderson SL (2008) Ethel: Toward a principled ethical eldercare system. In: AAAI Fall Symposium: AI in Eldercare: New Solutions to Old Problems"},{"key":"1569_CR5","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780199228782.001.0001","volume-title":"Intelligent virtue","author":"J Annas","year":"2011","unstructured":"Annas J (2011) Intelligent virtue. Oxford University Press"},{"issue":"124","key":"1569_CR6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1017\/S0031819100037943","volume":"33","author":"GEM Anscombe","year":"1958","unstructured":"Anscombe GEM (1958) Modern moral philosophy. Philosophy 33(124):1\u201319","journal-title":"Philosophy"},{"key":"1569_CR7","doi-asserted-by":"crossref","unstructured":"Arkin RC (2008) Governing lethal behavior: embedding ethics in a hybrid deliberative\/hybrid robot architecture. In: Proceedings of the 3rd ACM\/IEEE International Conference on Human Robot Interaction, pp 121\u2013128","DOI":"10.1145\/1349822.1349839"},{"key":"1569_CR8","unstructured":"Armstrong S (2015) Motivated value selection for artificial agents. In: Workshops at the Twenty-Ninth AAAI Conference on Artificial Intelligence"},{"issue":"4489","key":"1569_CR9","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.1126\/science.7466396","volume":"211","author":"R Axelrod","year":"1981","unstructured":"Axelrod R, Hamilton WD (1981) The evolution of cooperation. Science 211(4489):1390\u20131396","journal-title":"Science"},{"key":"1569_CR10","unstructured":"Bansal T, Pachocki J, Sidor S, et\u00a0al (2017) Emergent complexity via multi-agent competition. arXiv preprint arXiv:1710.03748"},{"issue":"1","key":"1569_CR11","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/s00146-018-0871-3","volume":"35","author":"WA Bauer","year":"2020","unstructured":"Bauer WA (2020) Virtuous vs. utilitarian artificial moral agents. AI & Soc 35(1):263\u2013271","journal-title":"AI & Soc"},{"key":"1569_CR12","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s11023-020-09525-8","volume":"30","author":"D Behdadi","year":"2020","unstructured":"Behdadi D, Munthe C (2020) A normative approach to artificial moral agency. Minds Mach 30:195\u2013218","journal-title":"Minds Mach"},{"key":"1569_CR13","unstructured":"Berberich N, Diepold K (2018) The virtuous machine-old ethics for new technology? arXiv preprint arXiv:1806.10322"},{"key":"1569_CR14","unstructured":"Berner C, Brockman G, Chan B, et\u00a0al (2019) Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680"},{"issue":"1","key":"1569_CR15","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s12369-020-00700-6","volume":"13","author":"ML Cappuccio","year":"2021","unstructured":"Cappuccio ML, Sandoval EB, Mubin O et al (2021) Can robots make us better humans? Int J Soc Robot 13(1):7\u201322","journal-title":"Int J Soc Robot"},{"issue":"10","key":"1569_CR16","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1038\/nrn1223","volume":"4","author":"WD Casebeer","year":"2003","unstructured":"Casebeer WD (2003) Moral cognition and its neural constituents. Nat Rev Neurosci 4(10):840\u2013846","journal-title":"Nat Rev Neurosci"},{"issue":"2","key":"1569_CR17","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1007\/s12559-015-9362-8","volume":"8","author":"JA Cervantes","year":"2016","unstructured":"Cervantes JA, Rodr\u00edguez LF, L\u00f3pez S et al (2016) Autonomous agents and ethical decision-making. Cogn Comput 8(2):278\u2013296","journal-title":"Cogn Comput"},{"issue":"2","key":"1569_CR18","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1007\/s11948-019-00151-x","volume":"26","author":"JA Cervantes","year":"2020","unstructured":"Cervantes JA, L\u00f3pez S, Rodr\u00edguez LF et al (2020) Artificial moral agents: a survey of the current status. Sci Eng Ethics 26(2):501\u2013532","journal-title":"Sci Eng Ethics"},{"key":"1569_CR19","first-page":"181","volume-title":"Feeling reasons","author":"PS Churchland","year":"1996","unstructured":"Churchland PS (1996) Feeling reasons. Springer, pp 181\u2013199"},{"issue":"4","key":"1569_CR20","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1023\/A:1013805017161","volume":"3","author":"KG Coleman","year":"2001","unstructured":"Coleman KG (2001) Android arete: toward a virtue ethic for computational agents. Ethics Inform Technol 3(4):247\u2013265","journal-title":"Ethics Inform Technol"},{"key":"1569_CR21","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-022-00887-w","author":"M Constantinescu","year":"2022","unstructured":"Constantinescu M, Crisp R (2022) Can robotic AI systems be virtuous and why does this matter? Int J Soc Robot. https:\/\/doi.org\/10.1007\/s12369-022-00887-w","journal-title":"Int J Soc Robot"},{"key":"1569_CR22","volume-title":"Virtue ethics","author":"R Crisp","year":"1997","unstructured":"Crisp R, Slote MA (1997) Virtue ethics. Oxford University Press, Oxford"},{"key":"1569_CR23","doi-asserted-by":"crossref","DOI":"10.4324\/9780203022344","volume-title":"Artificial morality: virtuous robots for virtual games","author":"P Danielson","year":"2002","unstructured":"Danielson P (2002) Artificial morality: virtuous robots for virtual games. Routledge"},{"key":"1569_CR24","unstructured":"Dehghani M, Tomai E, Forbus KD, Klenk M (2008) An integrated reasoning approach to moral decision-making. In: Proceedings of the 23rd AAI Conference on Artificial Intelligence, pp 1280\u20131286"},{"key":"1569_CR25","doi-asserted-by":"publisher","unstructured":"DeMoss D (1998) Aristotle, connectionism, and the morally excellent brain. In: The Paideia Archive: twentieth World Congress of Philosophy, vol 19, pp 13\u201320. https:\/\/doi.org\/10.5840\/wcp20-paideia199819352 .","DOI":"10.5840\/wcp20-paideia199819352"},{"key":"1569_CR26","volume-title":"Introduction to virtue ethics: insights of the ancient Greeks","author":"RJ Devettere","year":"2002","unstructured":"Devettere RJ (2002) Introduction to virtue ethics: insights of the ancient Greeks. Georgetown University Press"},{"issue":"3","key":"1569_CR27","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1177\/0270467604264992","volume":"24","author":"SE Dreyfus","year":"2004","unstructured":"Dreyfus SE (2004) The five-stage model of adult skill acquisition. Bull Sci Technol Soc 24(3):177\u2013181","journal-title":"Bull Sci Technol Soc"},{"key":"1569_CR28","unstructured":"Dulac-Arnold G, Mankowitz D, Hester T (2019) Challenges of real-world reinforcement learning. arXiv preprint arXiv:1904.12901"},{"issue":"4","key":"1569_CR29","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.tics.2004.02.007","volume":"8","author":"E Fehr","year":"2004","unstructured":"Fehr E, Fischbacher U (2004) Social norms and human cooperation. Trends Cogn Sci 8(4):185\u2013190","journal-title":"Trends Cogn Sci"},{"key":"1569_CR30","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1016\/B978-0-12-397025-1.00180-9","volume-title":"Brain Mapping","author":"O FeldmanHall","year":"2015","unstructured":"FeldmanHall O, Mobbs D (2015) A neural network for moral decision making. In: Toga AW (ed) Brain Mapping. Academic Press, pp 205\u2013210"},{"issue":"1654","key":"1569_CR31","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1098\/rspb.2008.0829","volume":"276","author":"JA Fletcher","year":"2009","unstructured":"Fletcher JA, Doebeli M (2009) A simple and general explanation for the evolution of altruism. Proc R Soc B 276(1654):13\u201319","journal-title":"Proc R Soc B"},{"issue":"3","key":"1569_CR32","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1007\/s11023-020-09539-2","volume":"30","author":"I Gabriel","year":"2020","unstructured":"Gabriel I (2020) Artificial intelligence, values, and alignment. Minds Mach 30(3):411\u2013437","journal-title":"Minds Mach"},{"issue":"4","key":"1569_CR33","doi-asserted-by":"crossref","first-page":"795","DOI":"10.1007\/s00146-020-00977-1","volume":"35","author":"P Gamez","year":"2020","unstructured":"Gamez P, Shank DB, Arnold C et al (2020) Artificial virtue: the machine question and perceptions of moral character in artificial moral agents. AI & Soc 35(4):795\u2013809","journal-title":"AI & Soc"},{"key":"1569_CR34","first-page":"243","volume-title":"Towards the ethical robot. Android epistemology","author":"J Gips","year":"1995","unstructured":"Gips J (1995) Towards the ethical robot. Android epistemology. MIT Press, pp 243\u2013252"},{"issue":"2","key":"1569_CR35","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/s13347-017-0285-z","volume":"31","author":"A Giubilini","year":"2018","unstructured":"Giubilini A, Savulescu J (2018) The artificial moral advisor. the \u201cideal observer\u2019\u2019 meets artificial intelligence. Philos Technol 31(2):169\u2013188","journal-title":"Philos Technol"},{"key":"1569_CR36","doi-asserted-by":"crossref","unstructured":"Govindarajulu NS, Bringsjord S, Ghosh R, Sarathy V (2019) Toward the engineering of virtuous machines. In: Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, pp 29\u201335","DOI":"10.1145\/3306618.3314256"},{"issue":"6","key":"1569_CR37","doi-asserted-by":"publisher","first-page":"1291","DOI":"10.1109\/TSMCC.2012.2218595","volume":"42","author":"I Grondman","year":"2012","unstructured":"Grondman I, Busoniu L, Lopes GAD et al (2012) A survey of actor-critic reinforcement learning: standard and natural policy gradients. IEEE Trans Syst Man Cybern Part C 42(6):1291\u20131307. https:\/\/doi.org\/10.1109\/TSMCC.2012.2218595","journal-title":"IEEE Trans Syst Man Cybern Part C"},{"issue":"4","key":"1569_CR38","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1109\/MIS.2006.76","volume":"21","author":"M Guarini","year":"2006","unstructured":"Guarini M (2006) Particularism and the classification and reclassification of moral cases. IEEE Intell Syst 21(4):22\u201328","journal-title":"IEEE Intell Syst"},{"key":"1569_CR39","first-page":"187","volume-title":"Coherence: insights from philosophy, jurisprudence and artificial intelligence","author":"M Guarini","year":"2013","unstructured":"Guarini M (2013) Case classification, similarities, spaces of reasons, and coherences. Coherence: insights from philosophy, jurisprudence and artificial intelligence. Springer, pp 187\u2013201"},{"issue":"2","key":"1569_CR40","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s11245-012-9130-2","volume":"32","author":"M Guarini","year":"2013","unstructured":"Guarini M (2013) Moral case classification and the nonlocality of reasons. Topoi 32(2):267\u2013289","journal-title":"Topoi"},{"issue":"3859","key":"1569_CR41","doi-asserted-by":"crossref","first-page":"1243","DOI":"10.1126\/science.162.3859.1243","volume":"162","author":"G Hardin","year":"1968","unstructured":"Hardin G (1968) The tragedy of the commons: the population problem has no technical solution; it requires a fundamental extension in morality. Science 162(3859):1243\u20131248","journal-title":"Science"},{"key":"1569_CR42","volume-title":"Filters against folly","author":"G Hardin","year":"1985","unstructured":"Hardin G (1985) Filters Against Folly: How to Survive Despite Economists, Ecologists, and the Merely Eloquent.\nViking Penguin"},{"issue":"2","key":"1569_CR43","doi-asserted-by":"crossref","first-page":"260","DOI":"10.5840\/monist20028529","volume":"85","author":"DM Haybron","year":"2002","unstructured":"Haybron DM (2002) Moral monsters and saints. Monist 85(2):260\u2013284","journal-title":"Monist"},{"issue":"4","key":"1569_CR44","doi-asserted-by":"crossref","first-page":"e1000","DOI":"10.1371\/journal.pcbi.1000758","volume":"6","author":"D Helbing","year":"2010","unstructured":"Helbing D, Szolnoki A, Perc M et al (2010) Evolutionary establishment of moral and double moral standards through spatial interactions. PLoS Comput Biol 6(4):e1000-758","journal-title":"PLoS Comput Biol"},{"issue":"12","key":"1569_CR45","doi-asserted-by":"crossref","first-page":"3999","DOI":"10.1073\/pnas.0308738101","volume":"101","author":"CA Holt","year":"2004","unstructured":"Holt CA, Roth AE (2004) The nash equilibrium: a perspective. Proc Natl Acad Sci 101(12):3999\u20134002","journal-title":"Proc Natl Acad Sci"},{"key":"1569_CR46","unstructured":"Hostallero DE, Kim D, Moon S, et\u00a0al (2020) Inducing cooperation through reward reshaping based on peer evaluations in deep multi-agent reinforcement learning. In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp 520\u2013528"},{"key":"1569_CR47","first-page":"121","volume-title":"Philos Technol","author":"D Howard","year":"2017","unstructured":"Howard D, Muntean I (2017) Artificial moral cognition: moral functionalism and autonomous moral agency. Philos Technol. Springer, pp 121\u2013159"},{"key":"1569_CR48","volume-title":"On virtue ethics","author":"R Hursthouse","year":"1999","unstructured":"Hursthouse R (1999) On virtue ethics. OUP Oxford"},{"key":"1569_CR49","volume-title":"The stanford encyclopedia of philosophy, winter","author":"R Hursthouse","year":"2018","unstructured":"Hursthouse R, Pettigrove G (2018) Virtue Ethics. In: Zalta EN (ed) The stanford encyclopedia of philosophy, winter, 2018th edn. Stanford University, Metaphysics Research Lab","edition":"2018"},{"key":"1569_CR50","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1007\/11564096_58","volume-title":"Machine learning: ECML 2005","author":"MT Izadi","year":"2005","unstructured":"Izadi MT, Precup D (2005) Using rewards for belief state updates in partially observable markov decision processes. In: Gama J, Camacho R, Brazdil PB et al (eds) Machine learning: ECML 2005. Springer, Heidelberg, pp 593\u2013600"},{"issue":"178","key":"1569_CR51","doi-asserted-by":"crossref","first-page":"20","DOI":"10.2307\/2219846","volume":"45","author":"F Jackson","year":"1995","unstructured":"Jackson F, Pettit P (1995) Moral functionalism and moral motivation. Philos Q (1950-) 45(178):20\u201340","journal-title":"Philos Q (1950-)"},{"issue":"1\u20132","key":"1569_CR52","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","volume":"101","author":"LP Kaelbling","year":"1998","unstructured":"Kaelbling LP, Littman ML, Cassandra AR (1998) Planning and acting in partially observable stochastic domains. Artif Intell 101(1\u20132):99\u2013134","journal-title":"Artif Intell"},{"key":"1569_CR53","doi-asserted-by":"crossref","DOI":"10.4324\/9781315197128","volume-title":"Ethics for robots: how to design a moral algorithm","author":"D Leben","year":"2018","unstructured":"Leben D (2018) Ethics for robots: how to design a moral algorithm. Routledge"},{"issue":"103","key":"1569_CR54","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1016\/j.artint.2020.103350","volume":"287","author":"F Lindner","year":"2020","unstructured":"Lindner F, Mattm\u00fcller R, Nebel B (2020) Evaluation of the moral permissibility of action plans. Artif Intell 287(103):350. https:\/\/doi.org\/10.1016\/j.artint.2020.103350","journal-title":"Artif Intell"},{"issue":"1\u20134","key":"1569_CR55","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.jhydrol.2004.04.006","volume":"297","author":"HA Lo\u00e1iciga","year":"2004","unstructured":"Lo\u00e1iciga HA (2004) Analytic game\u2013theoretic approach to ground-water extraction. J Hydrol 297(1\u20134):22\u201333","journal-title":"J Hydrol"},{"key":"1569_CR56","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-020-00400-z","author":"BA Mabaso","year":"2020","unstructured":"Mabaso BA (2020) Artificial moral agents within an ethos of ai4sg. Phil Technol. https:\/\/doi.org\/10.1007\/s13347-020-00400-z","journal-title":"Phil Technol"},{"key":"1569_CR57","unstructured":"Madani O, Hanks S, Condon A (1999) On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision problems. In: Proceedings of the 16th National Conference on Artificial Intelligence, Orlando, FL (AAAI\/IAAI), pp 541\u2013548"},{"issue":"4","key":"1569_CR58","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10676-015-9367-8","volume":"18","author":"BF Malle","year":"2016","unstructured":"Malle BF (2016) Integrating robot ethics and machine morality: the study and design of moral competence in robots. Ethics Inform Technol 18(4):243\u2013256","journal-title":"Ethics Inform Technol"},{"key":"1569_CR59","doi-asserted-by":"crossref","unstructured":"Maree C, Omlin C (2022) Reinforcement learning your way: Agent characterization through policy regularization. arXiv preprint arXiv:2201.10003","DOI":"10.3390\/ai3020015"},{"issue":"6","key":"1569_CR60","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1007\/s10458-019-09422-0","volume":"33","author":"A Morris-Martin","year":"2019","unstructured":"Morris-Martin A, De Vos M, Padget J (2019) Norm emergence in multiagent systems: a viewpoint paper. Auton Agents Multi-Agent Syst 33(6):706\u2013749","journal-title":"Auton Agents Multi-Agent Syst"},{"issue":"4","key":"1569_CR61","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1145\/347476.347480","volume":"47","author":"M Mundhenk","year":"2000","unstructured":"Mundhenk M, Goldsmith J, Lusena C et al (2000) Complexity of finite-horizon markov decision process problems. J ACM (JACM) 47(4):681\u2013720","journal-title":"J ACM (JACM)"},{"issue":"1","key":"1569_CR62","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1073\/pnas.36.1.48","volume":"36","author":"JF Nash","year":"1950","unstructured":"Nash JF et al (1950) Equilibrium points in n-person games. Proc Natl Acad Sci 36(1):48\u201349","journal-title":"Proc Natl Acad Sci"},{"key":"1569_CR63","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2021.715849","author":"M Navon","year":"2021","unstructured":"Navon M (2021) The virtuous servant owner-a paradigm whose time has come (again). Front Robot AI. https:\/\/doi.org\/10.3389\/frobt.2021.715849","journal-title":"Front Robot AI"},{"key":"1569_CR64","unstructured":"Ng AY, Russell SJ (2000) Algorithms for inverse reinforcement learning. In: Proceedings of the 17th International Conference on Machine Learning (ICML), vol 1, p 2"},{"key":"1569_CR65","doi-asserted-by":"crossref","unstructured":"Noothigattu R, Gaikwad S, Awad E, et\u00a0al (2018) A voting-based system for ethical decision making. In: Proceedings of the AAAI Conference on Artificial Intelligence","DOI":"10.1609\/aaai.v32i1.11512"},{"issue":"5805","key":"1569_CR66","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1126\/science.1133755","volume":"314","author":"MA Nowak","year":"2006","unstructured":"Nowak MA (2006) Five rules for the evolution of cooperation. Science 314(5805):1560\u20131563","journal-title":"Science"},{"key":"1569_CR67","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1111\/j.1475-4975.1988.tb00111.x","volume":"13","author":"MC Nussbaum","year":"1988","unstructured":"Nussbaum MC (1988) Non-relative virtues: an Aristotelian approach. Midwest Stud Philos 13:32\u201353","journal-title":"Midwest Stud Philos"},{"issue":"1\u20132","key":"1569_CR68","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/S0304-3800(02)00064-9","volume":"154","author":"JD Olden","year":"2002","unstructured":"Olden JD, Jackson DA (2002) Illuminating the \u201cblack box\u2019\u2019: a randomization approach for understanding variable contributions in artificial neural networks. Ecol Model 154(1\u20132):135\u2013150","journal-title":"Ecol Model"},{"issue":"1","key":"1569_CR69","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1007\/s12369-019-00592-1","volume":"13","author":"A Peeters","year":"2021","unstructured":"Peeters A, Haselager P (2021) Designing virtuous sex robots. Int J Soc Robot 13(1):55\u201366","journal-title":"Int J Soc Robot"},{"key":"1569_CR70","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-29354-7","volume-title":"Programming machine ethics","author":"LM Pereira","year":"2016","unstructured":"Pereira LM, Saptawijaya A et al (2016) Programming machine ethics. Springer"},{"key":"1569_CR71","unstructured":"Pereira LM, Lenaerts T, Martinez-Vaquero LA, Han TA (2017) Social manifestation of guilt leads to stable cooperation in multi-agent systems. In: Proceedings of the 16th Confrerence on Autonomous Agents and MultiAgent Systems (AAMAS), pp 1422\u20131430"},{"issue":"1","key":"1569_CR72","doi-asserted-by":"crossref","first-page":"10","DOI":"10.3390\/e24010010","volume":"24","author":"LM Pereira","year":"2021","unstructured":"Pereira LM, Han TA, Lopes AB (2021) Employing AI to better understand our morals. Entropy 24(1):10","journal-title":"Entropy"},{"issue":"3","key":"1569_CR73","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/s10551-007-9380-x","volume":"79","author":"RR Radtke","year":"2008","unstructured":"Radtke RR (2008) Role morality in the accounting profession-how do we compare to physicians and attorneys? J Bus Ethics 79(3):279\u2013297","journal-title":"J Bus Ethics"},{"key":"1569_CR74","unstructured":"Rodriguez-Soto M, Lopez-Sanchez M, Rodriguez-Aguilar JA (2020) A structural solution to sequential moral dilemmas. In: Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, pp 1152\u20131160"},{"key":"1569_CR75","doi-asserted-by":"crossref","unstructured":"Rodriguez-Soto M, Lopez-Sanchez M, Rodriguez-Aguilar JA (2021) Multi-objective reinforcement learning for designing ethical environments. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence, pp 1\u20137","DOI":"10.24963\/ijcai.2021\/76"},{"issue":"1","key":"1569_CR76","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/s10902-006-9023-4","volume":"9","author":"RM Ryan","year":"2008","unstructured":"Ryan RM, Huta V, Deci EL (2008) Living well: a self-determination theory perspective on eudaimonia. J Happiness Stud 9(1):139\u2013170. https:\/\/doi.org\/10.1007\/s10902-006-9023-4","journal-title":"J Happiness Stud"},{"issue":"1","key":"1569_CR77","doi-asserted-by":"crossref","first-page":"e10047,09","DOI":"10.1371\/journal.pcbi.1004709","volume":"12","author":"FP Santos","year":"2016","unstructured":"Santos FP, Santos FC, Pacheco JM (2016) Social norms of cooperation in small-scale societies. PLoS Comput Biol 12(1):e10047,09","journal-title":"PLoS Comput Biol"},{"issue":"1","key":"1569_CR78","doi-asserted-by":"crossref","first-page":"21","DOI":"10.3233\/MGS-2011-0167","volume":"7","author":"BTR Savarimuthu","year":"2011","unstructured":"Savarimuthu BTR, Cranefield S (2011) Norm creation, spreading and emergence: a survey of simulation models of norms in multi-agent systems. Multiagent Grid Syst 7(1):21\u201354","journal-title":"Multiagent Grid Syst"},{"key":"1569_CR79","volume-title":"Goods and Virtues","author":"MA Slote","year":"1983","unstructured":"Slote MA (1983) Goods and Virtues. Oxford University Press"},{"key":"1569_CR80","volume-title":"From morality to virtue","author":"MA Slote","year":"1992","unstructured":"Slote MA (1992) From morality to virtue. Oxford University Press on Demand"},{"key":"1569_CR81","doi-asserted-by":"crossref","unstructured":"Slote M (1995) Agent-based virtue ethics. Midwestern Studies in Philosophy 20(1):83\u2013101","DOI":"10.1111\/j.1475-4975.1995.tb00306.x"},{"issue":"5427","key":"1569_CR82","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1038\/246015a0","volume":"246","author":"J Smith","year":"1973","unstructured":"Smith J, Price GR (1973) The logic of animal conflict. Nature 246(5427):15\u201318","journal-title":"Nature"},{"issue":"1","key":"1569_CR83","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s12369-020-00631-2","volume":"13","author":"R Sparrow","year":"2021","unstructured":"Sparrow R (2021) Virtue and vice in our relationships with robots: is there an asymmetry and how might it be explained? Int J Soc Robot 13(1):23\u201329","journal-title":"Int J Soc Robot"},{"key":"1569_CR84","unstructured":"Stanley KO, Miikkulainen R (2002) Efficient evolution of neural network topologies. In: Proceedings of the 2002 Congress on Evolutionary Computation. CEC\u201902 (Cat. No. 02TH8600), vol\u00a02. IEEE, pp 1757\u20131762"},{"key":"1569_CR85","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-021-01325-7","author":"J Stenseke","year":"2021","unstructured":"Stenseke J (2021) Artificial virtuous agents: from theory to machine implementation. AI Soc. https:\/\/doi.org\/10.1007\/s00146-021-01325-7","journal-title":"AI Soc"},{"key":"1569_CR86","doi-asserted-by":"crossref","DOI":"10.4324\/9780203882566","volume-title":"Freedom and resentment and other essays","author":"PF Strawson","year":"2008","unstructured":"Strawson PF (2008) Freedom and resentment and other essays. Routledge"},{"key":"1569_CR87","doi-asserted-by":"crossref","unstructured":"Sullins JP (2021) Artificial phronesis: What It Is and What It Is Not. In: Ratti E, Stapleford TA (eds) Science, Technology, and Virtues: Contemporary Perspectives, Oxford University Press, p 136","DOI":"10.1093\/oso\/9780190081713.003.0008"},{"issue":"3","key":"1569_CR88","doi-asserted-by":"crossref","first-page":"38003","DOI":"10.1209\/0295-5075\/92\/38003","volume":"92","author":"A Szolnoki","year":"2010","unstructured":"Szolnoki A (2010) Reward and cooperation in the spatial public goods game. EPL (Europhys Lett) 92(3):38003","journal-title":"EPL (Europhys Lett)"},{"issue":"6","key":"1569_CR89","doi-asserted-by":"crossref","first-page":"1429","DOI":"10.1109\/TITS.2016.2609339","volume":"18","author":"SM Thornton","year":"2016","unstructured":"Thornton SM, Pan S, Erlien SM et al (2016) Incorporating ethical considerations into automated vehicle control. IEEE Trans Intell Transp Syst 18(6):1429\u20131439","journal-title":"IEEE Trans Intell Transp Syst"},{"issue":"6","key":"1569_CR90","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3419633","volume":"53","author":"S Tolmeijer","year":"2020","unstructured":"Tolmeijer S, Kneer M, Sarasua C et al (2020) Implementations in machine ethics: a survey. ACM Comput Surv (CSUR) 53(6):1\u201338","journal-title":"ACM Comput Surv (CSUR)"},{"issue":"2","key":"1569_CR91","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s10676-012-9290-1","volume":"14","author":"R Tonkens","year":"2012","unstructured":"Tonkens R (2012) Out of character: on the creation of virtuous machines. Ethics Inform Technol 14(2):137\u2013149","journal-title":"Ethics Inform Technol"},{"key":"1569_CR92","volume-title":"Moral machines: teaching robots right from wrong","author":"W Wallach","year":"2008","unstructured":"Wallach W, Allen C (2008) Moral machines: teaching robots right from wrong. Oxford University Press"},{"key":"1569_CR93","unstructured":"Wang JX, Hughes E, Fernando C, et\u00a0al (2018) Evolving intrinsic motivations for altruistic behavior. arXiv preprint arXiv:1811.05931"},{"issue":"4","key":"1569_CR94","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1016\/S0019-9958(77)90354-0","volume":"34","author":"IH Witten","year":"1977","unstructured":"Witten IH (1977) An adaptive optimal controller for discrete-time markov environments. Inf Control 34(4):286\u2013295. https:\/\/doi.org\/10.1016\/S0019-9958(77)90354-0","journal-title":"Inf Control"},{"issue":"1\u20132","key":"1569_CR95","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1111\/j.1467-9973.2009.01627.x","volume":"41","author":"L Zagzebski","year":"2010","unstructured":"Zagzebski L (2010) Exemplarist virtue theory. Metaphilosophy 41(1\u20132):41\u201357","journal-title":"Metaphilosophy"},{"issue":"4","key":"1569_CR96","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1007\/s10458-019-09411-3","volume":"33","author":"C Zhang","year":"2019","unstructured":"Zhang C, Li X, Hao J et al (2019) Sa-iga: a multiagent reinforcement learning method towards socially optimal outcomes. Auton Agents Multi-Agent Syst 33(4):403\u2013429","journal-title":"Auton Agents Multi-Agent Syst"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01569-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-022-01569-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-022-01569-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,28]],"date-time":"2024-06-28T17:01:57Z","timestamp":1719594117000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-022-01569-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,5]]},"references-count":96,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6]]}},"alternative-id":["1569"],"URL":"https:\/\/doi.org\/10.1007\/s00146-022-01569-x","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,5]]},"assertion":[{"value":"25 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The author has no competing financial or non-financial interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}