{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T12:18:55Z","timestamp":1775737135681,"version":"3.50.1"},"reference-count":55,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,1,24]],"date-time":"2022-01-24T00:00:00Z","timestamp":1642982400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,1,24]],"date-time":"2022-01-24T00:00:00Z","timestamp":1642982400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["FPU18\/03387"],"award-info":[{"award-number":["FPU18\/03387"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["COREDEM (H2020-785907)"],"award-info":[{"award-number":["COREDEM (H2020-785907)"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["AI4EU (H2020-825619)"],"award-info":[{"award-number":["AI4EU (H2020-825619)"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["LOGISTAR (H2020-769142)"],"award-info":[{"award-number":["LOGISTAR (H2020-769142)"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["Crowd4SDG (H2020-872944)"],"award-info":[{"award-number":["Crowd4SDG (H2020-872944)"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["CI-SUSTAIN (PID2019-104156GB-I00)"],"award-info":[{"award-number":["CI-SUSTAIN (PID2019-104156GB-I00)"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["COMRIDI18-1-0010-02"],"award-info":[{"award-number":["COMRIDI18-1-0010-02"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","award":["MISMIS PGC2018- 096212B-C33"],"award-info":[{"award-number":["MISMIS PGC2018- 096212B-C33"]}],"id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007601","name":"Horizon 2020","doi-asserted-by":"publisher","award":["TAILOR (H2020-952215)"],"award-info":[{"award-number":["TAILOR (H2020-952215)"]}],"id":[{"id":"10.13039\/501100007601","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002809","name":"Generalitat de Catalunya","doi-asserted-by":"publisher","award":["2017 SGR 172"],"award-info":[{"award-number":["2017 SGR 172"]}],"id":[{"id":"10.13039\/501100002809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002809","name":"Generalitat de Catalunya","doi-asserted-by":"publisher","award":["2017 SGR 341"],"award-info":[{"award-number":["2017 SGR 341"]}],"id":[{"id":"10.13039\/501100002809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Instituto de Investigaci\u00f3n en Inteligencia Artificial"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Ethics Inf Technol"],"published-print":{"date-parts":[[2022,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>AI research is being challenged with ensuring that autonomous agents learn to behave ethically, namely in alignment with moral values. Here, we propose a novel way of tackling the value alignment problem as a two-step process. The first step consists on formalising moral values and value aligned behaviour based on philosophical foundations. Our formalisation is compatible with the framework of (Multi-Objective) Reinforcement Learning, to ease the handling of an agent\u2019s individual and ethical objectives. The second step consists in designing an environment wherein an agent learns to behave ethically while pursuing its individual objective. We leverage on our theoretical results to introduce an algorithm that automates our two-step approach. In the cases where value-aligned behaviour is possible, our algorithm produces a learning environment for the agent wherein it will learn a value-aligned behaviour.<\/jats:p>","DOI":"10.1007\/s10676-022-09635-0","type":"journal-article","created":{"date-parts":[[2022,1,31]],"date-time":"2022-01-31T15:02:58Z","timestamp":1643641378000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Instilling moral value alignment by means of multi-objective reinforcement learning"],"prefix":"10.1007","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1339-2018","authenticated-orcid":false,"given":"Manel","family":"Rodriguez-Soto","sequence":"first","affiliation":[]},{"given":"Marc","family":"Serramia","sequence":"additional","affiliation":[]},{"given":"Maite","family":"Lopez-Sanchez","sequence":"additional","affiliation":[]},{"given":"Juan Antonio","family":"Rodriguez-Aguilar","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,1,24]]},"reference":[{"key":"9635_CR1","doi-asserted-by":"publisher","unstructured":"Abbeel, P., & Ng, A. Y. (2004). Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the Twenty-first International Conference on Machine Learning, ICML \u201904, p. 1. ACM, New York, NY, USA. https:\/\/doi.org\/10.1145\/1015330.1015430.","DOI":"10.1145\/1015330.1015430"},{"key":"9635_CR2","unstructured":"Abel, D., MacGlashan, J., & Littman, M. L. (2016). Reinforcement learning as a framework for ethical decision making. In AAAI Work.: AI, Ethics, and Society (vol. 92)."},{"key":"9635_CR4","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1007\/s10676-006-0004-4","volume":"7","author":"C Allen","year":"2005","unstructured":"Allen, C., Smit, I., & Wallach, W. (2005). Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics and Information Technology, 7, 149\u2013155. https:\/\/doi.org\/10.1007\/s10676-006-0004-4.","journal-title":"Ethics and Information Technology"},{"key":"9635_CR5","unstructured":"Amodei, D., Olah, C., Steinhardt, J., Christiano, P. F., Schulman, J., & Man\u00e9, D. (2016). Concrete problems in ai safety. CoRR arXiv:1606.06565."},{"key":"9635_CR6","unstructured":"Arnold, T., Kasenberg, D., & Scheutz, M. (2017). Value alignment or misalignment - what will keep systems accountable?. In AAAI Workshops"},{"key":"9635_CR7","volume-title":"The Cambridge Dictionary of Philosophy","author":"R Audi","year":"1999","unstructured":"Audi, R. (1999). The Cambridge Dictionary of Philosophy. Cambridge University Press."},{"key":"9635_CR8","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1609\/aaai.v33i01.33013","volume":"33","author":"A Balakrishnan","year":"2019","unstructured":"Balakrishnan, A., Bouneffouf, D., Mattei, N., & Rossi, F. (2019). Incorporating behavioral constraints in online ai systems. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 3\u201311. https:\/\/doi.org\/10.1609\/aaai.v33i01.33013.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"9635_CR9","doi-asserted-by":"publisher","first-page":"71","DOI":"10.26362\/20180204","volume":"17","author":"R Barcaro","year":"2018","unstructured":"Barcaro, R., Mazzoleni, M., & Virgili, P. (2018). Ethics of care and robot caregivers. Prolegomena, 17, 71\u201380. https:\/\/doi.org\/10.26362\/20180204.","journal-title":"Prolegomena"},{"key":"9635_CR10","doi-asserted-by":"publisher","unstructured":"Barrett, L., & Narayanan, S. (2008). Learning all optimal policies with multiple criteria. In Proceedings of the 25th International Conference on Machine Learning pp. 41\u201347. https:\/\/doi.org\/10.1145\/1390156.1390162","DOI":"10.1145\/1390156.1390162"},{"key":"9635_CR11","volume-title":"Brief history of ethics","author":"V Camps","year":"2013","unstructured":"Camps, V. (2013). Brief history of ethics. BA."},{"issue":"1","key":"9635_CR12","first-page":"1","volume":"5","author":"RM Chisholm","year":"1963","unstructured":"Chisholm, R. M. (1963). Supererogation and offence: A conceptual scheme for ethics. Ratio (Misc.), 5(1), 1.","journal-title":"Ratio (Misc.)"},{"key":"9635_CR13","unstructured":"Chow, Y., Nachum, O., Duenez-Guzman, E., & Ghavamzadeh, M. (2018). A lyapunov-based approach to safe reinforcement learning. NIPS\u201918."},{"key":"9635_CR14","doi-asserted-by":"crossref","unstructured":"Conee, E. (1982). Against moral dilemmas. The Philosophical Review,91(1), 87\u201397. http:\/\/www.jstor.org\/stable\/2184670.","DOI":"10.2307\/2184670"},{"key":"9635_CR15","volume-title":"Value pluralism and ethical choice","author":"D Cooper","year":"1993","unstructured":"Cooper, D. (1993). Value pluralism and ethical choice. St. Martin Press Inc."},{"key":"9635_CR16","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1016\/j.engappai.2016.12.023","volume":"59","author":"J Domingo-Ferrer","year":"2017","unstructured":"Domingo-Ferrer, J., Mart\u00ednez, S., S\u00ednchez, D., & Soria-Comas, J. (2017). Co-utility: Self-enforcing protocols for the mutual benefit of participants. Engineering Applications of Artificial Intelligence, 59, 148\u2013158. https:\/\/doi.org\/10.1016\/j.engappai.2016.12.023.","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"9635_CR17","unstructured":"Duignan, B. (2018). Ought implies can. Retrieved January 15, 2015, from https:\/\/www.britannica.com\/topic\/ought-implies-can"},{"issue":"9","key":"9635_CR18","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1145\/2955091","volume":"59","author":"A Etzioni","year":"2016","unstructured":"Etzioni, A., & Etzioni, O. (2016). Designing ai systems that obey our laws and values. Communications of the ACM, 59(9), 29\u201331. https:\/\/doi.org\/10.1145\/2955091.","journal-title":"Communications of the ACM"},{"key":"9635_CR19","unstructured":"Fieser, J., & Dowden, B. (2000). Ethics. https:\/\/www.iep.utm.edu\/ethics\/ (The Internet Encyclopedia of Philosophy)."},{"key":"9635_CR20","volume-title":"Ethics","author":"WK Frankena","year":"1973","unstructured":"Frankena, W. K. (1973). Ethics (2nd ed.). Prentice-Hall.","edition":"2"},{"key":"9635_CR21","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s11023-020-09539-2","volume":"30","author":"I Gabriel","year":"2020","unstructured":"Gabriel, I. (2020). Artificial intelligence, values, and alignment. Minds and Machines, 30, 411\u2013437. https:\/\/doi.org\/10.1007\/s11023-020-09539-2.","journal-title":"Minds and Machines"},{"issue":"1","key":"9635_CR22","first-page":"1437","volume":"16","author":"J Garc\u00eda","year":"2015","unstructured":"Garc\u00eda, J., & Fern\u00e1ndez, F. (2015). A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16(1), 1437\u20131480.","journal-title":"Journal of Machine Learning Research"},{"key":"9635_CR23","doi-asserted-by":"publisher","unstructured":"Gigerenzer, G. (2010). Moral satisficing: Rethinking moral behavior as bounded rationality. Topics in Cognitive Science,2(3), 528\u2013554. https:\/\/doi.org\/10.1111\/j.1756-8765.2010.01094.x. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1111\/j.1756-8765.2010.01094.x.","DOI":"10.1111\/j.1756-8765.2010.01094.x"},{"key":"9635_CR24","unstructured":"Hadfield-Menell, D., Russell, S. J., Abbeel, P., & Dragan, A. (2016). Cooperative inverse reinforcement learning. In Advances in Neural Information Processing Systems 29, pp. 3909\u20133917. Berkeley."},{"issue":"2","key":"9635_CR25","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1093\/logcom\/exs065","volume":"25","author":"SO Hansson","year":"2013","unstructured":"Hansson, S. O. (2013). Representing supererogation. Journal of Logic and Computation, 25(2), 443\u2013451. https:\/\/doi.org\/10.1093\/logcom\/exs065.","journal-title":"Journal of Logic and Computation"},{"key":"9635_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-77434-3","volume-title":"Introduction to Formal Philosophy","author":"SO Hansson","year":"2018","unstructured":"Hansson, S. O., & Hendricks, V. (2018). Introduction to Formal Philosophy. Springer."},{"key":"9635_CR27","doi-asserted-by":"crossref","unstructured":"Heyd, D. (2016). Supererogation. In Zalta E. N. (ed.) The Stanford encyclopedia of philosophy, spring 2016 edn. https:\/\/plato.stanford.edu\/entries\/supererogation\/","DOI":"10.1093\/obo\/9780195396577-0329"},{"issue":"1","key":"9635_CR28","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1613\/jair.301","volume":"4","author":"LP Kaelbling","year":"1996","unstructured":"Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. The Journal of Artificial Intelligence Research, 4(1), 237\u2013285.","journal-title":"The Journal of Artificial Intelligence Research"},{"key":"9635_CR29","unstructured":"Leike, J., Martic, M., Krakovna, V., Ortega, P., Everitt, T., Lefrancq, A., Orseau, L., & Legg, S. (2017). Ai safety gridworlds. arXiv:1711.09883."},{"key":"9635_CR30","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/978-3-662-45854-9_4","volume-title":"Why ethics matters for autonomous cars","author":"P Lin","year":"2015","unstructured":"Lin, P. (2015). Why ethics matters for autonomous cars (pp. 69\u201385). Springer. https:\/\/doi.org\/10.1007\/978-3-662-45854-9_4."},{"key":"9635_CR31","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1038\/nature14540","volume":"521","author":"M Littman","year":"2015","unstructured":"Littman, M. (2015). Reinforcement learning improves behaviour from evaluative feedback. Nature, 521, 445\u201351. https:\/\/doi.org\/10.1038\/nature14540.","journal-title":"Nature"},{"issue":"1","key":"9635_CR32","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/BF00370674","volume":"57","author":"P McNamara","year":"1996","unstructured":"McNamara, P. (1996). Doing well enough: Toward a logic for common-sense morality. Studia Logica, 57(1), 167\u2013192. https:\/\/doi.org\/10.1007\/BF00370674.","journal-title":"Studia Logica"},{"issue":"1","key":"9635_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18564\/jasss.3929","volume":"22","author":"R Mercuur","year":"2019","unstructured":"Mercuur, R., Dignum, V., Jonker, C., et al. (2019). The value of values and norms in social simulation. Journal Artificial Societies and Social Simulation, 22(1), 1\u20139.","journal-title":"Journal Artificial Societies and Social Simulation"},{"key":"9635_CR34","unstructured":"Miryoosefi, S., Brantley, K., Iii, H., Dud\u00edk, M., & Schapire, R. (2020). Reinforcement learning with convex constraints. In Advances in Neural Information Processing Systems."},{"key":"9635_CR35","doi-asserted-by":"publisher","unstructured":"Nisan, N., & Ronen, A. (2001). Algorithmic mechanism design. Games and Economic Behavior,35(1), 166\u2013196. https:\/\/doi.org\/10.1006\/game.1999.0790. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S089982569990790X.","DOI":"10.1006\/game.1999.0790"},{"key":"9635_CR36","doi-asserted-by":"publisher","first-page":"6377","DOI":"10.1147\/JRD.2019.2940428","volume":"PP","author":"R Noothigattu","year":"2019","unstructured":"Noothigattu, R., Bouneffouf, D., Mattei, N., Chandra, R., Madan, P., Kush, R., et al. (2019). Teaching ai agents ethical values using reinforcement learning and policy orchestration. IBM Journal of Research and Development, PP, 6377\u20136381. https:\/\/doi.org\/10.1147\/JRD.2019.2940428.","journal-title":"IBM Journal of Research and Development"},{"key":"9635_CR38","unstructured":"Riedl, M. O., & Harrison, B. (2016). Using stories to teach human values to artificial agents. In AAAI Workshop: AI, Ethics, and Society."},{"key":"9635_CR39","unstructured":"Rodriguez-Soto, M., Lopez-Sanchez, M., & Rodr\u00edguez-Aguilar, J. A. (2020). A structural solution to sequential moral dilemmas. In Proceedings of the 19th International Conference on Autonomous Agents and Multi-Agent Aystems (AAMAS 2020)."},{"key":"9635_CR40","doi-asserted-by":"publisher","unstructured":"Roijers, D., & Whiteson, S. (2017). Multi-Objective Decision Making. Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan and Claypool, California, USA. https:\/\/doi.org\/10.2200\/S00765ED1V01Y201704AIM034. http:\/\/www.morganclaypool.com\/doi\/abs\/10.2200\/S00765ED1V01Y201704AIM034.","DOI":"10.2200\/S00765ED1V01Y201704AIM034"},{"key":"9635_CR41","doi-asserted-by":"publisher","first-page":"9785","DOI":"10.1609\/aaai.v33i01.33019785","volume":"33","author":"F Rossi","year":"2019","unstructured":"Rossi, F., & Mattei, N. (2019). Building ethically bounded ai. Proceedings of the AAAI Conference on Artificial Intelligence, 33, 9785\u20139789. https:\/\/doi.org\/10.1609\/aaai.v33i01.33019785.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"9635_CR42","volume-title":"Human compatible. AI and the problem of control","author":"S Russell","year":"2019","unstructured":"Russell, S. (2019). Human compatible. AI and the problem of control. Penguin Books."},{"key":"9635_CR43","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1609\/aimag.v36i4.2577","volume":"36","author":"S Russell","year":"2015","unstructured":"Russell, S., Dewey, D., & Tegmark, M. (2015). Research priorities for robust and beneficial artificial intelligence. Ai Magazine, 36, 105\u2013114. https:\/\/doi.org\/10.1609\/aimag.v36i4.2577.","journal-title":"Ai Magazine"},{"key":"9635_CR44","unstructured":"Serramia, M., Lopez-Sanchez, M., & Rodriguez-Aguilar, J. A. (2020). A qualitative approach to composing value-aligned norm systems. In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS \u201920, p. 1233\u20131241. International Foundation for Autonomous Agents and Multiagent Systems, Richland, SC."},{"key":"9635_CR45","unstructured":"Serramia, M., Lopez-Sanchez, M., Rodriguez-Aguilar, J. A., Rodriguez, M., Wooldridge, M., Morales, J., & Ansotegui, C. (2018). Moral values in norm decision making. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (AAMAS\u201918), pp. 1294\u20131302. International Foundation for Autonomous Agents and Multiagent Systems."},{"key":"9635_CR46","unstructured":"Sierra, C., Osman, N., Noriega, P., Sabater-Mir, J., & Perello-Moragues, A. (2019). Value alignment: A formal approach. In Responsible Artificial Intelligence Agents Workshop (RAIA) in AAMAS 2019."},{"key":"9635_CR47","unstructured":"Soares, N., & Fallenstein, B. (2014). Aligning superintelligence with human interests: A technical research agenda. Machine Intelligence Research Institute (MIRI) technical report 8."},{"key":"9635_CR48","doi-asserted-by":"publisher","first-page":"54","DOI":"10.11590\/abhps.2020.2.04","volume":"8","author":"M Sutrop","year":"2020","unstructured":"Sutrop, M. (2020). Challenges of aligning artificial intelligence with human values. Acta Baltica Historiae et Philosophiae Scientiarum, 8, 54\u201372. https:\/\/doi.org\/10.11590\/abhps.2020.2.04.","journal-title":"Acta Baltica Historiae et Philosophiae Scientiarum"},{"key":"9635_CR49","unstructured":"Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning\u2014an introduction. Adaptive computation and machine learning. MIT Press. http:\/\/www.worldcat.org\/oclc\/37293240."},{"key":"9635_CR50","doi-asserted-by":"publisher","DOI":"10.1145\/3419633","author":"S Tolmeijer","year":"2021","unstructured":"Tolmeijer, S., Kneer, M., Sarasua, C., Christen, M., & Bernstein, A. (2021). Implementations in machine ethics: A survey. ACM Computing Surveys. https:\/\/doi.org\/10.1145\/3419633.","journal-title":"ACM Computing Surveys"},{"key":"9635_CR51","volume-title":"Essays in moral philosophy","author":"JO Urmson","year":"1958","unstructured":"Urmson, J. O. (1958). Saints and heroes. In A. I. Melden (Ed.), Essays in moral philosophy. University of Washington Press."},{"key":"9635_CR52","volume-title":"Ethics, technology, and engineering: An introduction","author":"I van de Poel","year":"2011","unstructured":"van de Poel, I., & Royakkers, L. (2011). Ethics, technology, and engineering: An introduction. Wiley-Blackwell."},{"key":"9635_CR54","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJCH Watkins","year":"1992","unstructured":"Watkins, C. J. C. H., & Dayan, P. (1992). Technical note q-learning. Machine Learning, 8, 279\u2013292. https:\/\/doi.org\/10.1007\/BF00992698.","journal-title":"Machine Learning"},{"key":"9635_CR55","unstructured":"Wu, Y. H., & Lin, S. D. (2017). A low-cost ethics shaping approach for designing reinforcement learning agents. arXiv."},{"issue":"4","key":"9635_CR56","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1007\/s10676-016-9409-x","volume":"18","author":"A Wynsberghe","year":"2016","unstructured":"Wynsberghe, A. (2016). Service robots, care ethics, and design. Ethics and Information Technology, 18(4), 311\u2013321. https:\/\/doi.org\/10.1007\/s10676-016-9409-x.","journal-title":"Ethics and Information Technology"},{"key":"9635_CR37","doi-asserted-by":"crossref","unstructured":"Yu, H., Shen, Z., Miao, C., Leung, C., Lesser, V. R., & Yang, Q. (2018). Building ethics into artificial intelligence. In: IJCAI, pp. 5527\u20135533.","DOI":"10.24963\/ijcai.2018\/779"},{"issue":"2","key":"9635_CR57","first-page":"199","volume":"24","author":"MJ Zimmerman","year":"1987","unstructured":"Zimmerman, M. J. (1987). Remote obligation. American Philosophical Quarterly, 24(2), 199\u2013205.","journal-title":"American Philosophical Quarterly"}],"container-title":["Ethics and Information Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-022-09635-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10676-022-09635-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10676-022-09635-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,22]],"date-time":"2022-03-22T19:18:43Z","timestamp":1647976723000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10676-022-09635-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,24]]},"references-count":55,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3]]}},"alternative-id":["9635"],"URL":"https:\/\/doi.org\/10.1007\/s10676-022-09635-0","relation":{},"ISSN":["1388-1957","1572-8439"],"issn-type":[{"value":"1388-1957","type":"print"},{"value":"1572-8439","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,24]]},"assertion":[{"value":"8 January 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 January 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"9"}}