{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T15:22:39Z","timestamp":1775056959579,"version":"3.50.1"},"reference-count":87,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T00:00:00Z","timestamp":1598918400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T00:00:00Z","timestamp":1598918400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010663","name":"H2020 European Research Council","doi-asserted-by":"publisher","award":["788321"],"award-info":[{"award-number":["788321"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Minds &amp; Machines"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Organizations such as the EU High-Level Expert Group on AI and the IEEE have recently formulated ethical principles and (moral) values that should be adhered to in the design and deployment of artificial intelligence (AI). These include respect for autonomy, non-maleficence, fairness, transparency, explainability, and accountability. But how can we ensure and verify that an AI system actually respects these values? To help answer this question, I propose an account for determining when an AI system can be said to embody certain values. This account understands embodied values as the result of design activities intended to embed those values in such systems. AI systems are here understood as a special kind of sociotechnical system that, like traditional sociotechnical systems, are composed of technical artifacts, human agents, and institutions but\u2014in addition\u2014contain artificial agents and certain technical norms that regulate interactions between artificial agents and other elements of the system. The specific challenges and opportunities of embedding values in AI systems are discussed, and some lessons for better embedding values in AI systems are drawn.<\/jats:p>","DOI":"10.1007\/s11023-020-09537-4","type":"journal-article","created":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T18:45:46Z","timestamp":1598985946000},"page":"385-409","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":185,"title":["Embedding Values in Artificial Intelligence (AI) Systems"],"prefix":"10.1007","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9553-5651","authenticated-orcid":false,"given":"Ibo","family":"van de Poel","sequence":"first","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,9,1]]},"reference":[{"key":"9537_CR1","first-page":"205","volume-title":"Shaping technology\/building society: Studies in sociotechnical change","author":"M Akrich","year":"1992","unstructured":"Akrich, M. (1992). The Description of Technical Objects. In W. Bijker & J. Law (Eds.), Shaping technology\/building society: Studies in sociotechnical change (pp. 205\u2013224). Cambridge: MIT Press."},{"key":"9537_CR2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-37756-3","volume-title":"Coordination, organizations, institutions, and norms in agent systems VIII: 14th International Workshop, COIN 2012 Lecture notes in artificial intelligence","author":"H Aldewereld","year":"2013","unstructured":"Aldewereld, H., & Sichman, J. S. (2013). Coordination, organizations, institutions, and norms in agent systems VIII: 14th International Workshop, COIN 2012 Lecture notes in artificial intelligence (Vol. 7756). New York: Springer."},{"issue":"3","key":"9537_CR3","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1007\/s10676-006-0004-4","volume":"7","author":"C Allen","year":"2005","unstructured":"Allen, C., Smit, I., & Wallach, W. (2005). Artificial morality: Top-down, bottom-up, and hybrid approaches. Ethics and Information Technology, 7(3), 149\u2013155. https:\/\/doi.org\/10.1007\/s10676-006-0004-4.","journal-title":"Ethics and Information Technology"},{"key":"9537_CR4","volume-title":"Value in ethics and economics","author":"E Anderson","year":"1993","unstructured":"Anderson, E. (1993). Value in ethics and economics. Cambridge: Harvard University Press."},{"key":"9537_CR5","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511978036","volume-title":"Machine ethics","author":"M Anderson","year":"2011","unstructured":"Anderson, M., & Anderson, S. L. (2011). Machine ethics. New York: Cambridge University Press."},{"key":"9537_CR6","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1016\/B978-0-444-51667-1.50026-4","volume-title":"Philosophy of Technology and Engineering Sciences","author":"JM Bauer","year":"2009","unstructured":"Bauer, J. M., & Herder, P. M. (2009). Designing Socio-Technical Systems. In A. Meijers (Ed.), Philosophy of Technology and Engineering Sciences (pp. 601\u2013630). Amsterdam: Elsevier."},{"issue":"1","key":"9537_CR7","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.intcom.2010.07.003","volume":"23","author":"G Baxter","year":"2011","unstructured":"Baxter, G., & Sommerville, I. (2011). Socio-technical systems: From design methods to systems engineering. Interacting with Computers, 23(1), 4\u201317. https:\/\/doi.org\/10.1016\/j.intcom.2010.07.003.","journal-title":"Interacting with Computers"},{"key":"9537_CR8","volume-title":"Outsiders. Studies in the Sociology of Deviance","author":"HS Becker","year":"1963","unstructured":"Becker, H. S. (1963). Outsiders. Studies in the Sociology of Deviance. New York: The Free Press of Glencoe."},{"issue":"2","key":"9537_CR9","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.sheji.2016.09.001","volume":"2","author":"KJ Behymer","year":"2016","unstructured":"Behymer, K. J., & Flach, J. M. (2016). From Autonomous Systems to Sociotechnical Systems: Designing Effective Collaborations. She Ji: The Journal of Design, Economics, and Innovation, 2(2), 105\u2013114. https:\/\/doi.org\/10.1016\/j.sheji.2016.09.001.","journal-title":"She Ji: The Journal of Design, Economics, and Innovation"},{"key":"9537_CR10","volume-title":"The grammar of society: the nature and dynamics of social norms","author":"C Bicchieri","year":"2006","unstructured":"Bicchieri, C. (2006). The grammar of society: the nature and dynamics of social norms. New York: Cambridge University Press."},{"key":"9537_CR11","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-60648-4","volume-title":"Towards a code of ethics for artificial intelligence research","author":"P Boddington","year":"2017","unstructured":"Boddington, P. (2017). Towards a code of ethics for artificial intelligence research. New York: Springer."},{"key":"9537_CR12","doi-asserted-by":"crossref","unstructured":"Boissier, O. (2006). Coordination, organizations, institutions, and norms in multi-agent systems: AAMAS 2005 International Workshops on Agents, Norms and Institutions for Regulated Multi-Agent Systems, ANIREM 2005, and From Organizations to Organization-Oriented Programming in Multi-Agent Systems, OOOP 2005 (Lecture notes in computer science,, Vol. 3913). Berlin; New York: Springer.","DOI":"10.1007\/11775331"},{"issue":"2","key":"9537_CR13","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s11948-017-0006-0","volume":"25","author":"J Borenstein","year":"2019","unstructured":"Borenstein, J., Herkert, J. R., & Miller, K. W. (2019). Self-driving cars and engineering ethics: The need for a system level analysis. Science and Engineering Ethics, 25(2), 383\u2013398. https:\/\/doi.org\/10.1007\/s11948-017-0006-0.","journal-title":"Science and Engineering Ethics"},{"key":"9537_CR14","first-page":"97","volume-title":"International handbook of internet research","author":"GC Bowker","year":"2010","unstructured":"Bowker, G. C., Baker, K., Millerand, F., & Ribes, D. (2010). Toward information infrastructure studies: Ways of knowing in a networked environment. In J. Hunsinger, L. Klastrup, & M. Allen (Eds.), International handbook of internet research (pp. 97\u2013117). Dordrecht: Springer."},{"key":"9537_CR15","volume-title":"Handbook of value: perspectives from economics, neuroscience, philosophy, psychology and sociology","author":"T Brosch","year":"2016","unstructured":"Brosch, T., Sander, D., Cl\u00e9ment, F., Deonna, J. A., Fehr, E., & Vuilleumier, P. (2016). Handbook of value: perspectives from economics, neuroscience, philosophy, psychology and sociology. Oxford: Oxford University Press."},{"issue":"5","key":"9537_CR16","doi-asserted-by":"publisher","first-page":"981","DOI":"10.1109\/TSMCA.2009.2025452","volume":"39","author":"H Bruijn","year":"2009","unstructured":"Bruijn, H., & Herder, P. M. (2009). System and actor perspectives on sociotechnical systems. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 39(5), 981\u2013992. https:\/\/doi.org\/10.1109\/TSMCA.2009.2025452.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans"},{"key":"9537_CR17","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1017\/CBO9780511625725.011","volume-title":"Modern political economy: Old topics, new directions (Political economy of institutions and decisions)","author":"RL Calvert","year":"1995","unstructured":"Calvert, R. L. (1995). The rational choice theory of social institutions: cooperation, coordination, and communication. In E. A. Hanushek & J. S. Banks (Eds.), Modern political economy: Old topics, new directions (Political economy of institutions and decisions) (pp. 216\u2013268). Cambridge: Cambridge University Press."},{"issue":"3","key":"9537_CR18","doi-asserted-by":"publisher","first-page":"562","DOI":"10.1109\/JPROC.2018.2865996","volume":"107","author":"S Cave","year":"2019","unstructured":"Cave, S., Nyrup, R., Vold, K., & Weller, A. (2019). Motivations and risks of machine ethics. Proceedings of the IEEE, 107(3), 562\u2013574. https:\/\/doi.org\/10.1109\/JPROC.2018.2865996.","journal-title":"Proceedings of the IEEE"},{"issue":"2","key":"9537_CR19","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1007\/s11948-019-00151-x","volume":"26","author":"J-A Cervantes","year":"2020","unstructured":"Cervantes, J.-A., L\u00f3pez, S., Rodr\u00edguez, L.-F., Cervantes, S., Cervantes, F., & Ramos, F. (2020). Artificial moral agents: A survey of the current status. Science and Engineering Ethics, 26(2), 501\u2013532. https:\/\/doi.org\/10.1007\/s11948-019-00151-x.","journal-title":"Science and Engineering Ethics"},{"key":"9537_CR20","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/12549.001.0001","volume-title":"AI ethics","author":"M Coeckelbergh","year":"2020","unstructured":"Coeckelbergh, M. (2020). AI ethics. Cambridge: The MIT Press."},{"issue":"3","key":"9537_CR21","doi-asserted-by":"publisher","first-page":"582","DOI":"10.2307\/2082975","volume":"89","author":"SES Crawford","year":"1995","unstructured":"Crawford, S. E. S., & Ostrom, E. (1995). A grammar of institutions. American Political Science Review, 89(3), 582\u2013600. https:\/\/doi.org\/10.2307\/2082975.","journal-title":"American Political Science Review"},{"key":"9537_CR22","doi-asserted-by":"crossref","DOI":"10.1007\/978-94-007-4933-7","volume-title":"Agent-based modelling of socio-technical systems","author":"KH Dam","year":"2013","unstructured":"Dam, K. H., Nikolic, I., & Lukszo, Z. (2013). Agent-based modelling of socio-technical systems (Vol. 9). New York: Springer."},{"key":"9537_CR23","volume-title":"The intentional stance","author":"DC Dennett","year":"1987","unstructured":"Dennett, D. C. (1987). The intentional stance. Cambridge: MIT Press."},{"key":"9537_CR24","doi-asserted-by":"crossref","unstructured":"Dybalova, D., Testerink, B., Dastani, M., & Logan, B. A Framework for Programming Norm-Aware Multi-agent Systems. In Cham, 2014 (pp. 364\u2013380, Coordination, Organizations, Institutions, and Norms in Agent Systems IX): Springer International Publishing","DOI":"10.1007\/978-3-319-14220-3_20"},{"key":"9537_CR25","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1017\/CBO9780511498725.017","volume-title":"Information technology and moral philosophy","author":"M Flanagan","year":"2008","unstructured":"Flanagan, M., Howe, D. C., & Nissenbaum, H. (2008). Embodying values in technology. Theory and practise. In J. Van den Hoven & J. Weckert (Eds.), Information technology and moral philosophy (pp. 322\u2013353). Cambridge: Cambridge University Press."},{"issue":"3","key":"9537_CR26","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1023\/B:MIND.0000035461.63578.9d","volume":"14","author":"L Floridi","year":"2004","unstructured":"Floridi, L., & Sanders, J. W. (2004). On the morality of artificial agents. Minds and Machines, 14(3), 349\u2013379.","journal-title":"Minds and Machines"},{"key":"9537_CR27","volume-title":"Persuasive technology: using computers to change what we think and do (The Morgan Kaufmann series in interactive technologies)","author":"BJ Fogg","year":"2003","unstructured":"Fogg, B. J. (2003). Persuasive technology: using computers to change what we think and do (The Morgan Kaufmann series in interactive technologies). Amsterdam: Morgan Kaufmann Publishers."},{"key":"9537_CR28","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/978-94-007-6970-0_8","volume-title":"Handbook of ethics, values, and technological design: sources, theory, values and application domains","author":"M Franssen","year":"2015","unstructured":"Franssen, M. (2015). Design for values and operator roles in sociotechnical systems. In J. van den Hoven, P. E. Vermaas, & I. van de Poel (Eds.), Handbook of ethics, values, and technological design: sources, theory, values and application domains (pp. 117\u2013149). Dordrecht: Springer."},{"key":"9537_CR29","doi-asserted-by":"crossref","unstructured":"Franssen, M. Modelling Systems in Technology as Instrumental Systems. In Berlin, Heidelberg, 2014 (pp. 543\u2013562, Model-Based Reasoning in Science and Technology): Springer Berlin Heidelberg","DOI":"10.1007\/978-3-642-37428-9_30"},{"issue":"6","key":"9537_CR30","doi-asserted-by":"publisher","first-page":"897","DOI":"10.1016\/j.respol.2004.01.015","volume":"33","author":"FW Geels","year":"2004","unstructured":"Geels, F. W. (2004). From sectoral systems of innovation to socio-technical systems: Insights about dynamics and change from sociology and institutional theory. Research Policy, 33(6), 897\u2013920. https:\/\/doi.org\/10.1016\/j.respol.2004.01.015.","journal-title":"Research Policy"},{"key":"9537_CR31","doi-asserted-by":"crossref","unstructured":"Ghorbani, A., Aldewereld, H., Dignum, V., & Noriega, P. Shared Strategies in Artificial Agent Societies. In Berlin, Heidelberg, 2013 (pp. 71\u201386, Coordination, Organizations, Institutions, and Norms in Agent Systems VIII): Springer Berlin Heidelberg","DOI":"10.1007\/978-3-642-37756-3_5"},{"issue":"2","key":"9537_CR32","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s10676-008-9163-9","volume":"10","author":"FS Grodzinsky","year":"2008","unstructured":"Grodzinsky, F. S., Miller, K. W., & Wolf, M. J. (2008). The ethics of designing artificial agents. Ethics and Information Technology, 10(2), 115\u2013121. https:\/\/doi.org\/10.1007\/s10676-008-9163-9.","journal-title":"Ethics and Information Technology"},{"key":"9537_CR33","volume-title":"Ethics guidelines for trustworthy AI","author":"High-Level Expert Group on AI","year":"2019","unstructured":"High-Level Expert Group on AI. (2019). Ethics guidelines for trustworthy AI. Brussels: EC."},{"key":"9537_CR34","doi-asserted-by":"crossref","DOI":"10.1093\/oxfordhb\/9780199959303.001.0001","volume-title":"The Oxford handbook of value theory","author":"I Hirose","year":"2015","unstructured":"Hirose, I., & Olson, J. (2015). The Oxford handbook of value theory. New York: Oxford University Press."},{"issue":"6","key":"9537_CR35","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1080\/03066150.2016.1239617","volume":"43","author":"P Ho","year":"2016","unstructured":"Ho, P. (2016). Empty institutions, non-credibility and pastoralism: China\u2019s grazing ban, mining and ethnicity. The Journal of Peasant Studies, 43(6), 1145\u20131176. https:\/\/doi.org\/10.1080\/03066150.2016.1239617.","journal-title":"The Journal of Peasant Studies"},{"issue":"2","key":"9537_CR36","doi-asserted-by":"publisher","first-page":"6","DOI":"10.18564\/jasss.1750","volume":"14","author":"CD Hollander","year":"2011","unstructured":"Hollander, C. D., & Wu, A. S. (2011). The current state of normative agent-based systems. Journal of Artificial Societies and Social Simulation, 14(2), 6. https:\/\/doi.org\/10.18564\/jasss.1750.","journal-title":"Journal of Artificial Societies and Social Simulation"},{"key":"9537_CR37","doi-asserted-by":"crossref","DOI":"10.1007\/978-90-481-3900-2","volume-title":"Technical functions: on the use and design of artifactartifacts (Philosophy of engineering and technology","author":"W Houkes","year":"2010","unstructured":"Houkes, W., & Vermaas, P. E. (2010). Technical functions: on the use and design of artifactartifacts (Philosophy of engineering and technology (Vol. 1). Dordrecht: Springer."},{"key":"9537_CR38","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/S0142-694X(01)00040-0","volume":"23","author":"W Houkes","year":"2002","unstructured":"Houkes, W., Vermaas, P. E., Dorst, K., & de Vries, M. J. (2002). Design and use as plans. An action-theoretical account. Design Studies, 23, 303\u2013320.","journal-title":"Design Studies"},{"key":"9537_CR39","unstructured":"IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems (2019). Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems. (1st ed.): IEEE."},{"key":"9537_CR40","doi-asserted-by":"crossref","DOI":"10.1515\/9781438442877","volume-title":"Experimental phenomenology: multistabilities","author":"D Ihde","year":"2012","unstructured":"Ihde, D. (2012). Experimental phenomenology: multistabilities (2nd ed.). Albany: State University of New York Press.","edition":"2"},{"issue":"3","key":"9537_CR41","doi-asserted-by":"crossref","first-page":"420","DOI":"10.5840\/monist200992324","volume":"92","author":"C Illies","year":"2009","unstructured":"Illies, C., & Meijers, A. (2009). Artifact artifacts without agency. The Monist, 92(3), 420\u2013440.","journal-title":"The Monist"},{"key":"9537_CR42","unstructured":"Jacobson, D. (2011). Fitting Attitude Theories of Value. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Spring 2011 Edition)."},{"key":"9537_CR43","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1007\/s10676-006-9111-5","volume":"8","author":"DG Johnson","year":"2006","unstructured":"Johnson, D. G. (2006). Computer systems: Moral entities but not moral agents. Ethics and Information Technology, 8, 195\u2013205.","journal-title":"Ethics and Information Technology"},{"issue":"2","key":"9537_CR44","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/s10676-008-9174-6","volume":"10","author":"DG Johnson","year":"2008","unstructured":"Johnson, D. G., & Miller, K. W. (2008). Un-making artificial moral agents. Ethics and Information Technology, 10(2), 123\u2013133. https:\/\/doi.org\/10.1007\/s10676-008-9174-6.","journal-title":"Ethics and Information Technology"},{"issue":"1","key":"9537_CR45","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10462-012-9387-2","volume":"39","author":"AJI Jones","year":"2013","unstructured":"Jones, A. J. I., Artikis, A., & Pitt, J. (2013). The design of intelligent socio-technical systems. Artificial Intelligence Review, 39(1), 5\u201320. https:\/\/doi.org\/10.1007\/s10462-012-9387-2.","journal-title":"Artificial Intelligence Review"},{"key":"9537_CR46","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-020-00401-y","author":"M Klenk","year":"2020","unstructured":"Klenk, M. (2020). How do technological artifact artifacts embody moral values? Philosophy & Technology. https:\/\/doi.org\/10.1007\/s13347-020-00401-y.","journal-title":"Philosophy & Technology"},{"issue":"1","key":"9537_CR47","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1093\/cje\/bep019","volume":"34","author":"P Kroes","year":"2010","unstructured":"Kroes, P. (2010). Engineering and the dual nature of technical artifactartifacts. Cambridge Journal of Economics, 34(1), 51\u201362. https:\/\/doi.org\/10.1093\/cje\/bep019.","journal-title":"Cambridge Journal of Economics"},{"issue":"6","key":"9537_CR48","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1002\/sres.703","volume":"23","author":"P Kroes","year":"2006","unstructured":"Kroes, P., Franssen, M., Van de Poel, I., & Ottens, M. (2006). Treating socio-technical systems as engineering systems: some conceptual problems. Systems Research and Behavioral Science, 23(6), 803\u2013814. https:\/\/doi.org\/10.1002\/sres.703.","journal-title":"Systems Research and Behavioral Science"},{"issue":"1","key":"9537_CR49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.shpsa.2005.12.001","volume":"37","author":"P Kroes","year":"2006","unstructured":"Kroes, P., & Meijers, A. (2006). The dual nature of technical artifactartifacts. Studies In History and Philosophy of Science Part A, 37(1), 1\u20134.","journal-title":"Studies In History and Philosophy of Science Part A"},{"key":"9537_CR50","volume-title":"The moral status of technical artifactartifacts","year":"2014","unstructured":"Kroes, P., & Verbeek, P.-P. (Eds.). (2014). The moral status of technical artifactartifacts. Dordrecht: Springer."},{"key":"9537_CR51","first-page":"225","volume-title":"Shaping Technology\/Building Society; Studies in Sociotechnical change","author":"B Latour","year":"1992","unstructured":"Latour, B. (1992). Where are the missing masses? In W. Bijker & J. Law (Eds.), Shaping Technology\/Building Society; Studies in Sociotechnical change (pp. 225\u2013258). Cambridge: MIT Press."},{"key":"9537_CR52","volume-title":"We have never been modern","author":"B Latour","year":"1993","unstructured":"Latour, B. (1993). We have never been modern. New York: Harvester Wheatsheaf."},{"issue":"2","key":"9537_CR53","doi-asserted-by":"publisher","first-page":"193","DOI":"10.5235\/17579961.6.2.193","volume":"6","author":"R Leenes","year":"2014","unstructured":"Leenes, R., & Lucivero, F. (2014). Laws on Robots, Laws by Robots, Laws in Robots: Regulating Robot Behaviour by Design. Law, Innovation and Technology, 6(2), 193\u2013220. https:\/\/doi.org\/10.5235\/17579961.6.2.193.","journal-title":"Law, Innovation and Technology"},{"key":"9537_CR54","volume-title":"Code and other laws of cyberspace","author":"L Lessig","year":"1999","unstructured":"Lessig, L. (1999). Code and other laws of cyberspace. New York: Basic Books."},{"key":"9537_CR55","doi-asserted-by":"publisher","first-page":"684587","DOI":"10.1155\/2014\/684587","volume":"2014","author":"MA Mahmoud","year":"2014","unstructured":"Mahmoud, M. A., Ahmad, M. S., Mohd Yusoff, M. Z., & Mustapha, A. (2014). A review of norms and normative multiagent systems. The Scientific World Journal, 2014, 684587. https:\/\/doi.org\/10.1155\/2014\/684587.","journal-title":"The Scientific World Journal"},{"key":"9537_CR56","doi-asserted-by":"publisher","DOI":"10.1177\/0162243919900965","author":"B Miller","year":"2020","unstructured":"Miller, B. (2020). Is Technology Value-Neutral? Science, Technology and Human Values. https:\/\/doi.org\/10.1177\/0162243919900965.","journal-title":"Science, Technology and Human Values"},{"issue":"4","key":"9537_CR57","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/MIS.2006.80","volume":"21","author":"JH Moor","year":"2006","unstructured":"Moor, J. H. (2006). The nature, importance, and difficulty of machine ethics. IEEE Intelligent Systems, 21(4), 18\u201321. https:\/\/doi.org\/10.1109\/MIS.2006.80.","journal-title":"IEEE Intelligent Systems"},{"key":"9537_CR58","unstructured":"M\u00fcller, V. C. (2020). Ethics of Artificial Intelligence and Robotics. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Fall 2020 Edition)."},{"key":"9537_CR59","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1007\/978-94-007-5243-6_14","volume-title":"Norms in technology","author":"PJ Nickel","year":"2013","unstructured":"Nickel, P. J. (2013). Trust in Technological Systems. In M. J. de Vries, S. O. Hansson, & A. W. M. Meijers (Eds.), Norms in technology (pp. 223\u2013237). Dordrecht: Springer."},{"key":"9537_CR60","volume-title":"The design of everyday things","author":"DA Norman","year":"2000","unstructured":"Norman, D. A. (2000). The design of everyday things. Cambridg: MIT Press."},{"key":"9537_CR61","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511808678","volume-title":"Institutions, institutional change and economic performance","author":"DC North","year":"1990","unstructured":"North, D. C. (1990). Institutions, institutional change and economic performance. Cambridge: Cambridge University Press."},{"key":"9537_CR62","volume-title":"Understanding institutional diversity (Princeton paperbacks)","author":"E Ostrom","year":"2005","unstructured":"Ostrom, E. (2005). Understanding institutional diversity (Princeton paperbacks). Princeton: Princeton University Press."},{"key":"9537_CR63","doi-asserted-by":"crossref","DOI":"10.3998\/mpub.9739","volume-title":"Rules, games, and common-pool resources","author":"E Ostrom","year":"1994","unstructured":"Ostrom, E., Gardner, R., & Walker, J. (1994). Rules, games, and common-pool resources. Ann Arbor: University of Michigan Press."},{"key":"9537_CR64","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1504\/IJCIS.2006.009433","volume":"2","author":"M Ottens","year":"2006","unstructured":"Ottens, M., Franssen, M., Kroes, P., & Van de Poel, I. (2006). Modeling engineering systems as socio-technical systems. International Journal of Critical Infrastructures, 2, 133\u2013145.","journal-title":"International Journal of Critical Infrastructures"},{"key":"9537_CR65","doi-asserted-by":"crossref","unstructured":"Panagiotidi, S., V\u00e1zquez-Salceda, J., & Dignum, F. Reasoning over Norm Compliance via Planning. In Berlin, Heidelberg, 2013 (pp. 35\u201352, Coordination, Organizations, Institutions, and Norms in Agent Systems VIII): Springer Berlin Heidelberg","DOI":"10.1007\/978-3-642-37756-3_3"},{"key":"9537_CR66","volume-title":"Sociotechnical systems : a sourcebook","author":"WA Pasmore","year":"1978","unstructured":"Pasmore, W. A., & Sherwood, J. J. (1978). Sociotechnical systems : a sourcebook. La Jolla: University Associates."},{"issue":"3","key":"9537_CR67","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1007\/s11948-010-9241-3","volume":"17","author":"M Peterson","year":"2011","unstructured":"Peterson, M., & Spahn, A. (2011). Can technological artifactArtifacts be moral agents? Science and Engineering Ethics, 17(3), 411\u2013424.","journal-title":"Science and Engineering Ethics"},{"key":"9537_CR68","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/978-94-007-7914-3_6","volume-title":"The moral status of technical artifactartifacts","author":"JC Pitt","year":"2014","unstructured":"Pitt, J. C. (2014). Guns don\u2019t kill, people kill\u201d; values in and\/or around technologies. In P. Kroes & P.-P. Verbeek (Eds.), The moral status of technical artifactartifacts (pp. 89\u2013101). Dordrecht: Springer."},{"key":"9537_CR69","volume-title":"Engaging reason. On the theory of value and action","author":"J Raz","year":"1999","unstructured":"Raz, J. (1999). Engaging reason. On the theory of value and action. Oxford: Oxford University Press."},{"key":"9537_CR70","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2018.00015","author":"F SantonideSio","year":"2018","unstructured":"SantonideSio, F., & van den Hoven, J. (2018). Meaningful human control over autonomous systems: A philosophical account. Frontiers in Robotics and AI. https:\/\/doi.org\/10.3389\/frobt.2018.00015.","journal-title":"Frontiers in Robotics and AI"},{"key":"9537_CR71","volume-title":"What we owe to each other","author":"TM Scanlon","year":"1998","unstructured":"Scanlon, T. M. (1998). What we owe to each other. Cambridge: Harvard University Press."},{"key":"9537_CR72","volume-title":"Minds, brains, and science (The 1984 Reith lectures)","author":"JR Searle","year":"1984","unstructured":"Searle, J. R. (1984). Minds, brains, and science (The 1984 Reith lectures). Cambridge: Harvard University Press."},{"issue":"1","key":"9537_CR73","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2542182.2542203","volume":"5","author":"MP Singh","year":"2014","unstructured":"Singh, M. P. (2014). Norms as a basis for governing sociotechnical systems. ACM Trans. Intell. Syst. Technol., 5(1), 1\u201323. https:\/\/doi.org\/10.1145\/2542182.2542203.","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"9537_CR74","volume-title":"Ethics and language","author":"CL Stevenson","year":"1944","unstructured":"Stevenson, C. L. (1944). Ethics and language. New Haven: Yale University Press."},{"issue":"12","key":"9537_CR75","doi-asserted-by":"crossref","first-page":"23","DOI":"10.29173\/irie136","volume":"6","author":"JP Sullins","year":"2006","unstructured":"Sullins, J. P. (2006). When is a robot a moral agent. International Review of Information Ethics, 6(12), 23\u201330.","journal-title":"International Review of Information Ethics"},{"key":"9537_CR76","volume-title":"Nudge: improving decisions about health, wealth, and happiness (Rev. and expanded ed.","author":"RH Thaler","year":"2009","unstructured":"Thaler, R. H., & Sunstein, C. R. (2009). Nudge: improving decisions about health, wealth, and happiness (Rev. and expanded ed.. New York: Penguin Books."},{"key":"9537_CR77","volume-title":"The emergence of norms (Clarendon library of logic and philosophy)","author":"E Ullmann-Margalit","year":"1977","unstructured":"Ullmann-Margalit, E. (1977). The emergence of norms (Clarendon library of logic and philosophy). Oxford Eng: Clarendon Press."},{"key":"9537_CR78","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1007\/978-94-007-7914-3_7","volume-title":"The moral status of technical artifacts","author":"I Van de Poel","year":"2014","unstructured":"Van de Poel, I., & Kroes, P. (2014). Can technology embody values? In P. Kroes & P.-P. Verbeek (Eds.), The moral status of technical artifacts (pp. 103\u2013124). Dordrecht: Springer."},{"issue":"3","key":"9537_CR79","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1007\/s11948-018-0030-8","volume":"25","author":"A van Wynsberghe","year":"2019","unstructured":"van Wynsberghe, A., & Robbins, S. (2019). Critiquing the reasons for making artificial moral agents. Science and Engineering Ethics, 25(3), 719\u2013735. https:\/\/doi.org\/10.1007\/s11948-018-0030-8.","journal-title":"Science and Engineering Ethics"},{"key":"9537_CR80","doi-asserted-by":"crossref","unstructured":"Vanderelst, D., & Winfield, A. (2018). The Dark Side of Ethical Robots. Paper presented at the Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, New Orleans, LA, USA,","DOI":"10.1145\/3278721.3278726"},{"key":"9537_CR81","doi-asserted-by":"crossref","DOI":"10.7208\/chicago\/9780226852904.001.0001","volume-title":"Moralizing technology: understanding and designing the morality of things","author":"P-P Verbeek","year":"2011","unstructured":"Verbeek, P.-P. (2011). Moralizing technology: understanding and designing the morality of things. Chicago, London: The University of Chicago Press."},{"issue":"1","key":"9537_CR82","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.shpsa.2005.12.002","volume":"37","author":"PE Vermaas","year":"2006","unstructured":"Vermaas, P. E., & Houkes, W. (2006). Technical functions: a drawbridge between the intentional and structural natures of technical artifacts. Studies In History and Philosophy of Science Part A, 37(1), 5\u201318.","journal-title":"Studies In History and Philosophy of Science Part A"},{"key":"9537_CR83","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195374049.001.0001","volume-title":"Moral machines: teaching robots right from wrong","author":"W Wallach","year":"2009","unstructured":"Wallach, W., & Allen, C. (2009). Moral machines: teaching robots right from wrong. Oxford: Oxford University Press."},{"issue":"3","key":"9537_CR84","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1109\/JPROC.2019.2900622","volume":"107","author":"AF Winfield","year":"2019","unstructured":"Winfield, A. F. (2019). Machine ethics: The design and governance of ethical AI and autonomous systems [scanning the issue]. Proceedings of the IEEE, 107(3), 509\u2013517.","journal-title":"Proceedings of the IEEE"},{"key":"9537_CR85","volume-title":"Autonomous Technology. Technics-out-of-Control as a Theme in Political Thought","author":"L Winner","year":"1977","unstructured":"Winner, L. (1977). Autonomous Technology. Technics-out-of-Control as a Theme in Political Thought. Cambridge: MIT Press."},{"key":"9537_CR86","first-page":"121","volume":"109","author":"L Winner","year":"1980","unstructured":"Winner, L. (1980). Do artifacts have politics? Daedalus, 109, 121\u2013136.","journal-title":"Daedalus"},{"key":"9537_CR87","volume-title":"The Oxford handbook of value theory","author":"MJ Zimmerman","year":"2015","unstructured":"Zimmerman, M. J. (2015). Value and Normativity. In I. Hirose & J. Olson (Eds.), The Oxford handbook of value theory. Oxford: Oxford University Press."}],"container-title":["Minds and Machines"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-020-09537-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11023-020-09537-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11023-020-09537-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,13]],"date-time":"2024-08-13T00:21:12Z","timestamp":1723508472000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11023-020-09537-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9]]},"references-count":87,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["9537"],"URL":"https:\/\/doi.org\/10.1007\/s11023-020-09537-4","relation":{},"ISSN":["0924-6495","1572-8641"],"issn-type":[{"value":"0924-6495","type":"print"},{"value":"1572-8641","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9]]},"assertion":[{"value":"13 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 September 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}