{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T15:43:07Z","timestamp":1781797387226,"version":"3.54.5"},"reference-count":74,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2021,5,15]],"date-time":"2021-05-15T00:00:00Z","timestamp":1621036800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,15]],"date-time":"2021-05-15T00:00:00Z","timestamp":1621036800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100006692","name":"Universit\u00e0 degli Studi di Torino","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006692","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J of Soc Robotics"],"published-print":{"date-parts":[[2022,3]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>One of the primary, if not most critical, difficulties in the design and implementation of autonomous systems is the black-boxed nature of the decision-making structures and logical pathways. How human values are embodied and actualised in situ may ultimately prove to be harmful if not outright recalcitrant. For this reason, the values of stakeholders become of particular significance given the risks posed by opaque structures of intelligent agents. This paper explores how decision matrix algorithms, via the<jats:italic>belief-desire-intention<\/jats:italic>model for autonomous vehicles, can be designed to minimize the risks of opaque architectures. Primarily through an explicit orientation towards designing<jats:italic>for<\/jats:italic>the values of explainability and verifiability. In doing so, this research adopts the Value Sensitive Design (VSD) approach as a principled framework for the incorporation of such values within design. VSD is recognized as a potential starting point that offers a systematic way for engineering teams to formally incorporate existing technical solutions within ethical design, while simultaneously remaining pliable to emerging issues and needs. It is concluded that the VSD methodology offers at least a strong enough foundation from which designers can begin to anticipate design needs and formulate salient design flows that can be adapted to the changing ethical landscapes required for utilisation in autonomous vehicles.<\/jats:p>","DOI":"10.1007\/s12369-021-00790-w","type":"journal-article","created":{"date-parts":[[2021,5,15]],"date-time":"2021-05-15T18:02:51Z","timestamp":1621101771000},"page":"313-322","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":55,"title":["Designing AI for Explainability and Verifiability: A Value Sensitive Design Approach to Avoid Artificial Stupidity in Autonomous Vehicles"],"prefix":"10.1007","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2594-6313","authenticated-orcid":false,"given":"Steven","family":"Umbrello","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9637-1161","authenticated-orcid":false,"given":"Roman V.","family":"Yampolskiy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,5,15]]},"reference":[{"key":"790_CR1","doi-asserted-by":"publisher","first-page":"15","DOI":"10.3389\/frobt.2018.00015","volume":"5","author":"F Santoni de Sio","year":"2018","unstructured":"Santoni de Sio F, van den Hoven J (2018) Meaningful human control over autonomous systems: a philosophical account. Front Robot AI. 5:15","journal-title":"Front Robot AI."},{"key":"790_CR2","unstructured":"Friedman B (1997) Human values and the design of computer technology. In: Friedman B (ed) CSLI Publications"},{"key":"790_CR3","doi-asserted-by":"publisher","unstructured":"van den Hoven J, Manders-Huits N (2009) Value-sensitive design. In: A companion to the philosophy of technology, Wiley-Blackwell, pp 477\u2013480. https:\/\/doi.org\/10.1002\/9781444310795.ch86","DOI":"10.1002\/9781444310795.ch86"},{"key":"790_CR4","unstructured":"Friedman B, Kahn Jr PH (2003) Human values, ethics, and design. Hum-Comput Interact Handb:1177\u20131201"},{"issue":"9","key":"790_CR5","doi-asserted-by":"publisher","first-page":"93001","DOI":"10.1088\/1402-4896\/aa7ca8","volume":"92","author":"RV Yampolskiy","year":"2017","unstructured":"Yampolskiy RV (2017) What are the ultimate limits to computational techniques: verifier theory and unverifiability. Phys Scr 92(9):93001","journal-title":"Phys Scr"},{"issue":"2","key":"790_CR6","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1080\/23299460.2018.1457401","volume":"5","author":"S Umbrello","year":"2018","unstructured":"Umbrello S (2018) The moral psychology of value sensitive design: the methodological issues of moral intuitions for responsible innovation. J Respons Innov 5(2):186\u2013200. https:\/\/doi.org\/10.1080\/23299460.2018.1457401","journal-title":"J Respons Innov"},{"key":"790_CR7","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/978-94-007-7844-3_4","volume-title":"Early engagement and new technologies: opening up the laboratory","author":"B Friedman","year":"2013","unstructured":"Friedman B, Kahn PH, Borning A, Huldtgren A (2013) Value sensitive design and information systems. In: Doorn N, Schuurbiers D, van de Poel I, Gorman ME (eds) Early engagement and new technologies: opening up the laboratory. Springer, Dordrecht, pp 55\u201395. https:\/\/doi.org\/10.1007\/978-94-007-7844-3_4"},{"issue":"2","key":"790_CR8","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1561\/1100000015","volume":"11","author":"B Friedman","year":"2017","unstructured":"Friedman B, Hendry DG, Borning A (2017) A survey of value sensitive design methods. Found Trends Hum-Comput Interact 11(2):63\u2013125. https:\/\/doi.org\/10.1561\/1100000015","journal-title":"Found Trends Hum-Comput Interact"},{"issue":"2","key":"790_CR9","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1007\/s11948-014-9536-x","volume":"21","author":"I Oosterlaken","year":"2015","unstructured":"Oosterlaken I (2015) Applying value sensitive design (VSD) to wind turbines and wind parks: an exploration. Sci Eng Ethics 21(2):359\u2013379. https:\/\/doi.org\/10.1007\/s11948-014-9536-x","journal-title":"Sci Eng Ethics"},{"key":"790_CR10","doi-asserted-by":"publisher","DOI":"10.3990\/1.9789036533911","author":"A van Wynsberghe","year":"2012","unstructured":"van Wynsberghe A (2012) Designing robots with care: creating an ethical framework for the future design and implementation of care robots. University of Twente. https:\/\/doi.org\/10.3990\/1.9789036533911","journal-title":"University of Twente"},{"issue":"4","key":"790_CR11","doi-asserted-by":"publisher","first-page":"311","DOI":"10.1007\/s10676-016-9409-x","volume":"18","author":"A van Wynsberghe","year":"2016","unstructured":"van Wynsberghe A (2016) Service robots, care ethics, and design. Ethics Inf Technol 18(4):311\u2013321. https:\/\/doi.org\/10.1007\/s10676-016-9409-x","journal-title":"Ethics Inf Technol"},{"key":"790_CR12","doi-asserted-by":"publisher","unstructured":"Umbrello S, De Bellis AF (2018) A value-sensitive design approach to intelligent agents. In: Yampolskiy RV (ed) Artificial intelligence safety and security, CRC Press, pp 395\u2013410. https:\/\/doi.org\/10.13140\/RG.2.2.17162.77762","DOI":"10.13140\/RG.2.2.17162.77762"},{"issue":"1","key":"790_CR13","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3390\/bdcc3010005","volume":"3","author":"S Umbrello","year":"2019","unstructured":"Umbrello S (2019) Beneficial artificial intelligence coordination by means of a value sensitive design approach. Big Data Cogn Comput 3(1):5. https:\/\/doi.org\/10.3390\/bdcc3010005","journal-title":"Big Data Cogn Comput"},{"issue":"2","key":"790_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4018\/IJT.2019070101","volume":"10","author":"S Umbrello","year":"2019","unstructured":"Umbrello S (2019) Atomically precise manufacturing and responsible innovation. Int J Technoethics 10(2):1\u201321. https:\/\/doi.org\/10.4018\/IJT.2019070101","journal-title":"Int J Technoethics"},{"issue":"3","key":"790_CR15","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1007\/s11569-011-0135-x","volume":"5","author":"J Timmermans","year":"2011","unstructured":"Timmermans J, Zhao Y, van den Hoven J (2011) Ethics and nanopharmacy: value sensitive design of new drugs. NanoEthics 5(3):269\u2013283. https:\/\/doi.org\/10.1007\/s11569-011-0135-x","journal-title":"NanoEthics"},{"issue":"3","key":"790_CR16","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1007\/s10506-017-9211-z","volume":"25","author":"G Contissa","year":"2017","unstructured":"Contissa G, Lagioia F, Sartor G (2017) The ethical knob: ethically-customisable automated vehicles and the law. Artif Intell Law 25(3):365\u2013378. https:\/\/doi.org\/10.1007\/s10506-017-9211-z","journal-title":"Artif Intell Law"},{"key":"790_CR17","doi-asserted-by":"crossref","unstructured":"Thornton SM, Lewis FE, Zhang V, Kochenderfer MJ, Gerdes JC (2018) Value sensitive design for autonomous vehicle motion planning. In: 2018 IEEE intelligent vehicles symposium (IV); IEEE, pp 1157\u20131162","DOI":"10.1109\/IVS.2018.8500441"},{"key":"790_CR18","unstructured":"Dogan E, Chatila R, Chauvier S, Evans K, Hadjixenophontos P, Perrin J (2016) Ethics in the design of automated vehicles: the AVEthics project. In: CEUR workshop proceedings; pp 10\u201313"},{"key":"790_CR19","unstructured":"Contissa G, Lagioia F, Sartor G (2017) Accidents involving autonomous vehicles: legal issues and ethical dilemmas. JUSLETTER:1\u20137"},{"key":"790_CR20","unstructured":"Gupta A (2019) Machine learning algorithms in autonomous driving https:\/\/iiot-world.com\/machine-learning\/machine-learning-algorithms-in-autonomous-driving\/. Accessed 16 Jul 2019"},{"key":"790_CR21","doi-asserted-by":"publisher","unstructured":"Wachter S, Mittelstadt B, Floridi L (2016) European union regulations on algorithmic decision-making and a \u201cright to explanation\u201d. https:\/\/doi.org\/10.1609\/aimag.v38i3.2741","DOI":"10.1609\/aimag.v38i3.2741"},{"issue":"2","key":"790_CR22","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1007\/s10676-017-9419-3","volume":"19","author":"D Leben","year":"2017","unstructured":"Leben D (2017) A Rawlsian algorithm for autonomous vehicles. Ethics Inf Technol 19(2):107\u2013115","journal-title":"Ethics Inf Technol"},{"key":"790_CR23","doi-asserted-by":"publisher","unstructured":"Pitt J, Diaconescu A (2016) Interactive self-governance and value-sensitive design for self-organising socio-technical systems. In: 2016 IEEE 1st international workshops on foundations and applications of self* systems (FAS*W), pp 30\u201335. https:\/\/doi.org\/10.1109\/FAS-W.2016.20","DOI":"10.1109\/FAS-W.2016.20"},{"key":"790_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-94-007-6994-6_3-1","volume-title":"Handbook of ethics, values, and technological design: sources, theory, values and application domains","author":"J Davis","year":"2014","unstructured":"Davis J, Nathan LP (2014) Value sensitive design: applications, adaptations, and critiques. In: van den Hoven J, Vermaas PE, van de Poel I (eds) Handbook of ethics, values, and technological design: sources, theory, values and application domains. Springer, Dordrecht, pp 1\u201326. https:\/\/doi.org\/10.1007\/978-94-007-6994-6_3-1"},{"issue":"1","key":"790_CR25","doi-asserted-by":"publisher","first-page":"4","DOI":"10.7146\/aahcc.v1i1.21619","volume":"1","author":"B Friedman","year":"2015","unstructured":"Friedman B, Hendry DG, Huldtgren A, Jonker C, Van den Hoven J, Van Wynsberghe A (2015) Charting the Next decade for value sensitive design. Aarhus Ser Hum Centered Comput 1(1):4. https:\/\/doi.org\/10.7146\/aahcc.v1i1.21619","journal-title":"Aarhus Ser Hum Centered Comput"},{"key":"790_CR26","first-page":"1177","volume-title":"The human-computer interaction handbook","author":"B Friedman","year":"2003","unstructured":"Friedman B, Kahn PH Jr (2003) Human values, ethics, and design. In: Jacko JA, Sears A (eds) The human-computer interaction handbook. L. Erlbaum Associates Inc., Hillsdale, pp 1177\u20131201"},{"issue":"1","key":"790_CR27","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1007\/s11948-011-9277-z","volume":"18","author":"J van den Hoven","year":"2012","unstructured":"van den Hoven J, Lokhorst GJ, van de Poel I (2012) Engineering and the problem of moral overload. Sci Eng Ethics 18(1):143\u2013155. https:\/\/doi.org\/10.1007\/s11948-011-9277-z","journal-title":"Sci Eng Ethics"},{"issue":"37","key":"790_CR28","first-page":"40","volume":"13","author":"S Umbrello","year":"2020","unstructured":"Umbrello S (2020) Meaningful human control over smart home systems: a value sensitive design approach. Humana Mente J Philos Stud 13(37):40\u201365","journal-title":"Humana Mente J Philos Stud"},{"issue":"2","key":"790_CR29","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1007\/s11948-011-9343-6","volume":"19","author":"A van Wynsberghe","year":"2013","unstructured":"van Wynsberghe A (2013) Designing robots for care: care centered value-sensitive design. Sci Eng Ethics 19(2):407\u2013433. https:\/\/doi.org\/10.1007\/s11948-011-9343-6","journal-title":"Sci Eng Ethics"},{"issue":"1","key":"790_CR30","doi-asserted-by":"publisher","first-page":"194","DOI":"10.1109\/TCST.2017.2654063","volume":"26","author":"Y Chen","year":"2018","unstructured":"Chen Y, Peng H, Grizzle J (2018) Obstacle avoidance for low-speed autonomous vehicles with barrier function. IEEE Trans Control Syst Technol 26(1):194\u2013206. https:\/\/doi.org\/10.1109\/TCST.2017.2654063","journal-title":"IEEE Trans Control Syst Technol"},{"key":"790_CR31","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1016\/j.scico.2017.05.006","volume":"148","author":"M Kamali","year":"2017","unstructured":"Kamali M, Dennis LA, McAree O, Fisher M, Veres SM (2017) Formal verification of autonomous vehicle platooning. Sci Comput Program 148:88\u2013106. https:\/\/doi.org\/10.1016\/j.scico.2017.05.006","journal-title":"Sci Comput Program"},{"key":"790_CR32","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1017\/9780511844317.004","volume-title":"Designing in ethics","author":"I van de Poel","year":"2017","unstructured":"van de Poel I (2017) Dealing with moral dilemmas through design. In: van den Hoven J, Miller S, Pogge T (eds) Designing in ethics. Cambridge University Press, Cambridge, pp 57\u201377"},{"key":"790_CR33","unstructured":"Abraham H, Lee C, Brady S, Fitzgerald C, Mehler B, Reimer B, Coughlin JF (2017) Autonomous vehicles, trust, and driving alternatives: a survey of consumer preferences. In: Transportation research board 96th annual meeting, Washington, pp 8\u201312"},{"key":"790_CR34","unstructured":"Yan C, Xu W, Liu J (2016) Can you trust autonomous vehicles: contactless attacks against sensors of self-driving vehicle. DEF CON 24"},{"key":"790_CR35","unstructured":"Prokhorov DV (2018) Mixed autonomous and manual control of autonomous vehicles. Google Patents"},{"key":"790_CR36","doi-asserted-by":"crossref","unstructured":"Wang C, Gong S, Zhou A, Li T, Peeta S (2018) Cooperative adaptive cruise control for connected autonomous vehicles by factoring communication-related constraints. http:\/\/arxiv.org\/abs\/1807.07232","DOI":"10.1016\/j.trpro.2019.05.014"},{"key":"790_CR37","doi-asserted-by":"publisher","DOI":"10.1145\/2398356.2398377","author":"T Denning","year":"2013","unstructured":"Denning T, Kohno T, Levy HM (2013) A framework for evaluating security risks associated with technologies used at home. Commun ACM. https:\/\/doi.org\/10.1145\/2398356.2398377","journal-title":"Commun ACM"},{"key":"790_CR38","doi-asserted-by":"publisher","unstructured":"Friedman B, Kahn Jr PH (2002) Value sensitive design: theory and methods. Univ Washingt Tech https:\/\/doi.org\/10.1016\/j.neuropharm.2007.08.009","DOI":"10.1016\/j.neuropharm.2007.08.009"},{"key":"790_CR39","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7585.001.0001","volume-title":"Value sensitive design: shaping technology with moral imagination","author":"B Friedman","year":"2019","unstructured":"Friedman B, Hendry DG (2019) Value sensitive design: shaping technology with moral imagination. Mit Press, Cambridge"},{"key":"790_CR40","unstructured":"Umbrello S (2020) Combinatory and complementary practices of values and virtues in design: a reply to Reijers and Gordijn. Filosofia"},{"key":"790_CR41","doi-asserted-by":"crossref","unstructured":"Czeskis A, Dermendjieva I, Yapit H, Borning A, Friedman B, Gill B, Kohno T (2010) Parenting from the pocket: value tensions and technical directions for secure and private parent-teen mobile safety. In: Proceedings of the sixth symposium on usable privacy and security, p 15","DOI":"10.1145\/1837110.1837130"},{"key":"790_CR42","doi-asserted-by":"crossref","unstructured":"Yoo D (2017) Stakeholder tokens: a constructive method for value sensitive design stakeholder analysis. In: Proceedings of the 2017 ACM conference companion publication on designing interactive systems, pp 280\u2013284","DOI":"10.1145\/3064857.3079161"},{"key":"790_CR43","doi-asserted-by":"publisher","unstructured":"Friedman B, Hendry DG (2012) The envisioning cards: a toolkit for catalyzing humanistic and technical imaginations. In: Proceedings of the 30th international conference on human factors in computing systems\u2014CHI \u201912, pp 1145\u20131148. https:\/\/doi.org\/10.1145\/2207676.2208562","DOI":"10.1145\/2207676.2208562"},{"issue":"1","key":"790_CR44","doi-asserted-by":"publisher","first-page":"148","DOI":"10.2307\/20024652","volume":"109","author":"L Winner","year":"2003","unstructured":"Winner L (2003) Do artifacts have politics? Technol Futur 109(1):148\u2013164. https:\/\/doi.org\/10.2307\/20024652","journal-title":"Technol Futur"},{"key":"790_CR45","unstructured":"Pinch T, Bijker WE (1987) The social construction of facts and artifacts. In: Bijker W E, Hughes TP, Pinch T (eds) The Social construction of technological systems\u202f: new directions in the sociology and history of technology, MIT Press, p 405"},{"key":"790_CR46","doi-asserted-by":"crossref","unstructured":"Borning A, Friedman B, Davis J, Lin P (2005) Informing public deliberation: value sensitive design of indicators for a large-scale urban simulation. In: ECSCW 2005, Springer, pp 449\u2013468","DOI":"10.1007\/1-4020-4023-7_23"},{"key":"790_CR47","first-page":"3","volume":"03\u201304","author":"PH Kahn Jr","year":"2003","unstructured":"Kahn PH Jr, Friedman B, Freier N, Severson R (2003) Coding manual for children\u2019s interactions with AIBO, the robotic dog-the preschool study. Univ Washingt CSE Tech Rep 03\u201304:3","journal-title":"Univ Washingt CSE Tech Rep"},{"issue":"11","key":"790_CR48","doi-asserted-by":"publisher","first-page":"2300","DOI":"10.1002\/asi.21146","volume":"60","author":"JP Woelfer","year":"2009","unstructured":"Woelfer JP, Hendry DG (2009) Stabilizing homeless young people with information and place. J Am Soc Inf Sci Technol 60(11):2300\u20132312","journal-title":"J Am Soc Inf Sci Technol"},{"key":"790_CR49","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1007\/978-94-007-7762-0_20","volume-title":"Philosophy and engineering: reflections on practice, principles and process","author":"I van de Poel","year":"2013","unstructured":"van de Poel I (2013) Translating values into design requirements. In: Michelfelder DP, McCarthy N, Goldberg DE (eds) Philosophy and engineering: reflections on practice, principles and process. Springer, Dordrecht, pp 253\u2013266. https:\/\/doi.org\/10.1007\/978-94-007-7762-0_20"},{"key":"790_CR50","doi-asserted-by":"crossref","unstructured":"Mortier HH, Henderson T, McAuely D, Crowcroft J (2014) Human-data interaction: the human face of the data-driven society. Soc Sci Res Netw","DOI":"10.2139\/ssrn.2508051"},{"issue":"3","key":"790_CR51","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1108\/09593841111158383","volume":"24","author":"A Johri","year":"2011","unstructured":"Johri A, Nair S (2011) The role of design values in information system development for human benefit. Inf Technol People 24(3):281\u2013302. https:\/\/doi.org\/10.1108\/09593841111158383","journal-title":"Inf Technol People"},{"key":"790_CR52","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/978-94-007-6994-6_10-1","volume-title":"Handbook of ethics, values, and technological design: sources, theory, values and application domains","author":"PE Vermaas","year":"2014","unstructured":"Vermaas PE, Hekkert P, Manders-Huits N, Tromp N (2014) Design methods in design for values. In: van den Hoven J, Vermaas PE, van de Poel I (eds) Handbook of ethics, values, and technological design: sources, theory, values and application domains. Springer, Dordrecht, pp 1\u201319. https:\/\/doi.org\/10.1007\/978-94-007-6994-6_10-1"},{"key":"790_CR53","doi-asserted-by":"crossref","unstructured":"Mecacci G, de Sio FS (2019) Meaningful human control as reason-responsiveness: the case of dual-mode vehicles. Ethics Inf Technol:1\u201313","DOI":"10.1007\/s10676-019-09519-w"},{"key":"790_CR54","unstructured":"Ghani R (2019) you say you want transparency and interpretability? https:\/\/dssg.uchicago.edu\/2016\/04\/27\/you-say-you-want-transparency-and-interpretability\/. Accessed 13 Feb 2019"},{"issue":"1","key":"790_CR55","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s11051-007-9269-8","volume":"10","author":"MC Roco","year":"2008","unstructured":"Roco MC (2008) Possibilities for global governance of converging technologies. J Nanoparticle Res 10(1):11\u201329. https:\/\/doi.org\/10.1007\/s11051-007-9269-8","journal-title":"J Nanoparticle Res"},{"key":"790_CR56","doi-asserted-by":"crossref","unstructured":"Boscoe B (2019) Creating transparency in algorithmic processes. Delphi Interdiscip Rev Emerg Technol 2 (1)","DOI":"10.21552\/delphi\/2019\/1\/5"},{"key":"790_CR57","unstructured":"Wooldridge M (2002) An introduction to multiagent systems, John Wiley & Sons"},{"issue":"9","key":"790_CR58","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/2494558","volume":"56","author":"M Fisher","year":"2013","unstructured":"Fisher M, Dennis L, Webster M (2013) Verifying autonomous systems. Commun ACM 56(9):84\u201393","journal-title":"Commun ACM"},{"key":"790_CR59","unstructured":"Rao AS, Georgeff MP (1992) An abstract architecture for rational agents. In: 3rd international conference on principles of knowledge representation and reasoning, pp 439\u2013449"},{"key":"790_CR60","unstructured":"Cointe N, Bonnet G, Boissier O (2016) Multi-agent based ethical asset management. In: CEUR workshop proceedings, pp 52\u201357"},{"key":"790_CR61","doi-asserted-by":"crossref","unstructured":"Caillou P, Gaudou B, Grignard A, Truong CQ, Taillandier PA (2017) Simple-to-use BDI architecture for agent-based modeling and simulation. In: Advances in social simulation 2015, Springer, pp 15\u201328","DOI":"10.1007\/978-3-319-47253-9_2"},{"key":"790_CR62","doi-asserted-by":"crossref","unstructured":"Lee S, Son Y-J (2008) Integrated human decision making model under belief-desire-intention framework for crowd simulation. In: Simulation conference, 2008. WSC 2008. Winter; IEEE, pp 886\u2013894","DOI":"10.1109\/WSC.2008.4736153"},{"issue":"3","key":"790_CR63","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1007\/s10515-014-0168-9","volume":"23","author":"LA Dennis","year":"2016","unstructured":"Dennis LA, Fisher M, Lincoln NK, Lisitsa A, Veres SM (2016) Practical verification of decision-making in agent-based autonomous systems. Autom Softw Eng 23(3):305\u2013359","journal-title":"Autom Softw Eng"},{"key":"790_CR64","unstructured":"Kamali M, Linker S, Fisher M (2018) Modular verification of vehicle platooning with respect to decisions, space and time. http:\/\/arxiv.org\/abs\/1804.06647"},{"key":"790_CR65","unstructured":"Hendrickson CS, Van Nieuwstadt MJ (2018) System and method for platooning vehicles. Google Patents"},{"key":"790_CR66","doi-asserted-by":"crossref","unstructured":"Calvert SC, Mecacci G, Heikoop DD, de Sio FS (2018) Full platoon control in truck platooning: a meaningful human control perspective. In: 2018 21st international conference on intelligent transportation systems (ITSC); IEEE, pp 3320\u20133326","DOI":"10.1109\/ITSC.2018.8570013"},{"issue":"2","key":"790_CR67","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1080\/15027570.2013.818399","volume":"12","author":"AM Johnson","year":"2013","unstructured":"Johnson AM, Axinn S (2013) The morality of autonomous robots. J Mil Ethics 12(2):129\u2013141. https:\/\/doi.org\/10.1080\/15027570.2013.818399","journal-title":"J Mil Ethics"},{"issue":"2","key":"790_CR68","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1080\/15027570.2015.1069013","volume":"14","author":"M Klincewicz","year":"2015","unstructured":"Klincewicz M (2015) Autonomous weapons systems, the frame problem and computer security. J Mil Ethics 14(2):162\u2013176. https:\/\/doi.org\/10.1080\/15027570.2015.1069013","journal-title":"J Mil Ethics"},{"issue":"1","key":"790_CR69","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/s00146-019-00879-x","volume":"35","author":"S Umbrello","year":"2020","unstructured":"Umbrello S, Torres P, De Bellis AF (2020) The future of war: Could lethal autonomous weapons make conflict more ethical? AI Soc 35(1):273\u2013282. https:\/\/doi.org\/10.1007\/s00146-019-00879-x","journal-title":"AI Soc"},{"key":"790_CR70","doi-asserted-by":"crossref","unstructured":"Mordatch I, Abbeel P (2018) Emergence of grounded compositional language in multi-agent populations. In: Thirty-second AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v32i1.11492"},{"issue":"4","key":"790_CR71","doi-asserted-by":"publisher","first-page":"e0172395","DOI":"10.1371\/journal.pone.0172395","volume":"12","author":"A Tampuu","year":"2017","unstructured":"Tampuu A, Matiisen T, Kodelja D, Kuzovkin I, Korjus K, Aru J, Aru J, Vicente R (2017) Multiagent cooperation and competition with deep reinforcement learning. PLoS ONE 12(4):e0172395","journal-title":"PLoS ONE"},{"key":"790_CR72","unstructured":"Mermet B, Simon G (2016) Formal verification of ethical properties in multiagent systems. In: CEUR workshop proceedings. pp 26\u201331"},{"key":"790_CR73","doi-asserted-by":"crossref","unstructured":"Bau D, Zhou B, Khosla A, Oliva A, Torralba A (2017) Network dissection: quantifying interpretability of deep visual representations. http:\/\/arxiv.org\/abs\/1704.05796","DOI":"10.1109\/CVPR.2017.354"},{"key":"790_CR74","unstructured":"Kim B, Wattenberg M, Gilmer J, Cai C, Wexler J, Viegas F (2018) Interpretability beyond feature attribution: quantitative testing with concept activation vectors (Tcav). In: International conference on machine learning, pp 2673\u20132682"}],"container-title":["International Journal of Social Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-021-00790-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12369-021-00790-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12369-021-00790-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T13:28:50Z","timestamp":1672147730000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12369-021-00790-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,15]]},"references-count":74,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,3]]}},"alternative-id":["790"],"URL":"https:\/\/doi.org\/10.1007\/s12369-021-00790-w","relation":{},"ISSN":["1875-4791","1875-4805"],"issn-type":[{"value":"1875-4791","type":"print"},{"value":"1875-4805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,15]]},"assertion":[{"value":"20 April 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 May 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}