{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,7]],"date-time":"2026-06-07T08:24:35Z","timestamp":1780820675714,"version":"3.54.1"},"reference-count":149,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T00:00:00Z","timestamp":1635292800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T00:00:00Z","timestamp":1635292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2023,2]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Organisations increasingly use automated decision-making systems (ADMS) to inform decisions that affect humans and their environment. While the use of ADMS can improve the accuracy and efficiency of decision-making processes, it is also coupled with ethical challenges. Unfortunately, the governance mechanisms currently used to oversee human decision-making often fail when applied to ADMS. In previous work, we proposed that ethics-based auditing (EBA)\u2014that is, a structured process by which ADMS are assessed for consistency with relevant principles or norms\u2014can (a) help organisations verify claims about their ADMS and (b) provide decision-subjects with justifications for the outputs produced by ADMS. In this article, we outline the conditions under which EBA procedures can be feasible and effective in practice. First, we argue that EBA is best understood as a \u2018soft\u2019 yet \u2018formal\u2019 governance mechanism. This implies that the main responsibility of auditors should be to spark ethical deliberation at key intervention points throughout the software development process and ensure that there is sufficient documentation to respond to potential inquiries. Second, we frame AMDS as parts of larger sociotechnical systems to demonstrate that to be feasible and effective, EBA procedures must link to intervention points that span all levels of organisational governance and all phases of the software lifecycle. The main function of EBA should, therefore, be to inform, formalise, assess, and interlink existing governance structures. Finally, we discuss the policy implications of our findings. To support the emergence of feasible and effective EBA procedures, policymakers and regulators could provide standardised reporting formats, facilitate knowledge exchange, provide guidance on how to resolve normative tensions, and create an independent body to oversee EBA of ADMS.<\/jats:p>","DOI":"10.1007\/s00146-021-01286-x","type":"journal-article","created":{"date-parts":[[2021,10,27]],"date-time":"2021-10-27T14:02:58Z","timestamp":1635343378000},"page":"153-171","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["Ethics-based auditing of automated decision-making systems: intervention points and policy implications"],"prefix":"10.1007","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8691-2582","authenticated-orcid":false,"given":"Jakob","family":"M\u00f6kander","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9488-3419","authenticated-orcid":false,"given":"Maria","family":"Axente","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,10,27]]},"reference":[{"key":"1286_CR2","unstructured":"Accenture (2018) Explainable AI: the next stage of human-machine collaboration. Accenture Labs. https:\/\/www.accenture.com\/gb-en\/insights\/technology\/explainable-ai-human-machine"},{"issue":"1","key":"1286_CR3","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1007\/s10115-017-1116-3","volume":"54","author":"P Adler","year":"2018","unstructured":"Adler P, Falk C, Friedler SA, Nix T, Rybeck G, Scheidegger C, Smith B, Venkatasubramanian S (2018) Auditing black-box models for indirect influence. Knowl Inf Syst 54(1):95\u2013122. https:\/\/doi.org\/10.1007\/s10115-017-1116-3","journal-title":"Knowl Inf Syst"},{"key":"1286_CR4","unstructured":"AI HLEG (2019) Ethics Guidelines for Trustworthy AI. European Commission"},{"key":"1286_CR5","doi-asserted-by":"publisher","unstructured":"AIEIG (2020) From principles to practice\u2014an interdisciplinary framework to operationalise AI ethics. AI Ethics Impact Group, VDE Association for Electrical Electronic and Information Technologies e.V., Bertelsmann Stiftung, pp 1\u201356. https:\/\/doi.org\/10.11586\/2020013","DOI":"10.11586\/2020013"},{"key":"1286_CR6","unstructured":"AlgorithmWatch (2019) Automating society: taking stock of automated decision-making in the EU. Bertelsmann Stiftung, Open Society Foundations"},{"key":"1286_CR7","doi-asserted-by":"crossref","unstructured":"Andrus M, Dean S, Gilbert TK, Lambert N, Zick T (2021) AI development for the public interest: from abstraction traps to sociotechnical risks. ArXiv: 2102.04255. http:\/\/arxiv.org\/abs\/2102.04255","DOI":"10.1109\/ISTAS50296.2020.9462193"},{"key":"1286_CR8","unstructured":"Auer F, Felderer M (2018) Shifting quality assurance of machine learning algorithms to live systems, pp 211\u2013212"},{"key":"1286_CR9","volume-title":"Understanding regulation: theory, strategy, and practice","author":"R Baldwin","year":"1999","unstructured":"Baldwin R, Cave M (1999) Understanding regulation: theory, strategy, and practice. Oxford University Press"},{"key":"1286_CR10","doi-asserted-by":"crossref","unstructured":"Bandy J (2021) Problematic machine behavior: a systematic literature review of algorithm audits. ArXiv: 2102.04256. http:\/\/arxiv.org\/abs\/2102.04256","DOI":"10.1145\/3449148"},{"key":"1286_CR11","doi-asserted-by":"publisher","unstructured":"Bashir MA, Farooq U, Shahid M, Zaffar MF, Wilson C (2019) Quantity vs. quality: evaluating user interest profiles using ad preference managers. In: Proceedings 2019 Network and Distributed System Security Symposium. Network and Distributed System Security Symposium, San Diego, CA. https:\/\/doi.org\/10.14722\/ndss.2019.23392","DOI":"10.14722\/ndss.2019.23392"},{"key":"1286_CR12","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3218675","author":"J Bauer","year":"2017","unstructured":"Bauer J (2017) The necessity of auditing artificial intelligence algorithms. SSRN Electron J. https:\/\/doi.org\/10.2139\/ssrn.3218675","journal-title":"SSRN Electron J"},{"key":"1286_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-017-0760-1","author":"SD Baum","year":"2017","unstructured":"Baum SD (2017) Social choice ethics in artificial intelligence. AI & Soc. https:\/\/doi.org\/10.1007\/s00146-017-0760-1","journal-title":"AI & Soc"},{"issue":"4","key":"1286_CR14","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1007\/s13347-017-0263-5","volume":"31","author":"R Binns","year":"2018","unstructured":"Binns R (2018) Algorithmic accountability and public reason. Philos Technol 31(4):543\u2013556. https:\/\/doi.org\/10.1007\/s13347-017-0263-5","journal-title":"Philos Technol"},{"key":"1286_CR15","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-60648-4","volume-title":"Towards a code of ethics for artificial intelligence","author":"P Boddington","year":"2017","unstructured":"Boddington P (2017) Towards a code of ethics for artificial intelligence. Springer"},{"issue":"1","key":"1286_CR16","doi-asserted-by":"publisher","first-page":"205395172098386","DOI":"10.1177\/2053951720983865","volume":"8","author":"S Brown","year":"2021","unstructured":"Brown S, Davidovic J, Hasan A (2021) The algorithm audit: scoring the algorithms that score us. Big Data Soc 8(1):2053951720983865. https:\/\/doi.org\/10.1177\/2053951720983865","journal-title":"Big Data Soc"},{"key":"1286_CR17","unstructured":"Brundage M, Avin S, Wang J, Belfield H, Krueger G, Hadfield G, Khlaaf H, Yang J, Toner H, Fong R, Maharaj T, Koh PW, Hooker S, Leung J, Trask A, Bluemke E, Lebensold J, O\u2019Keefe C, Koren M, Anderljung M (2020) Toward trustworthy AI development: mechanisms for supporting verifiable claims. ArXiv: 2004.07213. http:\/\/arxiv.org\/abs\/2004.07213"},{"issue":"2","key":"1286_CR18","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/s10551-019-04226-4","volume":"163","author":"A Buhmann","year":"2020","unstructured":"Buhmann A, Pa\u00dfmann J, Fieseler C (2020) Managing algorithmic accountability: balancing reputational concerns, engagement strategies, and the potential of rational discourse. J Bus Ethics 163(2):265\u2013280. https:\/\/doi.org\/10.1007\/s10551-019-04226-4","journal-title":"J Bus Ethics"},{"issue":"1","key":"1286_CR19","doi-asserted-by":"publisher","first-page":"205395171562251","DOI":"10.1177\/2053951715622512","volume":"3","author":"J Burrell","year":"2016","unstructured":"Burrell J (2016) How the machine \u2018thinks\u2019: understanding opacity in machine learning algorithms. Big Data Soc 3(1):205395171562251. https:\/\/doi.org\/10.1177\/2053951715622512","journal-title":"Big Data Soc"},{"key":"1286_CR20","unstructured":"Caba\u00f1as JG, Cuevas \u00c1, Cuevas R (2018) Unveiling and quantifying facebook exploitation of sensitive personal data for advertising purposes, p 18"},{"key":"1286_CR21","doi-asserted-by":"crossref","unstructured":"Cabrera \u00c1A, Epperson W, Hohman F, Kahng M, Morgenstern J, Chau DH (2019) FairVis: visual analytics for discovering intersectional bias in machine learning. ArXiv: 1904.05419 [Cs, Stat]. http:\/\/arxiv.org\/abs\/1904.05419","DOI":"10.1109\/VAST47406.2019.8986948"},{"issue":"2133","key":"1286_CR22","doi-asserted-by":"publisher","first-page":"20180080","DOI":"10.1098\/rsta.2018.0080","volume":"376","author":"C Cath","year":"2018","unstructured":"Cath C (2018) Governing artificial intelligence: Ethical, legal and technical opportunities and challenges. Philos Trans R Soc A Math Phys Eng Sci 376(2133):20180080. https:\/\/doi.org\/10.1098\/rsta.2018.0080","journal-title":"Philos Trans R Soc A Math Phys Eng Sci"},{"key":"1286_CR23","unstructured":"CDEI (2021) Types of assurance in AI and the role of standards. https:\/\/cdei.blog.gov.uk\/2021\/04\/17\/134\/"},{"issue":"4","key":"1286_CR24","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1023\/A:1010741324766","volume":"30","author":"TTY Chen","year":"2001","unstructured":"Chen TTY (2001) Ethics control mechanisms: a comparative observation of Hong Kong companies. J Bus Ethics 30(4):391\u2013400. https:\/\/doi.org\/10.1023\/A:1010741324766","journal-title":"J Bus Ethics"},{"key":"1286_CR25","doi-asserted-by":"publisher","unstructured":"Chopra AK, Singh MP (2018) Sociotechnical systems and ethics in the large. In: AIES 2018\u2014Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, pp 48\u201353. https:\/\/doi.org\/10.1145\/3278721.3278740","DOI":"10.1145\/3278721.3278740"},{"key":"1286_CR26","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/12549.001.0001","volume-title":"AI ethics","author":"M Coeckelbergh","year":"2020","unstructured":"Coeckelbergh M (2020) AI ethics. MIT Press"},{"key":"1286_CR27","unstructured":"Cookson C (2018) Artificial intelligence faces public backlash, warns scientist. Financial Times. https:\/\/www.ft.com\/content\/0b301152-b0f8-11e8-99ca-68cf89602132"},{"key":"1286_CR03","unstructured":"Crane A, Matten D (2016) Business ethics: managing corporate citizenship and sustainability in the age of globalization (Fourth edn). Oxford"},{"key":"1286_CR28","unstructured":"Crawford K, Dobbe R, Dryer T, Fried G, Green B, Kaziunas E, Kak A, Mathur V, McElroy E, S\u00e1nchez AN, Raji D, Rankin JL, Richardson R, Schultz J, West SM, Whittaker M (2019) AI Now 2019 Report. 100."},{"issue":"4","key":"1286_CR29","doi-asserted-by":"publisher","first-page":"499","DOI":"10.1007\/s13347-018-0337-z","volume":"31","author":"M D\u2019Agostino","year":"2018","unstructured":"D\u2019Agostino M, Durante M (2018) Introduction: the governance of algorithms. Philos Technol 31(4):499\u2013505. https:\/\/doi.org\/10.1007\/s13347-018-0337-z","journal-title":"Philos Technol"},{"key":"1286_CR30","volume-title":"AI governance: a research agenda","author":"A Dafoe","year":"2018","unstructured":"Dafoe A (2018) AI governance: a research agenda. Future of Humanity Institute"},{"key":"1286_CR31","unstructured":"Danaher J (2017) Is technology value-neutral? New technologies and collective action problems. Institute for Ethics and Emerging Technologies. https:\/\/ieet.org\/index.php\/IEET\/more\/Danaher20171218"},{"issue":"01","key":"1286_CR33","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1109\/MIS.2017.1","volume":"32","author":"D Danks","year":"2017","unstructured":"Danks D, London AJ (2017a) Regulating autonomous systems: beyond standards. IEEE Intell Syst 32(01):88\u201391. https:\/\/doi.org\/10.1109\/MIS.2017.1","journal-title":"IEEE Intell Syst"},{"key":"1286_CR32","doi-asserted-by":"publisher","unstructured":"Danks D, London AJ (2017b) Algorithmic bias in autonomous systems. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, pp 4691\u20134697. https:\/\/doi.org\/10.24963\/ijcai.2017\/654","DOI":"10.24963\/ijcai.2017\/654"},{"key":"1286_CR34","doi-asserted-by":"crossref","unstructured":"Dash A, Mukherjee A, Ghosh S (2019) A Network-centric Framework for Auditing Recommendation Systems. http:\/\/arxiv.org\/abs\/1902.02710","DOI":"10.1109\/INFOCOM.2019.8737486"},{"key":"1286_CR35","unstructured":"Datta A, Datta A, Makagon J, Mulligan DK, Tschantz MC (2018) Discrimination in online advertising: a multidisciplinary inquiry. In: Conference on Fairness, Accountability and Transparency, pp 20\u201334. http:\/\/proceedings.mlr.press\/v81\/datta18a.html"},{"issue":"2","key":"1286_CR36","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1109\/TTS.2021.3074097","volume":"2","author":"S Dean","year":"2021","unstructured":"Dean S, Gilbert TK, Lambert N, Zick T (2021) Axes for sociotechnical inquiry in AI research. IEEE Trans Technol Soc 2(2):62\u201370. https:\/\/doi.org\/10.1109\/TTS.2021.3074097","journal-title":"IEEE Trans Technol Soc"},{"key":"1286_CR37","unstructured":"Deloitte (2020) Deloitte introduces trustworthy AI framework to guide organizations in ethical application of technology in the age of with. Press Release, New York, Aug. 26-New York, Aug. 26. https:\/\/www2.deloitte.com\/us\/en\/pages\/about-deloitte\/articles\/press-releases\/deloitte-introduces-trustworthy-ai-framework.html"},{"key":"1286_CR38","unstructured":"DeVries T, Misra I, Wang C, van der Maaten L (2019) Does object recognition work for everyone? https:\/\/arxiv.org\/abs\/1906.02659"},{"issue":"3","key":"1286_CR39","doi-asserted-by":"publisher","first-page":"273","DOI":"10.3390\/systems2030273","volume":"2","author":"P Di Maio","year":"2014","unstructured":"Di Maio P (2014) Towards a metamodel to support the joint optimization of socio technical systems. Systems 2(3):273\u2013296. https:\/\/doi.org\/10.3390\/systems2030273","journal-title":"Systems"},{"key":"1286_CR40","doi-asserted-by":"crossref","unstructured":"Dobbe R, Gilbert TK, Mintz Y (2019) Hard choices in artificial intelligence: addressing normative uncertainty through sociotechnical commitments. ArXiv: 1911.09005 [Cs, Eess]. http:\/\/arxiv.org\/abs\/1911.09005","DOI":"10.1145\/3375627.3375861"},{"key":"1286_CR41","unstructured":"EIOPA T (2018) About EIOPA [Text]. Eiopa\u2014European Commission. https:\/\/www.eiopa.europa.eu\/about_en"},{"key":"1286_CR42","unstructured":"EIU (2020) Staying ahead of the curve: the business case for responsible AI. Economist Intelligence Unit. https:\/\/pages.eiu.com\/rs\/753-RIQ-438\/images\/EIUStayingAheadOfTheCurve.pdf"},{"key":"1286_CR43","doi-asserted-by":"publisher","unstructured":"Epstein Z, Payne BH, Shen JH, Hong CJ, Felbo B, Dubey A, Groh M, Obradovich N, Cebrian M, Rahwan I (2018) TuringBox: an experimental platform for the evaluation of AI systems. In: Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, pp 5826\u20135828. https:\/\/doi.org\/10.24963\/ijcai.2018\/851","DOI":"10.24963\/ijcai.2018\/851"},{"key":"1286_CR44","doi-asserted-by":"crossref","unstructured":"Erdelyi OJ, Goldsmith J (2018) Regulating artificial intelligence P. In: AAAI\/ACM Conference on Artificial Intelligence, Ethics and Society. http:\/\/www.aies-conference.com\/wp-content\/papers\/main\/AIES_2018_paper_13.pdf","DOI":"10.1145\/3278721.3278731"},{"key":"1286_CR45","unstructured":"Eubanks V (2019) Automating inequality: how high tools profile, police, and punish the poor (First Picador edition). Picador"},{"key":"1286_CR47","unstructured":"European Commission (2019a) Communication: Building Trust in Human Centric Artificial Intelligence. https:\/\/ec.europa.eu\/digital-single-market\/en\/news\/communication-building-trust-human-centric-artificial-intelligence"},{"key":"1286_CR48","unstructured":"European Commission (2019b) Policy and investment recommendations for trustworthy Artificial Intelligence [Text]. Shaping Europe\u2019s Digital Future\u2014European Commission. https:\/\/ec.europa.eu\/digital-single-market\/en\/news\/policy-and-investment-recommendations-trustworthy-artificial-intelligence"},{"key":"1286_CR46","unstructured":"European Commission (2021) Proposal for a regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence and amending certain union legislative acts (Draft Regulation COM(2021) 206 final)"},{"key":"1286_CR01","unstructured":"EY (2018) Assurance in the age of AI. https:\/\/www.ey.com\/en_uk\/consulting\/trusted-ai-platform"},{"issue":"2","key":"1286_CR49","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/BF00872018","volume":"14","author":"L Falkenberg","year":"1995","unstructured":"Falkenberg L, Herremans I (1995) Ethical behaviours in organizations: directed by the formal or informal systems? J Bus Ethics 14(2):133\u2013143. https:\/\/doi.org\/10.1007\/BF00872018","journal-title":"J Bus Ethics"},{"issue":"3","key":"1286_CR50","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1109\/MIM.2020.9082795","volume":"23","author":"J Fjeld","year":"2020","unstructured":"Fjeld J (2020) Principled artificial intelligence. IEEE Instrum Meas Mag 23(3):27\u201331. https:\/\/doi.org\/10.1109\/MIM.2020.9082795","journal-title":"IEEE Instrum Meas Mag"},{"issue":"3","key":"1286_CR51","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11023-008-9113-7","volume":"18","author":"L Floridi","year":"2008","unstructured":"Floridi L (2008) The method of levels of abstraction. Mind Mach 18(3):303\u2013329. https:\/\/doi.org\/10.1007\/s11023-008-9113-7","journal-title":"Mind Mach"},{"key":"1286_CR52","volume-title":"The 4th revolution: how the infosphere is reshaping human reality","author":"L Floridi","year":"2014","unstructured":"Floridi L (2014a) The 4th revolution: how the infosphere is reshaping human reality. Oxford University Press"},{"key":"1286_CR63","doi-asserted-by":"crossref","unstructured":"Floridi L (2014b) The ethics of information. Oxford University Press","DOI":"10.1093\/acprof:oso\/9780199641321.001.0001"},{"issue":"6","key":"1286_CR53","doi-asserted-by":"publisher","first-page":"1669","DOI":"10.1007\/s11948-015-9733-2","volume":"22","author":"L Floridi","year":"2016","unstructured":"Floridi L (2016a) Tolerant paternalism: pro-ethical design as a resolution of the dilemma of toleration. Sci Eng Ethics 22(6):1669\u20131688. https:\/\/doi.org\/10.1007\/s11948-015-9733-2","journal-title":"Sci Eng Ethics"},{"issue":"2083","key":"1286_CR54","doi-asserted-by":"publisher","first-page":"20160112","DOI":"10.1098\/rsta.2016.0112","volume":"374","author":"L Floridi","year":"2016","unstructured":"Floridi L (2016b) Faultless responsibility: on the nature and allocation of moral responsibility for distributed moral actions. Philos Trans R Soc A Math Phys Eng Sci 374(2083):20160112. https:\/\/doi.org\/10.1098\/rsta.2016.0112","journal-title":"Philos Trans R Soc A Math Phys Eng Sci"},{"issue":"4","key":"1286_CR55","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1007\/s13347-017-0291-1","volume":"30","author":"L Floridi","year":"2017","unstructured":"Floridi L (2017a) Infraethics\u2013on the conditions of possibility of morality. Philos Technol 30(4):391\u2013394. https:\/\/doi.org\/10.1007\/s13347-017-0291-1","journal-title":"Philos Technol"},{"issue":"3","key":"1286_CR56","doi-asserted-by":"publisher","first-page":"495","DOI":"10.1007\/s11023-017-9438-1","volume":"27","author":"L Floridi","year":"2017","unstructured":"Floridi L (2017b) The logic of design as a conceptual logic of information. Mind Mach 27(3):495\u2013519. https:\/\/doi.org\/10.1007\/s11023-017-9438-1","journal-title":"Mind Mach"},{"issue":"1","key":"1286_CR57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13347-018-0303-9","volume":"31","author":"L Floridi","year":"2018","unstructured":"Floridi L (2018) Soft ethics and the governance of the digital. Philos Technol 31(1):1\u20138. https:\/\/doi.org\/10.1007\/s13347-018-0303-9","journal-title":"Philos Technol"},{"issue":"6","key":"1286_CR58","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s42256-019-0055-y","volume":"1","author":"L Floridi","year":"2019","unstructured":"Floridi L (2019a) Establishing the rules for building trustworthy AI. Nat Mach Intell 1(6):261\u2013262. https:\/\/doi.org\/10.1038\/s42256-019-0055-y","journal-title":"Nat Mach Intell"},{"issue":"2","key":"1286_CR59","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/s13347-019-00354-x","volume":"32","author":"L Floridi","year":"2019","unstructured":"Floridi L (2019b) Translating principles into practices of digital ethics: Five risks of being unethical. Philos Technol 32(2):185\u2013193. https:\/\/doi.org\/10.1007\/s13347-019-00354-x","journal-title":"Philos Technol"},{"key":"1286_CR60","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/99608f92.8cd550d1","volume":"1","author":"L Floridi","year":"2019","unstructured":"Floridi L, Cowls J (2019) A unified framework of five principles for AI in society. Harv Data Sci Rev 1:1\u201313. https:\/\/doi.org\/10.1162\/99608f92.8cd550d1","journal-title":"Harv Data Sci Rev"},{"issue":"1","key":"1286_CR61","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/s11023-020-09521-y","volume":"30","author":"L Floridi","year":"2020","unstructured":"Floridi L, Strait A (2020) Ethical foresight analysis: what it is and why it is needed? Mind Mach 30(1):77\u201397. https:\/\/doi.org\/10.1007\/s11023-020-09521-y","journal-title":"Mind Mach"},{"issue":"4","key":"1286_CR62","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","volume":"28","author":"L Floridi","year":"2018","unstructured":"Floridi L, Cowls J, Beltrametti M, Chatila R, Chazerand P, Dignum V, Luetge C, Madelin R, Pagallo U, Rossi F, Schafer B, Valcke P, Vayena E (2018) AI4People\u2014an ethical framework for a good AI society: opportunities, risks, principles, and recommendations. Mind Mach 28(4):689\u2013707. https:\/\/doi.org\/10.1007\/s11023-018-9482-5","journal-title":"Mind Mach"},{"issue":"3","key":"1286_CR64","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1007\/s11023-020-09539-2","volume":"30","author":"I Gabriel","year":"2020","unstructured":"Gabriel I (2020) Artificial intelligence, values, and alignment. Mind Mach 30(3):411\u2013437. https:\/\/doi.org\/10.1007\/s11023-020-09539-2","journal-title":"Mind Mach"},{"issue":"6","key":"1286_CR65","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1109\/MIC.2017.4180835","volume":"21","author":"U Gasser","year":"2017","unstructured":"Gasser U, Almeida VAF (2017) A layered model for AI governance. IEEE Internet Comput 21(6):58\u201362. https:\/\/doi.org\/10.1109\/MIC.2017.4180835","journal-title":"IEEE Internet Comput"},{"key":"1286_CR66","doi-asserted-by":"publisher","unstructured":"Gasser U, Schmitt C (2019) The role of professional norms in the governance of artificial intelligence. In: The Oxford handbook of ethics of AI. Oxford University Press, https:\/\/doi.org\/10.2139\/ssrn.3378267","DOI":"10.2139\/ssrn.3378267"},{"key":"1286_CR67","unstructured":"Goodman BW (2016) A step towards accountable algorithms?: algorithmic discrimination and the European Union General Data Protection, 9"},{"key":"1286_CR68","doi-asserted-by":"publisher","unstructured":"Goodman B (2021) Hard Choices and hard limits for artificial intelligence. In: Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society, 112\u2013121. https:\/\/doi.org\/10.1145\/3461702.3462539","DOI":"10.1145\/3461702.3462539"},{"key":"1286_CR69","unstructured":"Google (2020) What-if-tool. In: Partnership on AI. https:\/\/pair-code.github.io\/what-if-tool\/index.html"},{"issue":"3","key":"1286_CR70","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1136\/medethics-2019-105586","volume":"46","author":"T Grote","year":"2020","unstructured":"Grote T, Berens P (2020) On the ethics of algorithmic decision-making in healthcare. J Med Ethics 46(3):205\u2013211. https:\/\/doi.org\/10.1136\/medethics-2019-105586","journal-title":"J Med Ethics"},{"key":"1286_CR71","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09517-8","author":"T Hagendorff","year":"2020","unstructured":"Hagendorff T (2020) The ethics of ai ethics: an evaluation of guidelines. Minds Mach. https:\/\/doi.org\/10.1007\/s11023-020-09517-8","journal-title":"Minds Mach"},{"key":"1286_CR72","volume-title":"Computer-assisted theory building: modeling dynamic social systems","author":"R Hanneman","year":"1988","unstructured":"Hanneman R (1988) Computer-assisted theory building: modeling dynamic social systems. Sage"},{"issue":"1","key":"1286_CR73","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/s13347-014-0184-5","volume":"29","author":"RK Hill","year":"2016","unstructured":"Hill RK (2016) What an algorithm is. Philos Technol 29(1):35\u201359. https:\/\/doi.org\/10.1007\/s13347-014-0184-5","journal-title":"Philos Technol"},{"key":"1286_CR74","doi-asserted-by":"publisher","DOI":"10.5040\/9781474201124","author":"C Hodges","year":"2015","unstructured":"Hodges C (2015) Ethics in business practice and regulation. Law Corp Behav Integr Theor Regul Enforc Compliance Ethics. https:\/\/doi.org\/10.5040\/9781474201124","journal-title":"Law Corp Behav Integr Theor Regul Enforc Compliance Ethics"},{"issue":"1","key":"1286_CR75","doi-asserted-by":"publisher","first-page":"694","DOI":"10.1002\/pra2.2018.14505501084","volume":"55","author":"AL Hoffmann","year":"2018","unstructured":"Hoffmann AL, Roberts ST, Wolf CT, Wood S (2018) Beyond fairness, accountability, and transparency in the ethics of algorithms: contributions and perspectives from LIS. Proc Assoc Inform Sci Technol 55(1):694\u2013696. https:\/\/doi.org\/10.1002\/pra2.2018.14505501084","journal-title":"Proc Assoc Inform Sci Technol"},{"key":"1286_CR76","unstructured":"Holland S, Hosny A, Newman S, Joseph J, Chmielinski K (2018) The dataset nutrition label: a framework to drive higher data quality standards. May. http:\/\/arxiv.org\/abs\/1805.03677"},{"key":"1286_CR77","unstructured":"ICO (2020) Guidance on the AI auditing framework: draft guidance for consultation. Information Commissioner\u2019s Office (UK)"},{"key":"1286_CR78","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/978-3-030-12524-0_2","volume":"95","author":"IEEE","year":"2019","unstructured":"IEEE (2019) Ethically aligned design. Intell Syst Control Autom Sci Eng 95:11\u201316. https:\/\/doi.org\/10.1007\/978-3-030-12524-0_2","journal-title":"Intell Syst Control Autom Sci Eng"},{"key":"1286_CR79","first-page":"389","volume":"1","author":"A Jobin","year":"2019","unstructured":"Jobin A, Ienca M, Vayena E (2019) Artificial intelligence: the global landscape of ethics guidelines. Web 1:389\u2013399","journal-title":"Web"},{"key":"1286_CR80","volume-title":"Thinking, fast and slow","author":"D Kahneman","year":"2012","unstructured":"Kahneman D (2012) Thinking, fast and slow. Penguin"},{"key":"1286_CR81","volume-title":"Noise: a flaw in human judgment","author":"D Kahneman","year":"2021","unstructured":"Kahneman D (2021) Noise: a flaw in human judgment. William Collins"},{"key":"1286_CR82","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3609292","author":"E Kazim","year":"2020","unstructured":"Kazim E, Koshiyama A (2020a) A high-level overview of AI ethics. Soc Sci Res Netw. https:\/\/doi.org\/10.2139\/ssrn.3609292 (SSRN Scholarly Paper ID 3609292)","journal-title":"Soc Sci Res Netw"},{"key":"1286_CR83","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-020-00029-w","author":"E Kazim","year":"2020","unstructured":"Kazim E, Koshiyama A (2020b) The interrelation between data and AI ethics in the context of impact assessments. AI & Ethics. https:\/\/doi.org\/10.1007\/s43681-020-00029-w","journal-title":"AI & Ethics"},{"key":"1286_CR84","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-021-00039-2","author":"E Kazim","year":"2021","unstructured":"Kazim E, Denny DMT, Koshiyama A (2021) AI auditing and impact assessment: according to the UK information commissioner\u2019s office. AI & Ethics. https:\/\/doi.org\/10.1007\/s43681-021-00039-2","journal-title":"AI & Ethics"},{"issue":"1","key":"1286_CR85","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3440959.3440966","volume":"18","author":"M Kearns","year":"2020","unstructured":"Kearns M, Roth A (2020) Ethical algorithm design. Sigecom Exch 18(1):31\u201336. https:\/\/doi.org\/10.1145\/3440959.3440966","journal-title":"Sigecom Exch"},{"key":"1286_CR86","doi-asserted-by":"publisher","unstructured":"Keyes O, Hutson J, Durbin M (2019) A mulching proposal: analysing and improving an algorithmic system for turning the elderly into high-nutrient slurry. In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems\u2014CHI EA\u201919, 1\u201311. https:\/\/doi.org\/10.1145\/3290607.3310433","DOI":"10.1145\/3290607.3310433"},{"key":"1286_CR87","unstructured":"Kim P (2017) Auditing algorithms for discrimination (SSRN Scholarly Paper ID 3093982). Social Science Research Network. https:\/\/papers.ssrn.com\/abstract=3093982"},{"key":"1286_CR88","doi-asserted-by":"publisher","unstructured":"Kleinberg J, Mullainathan S, Raghavan M (2017) Inherent tradeoffs in the fair determination of risk scores. In: Leibniz International Proceedings in Informatics, LIPIcs, 67, 1\u201323. https:\/\/doi.org\/10.4230\/LIPIcs.ITCS.2017.43","DOI":"10.4230\/LIPIcs.ITCS.2017.43"},{"key":"1286_CR89","doi-asserted-by":"publisher","unstructured":"Koene A, Clifton C, Hatada Y, Webb H, Richardson R (2019) A governance framework for algorithmic accountability and transparency (Issue April). https:\/\/doi.org\/10.2861\/59990","DOI":"10.2861\/59990"},{"key":"1286_CR90","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3778998","author":"A Koshiyama","year":"2021","unstructured":"Koshiyama A, Kazim E, Treleaven P, Rai P, Szpruch L, Pavey G, Ahamat G, Leutner F, Goebel R, Knight A, Adams J, Hitrova C, Barnett J, Nachev P, Barber D, Chamorro-Premuzic T, Klemmer K, Gregorovic M, Khan S, Lomas E (2021) Towards algorithm auditing: a survey on managing legal, ethical and technological risks of AI, ML and associated algorithms. Soc Sci Res Netw. https:\/\/doi.org\/10.2139\/ssrn.3778998 (SSRN Scholarly Paper ID 3778998)","journal-title":"Soc Sci Res Netw"},{"key":"1286_CR02","unstructured":"KPMG (2020) Achieving trustworthy AI. https:\/\/home.kpmg\/au\/en\/home\/insights\/2020\/11\/trustworthy-ai.html"},{"issue":"2133","key":"1286_CR91","doi-asserted-by":"publisher","first-page":"20180084","DOI":"10.1098\/rsta.2018.0084","volume":"376","author":"JA Kroll","year":"2018","unstructured":"Kroll JA (2018) The fallacy of inscrutability. Philos Trans R Soc A Math Phys Eng Sci 376(2133):20180084. https:\/\/doi.org\/10.1098\/rsta.2018.0084","journal-title":"Philos Trans R Soc A Math Phys Eng Sci"},{"issue":"3","key":"1286_CR92","first-page":"633","volume":"165","author":"J Kroll","year":"2017","unstructured":"Kroll J, Huey J, Barocas S, Felten E, Reidenberg J, Robinson D, Yu H (2017) Accountable algorithms. Univ PA Law Rev 165(3):633","journal-title":"Univ PA Law Rev"},{"key":"1286_CR93","unstructured":"Kusner M, Loftus J, Russell C, Silva R (2017) Counterfactual fairness. Advances in Neural Information Processing Systems, December, 4067\u20134077"},{"key":"1286_CR94","unstructured":"LaBrie RC, Steinke G (2019) Towards a framework for ethical audits of AI algorithms. https:\/\/aisel.aisnet.org\/amcis2019\/data_science_analytics_for_decision_support\/data_science_analytics_for_decision_support\/24"},{"key":"1286_CR95","doi-asserted-by":"crossref","unstructured":"Larsson S, Heintz F (2020) Transparency in artificial intelligence. Internet Policy Rev 9(2). https:\/\/policyreview.info\/concepts\/transparency-artificial-intelligence","DOI":"10.14763\/2020.2.1469"},{"key":"1286_CR96","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s43681-020-00013-4","volume":"0123456789","author":"D Lauer","year":"2020","unstructured":"Lauer D (2020) You cannot have AI ethics without ethics. AI & Ethics 0123456789:1\u20135. https:\/\/doi.org\/10.1007\/s43681-020-00013-4","journal-title":"AI & Ethics"},{"key":"1286_CR97","doi-asserted-by":"crossref","unstructured":"Lee M, Floridi L, Denev A (2020) Innovating with confidence: embedding governance and fairness in a financial services risk management framework. Berkeley Technol Law J","DOI":"10.2139\/ssrn.3830579"},{"issue":"4","key":"1286_CR98","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1007\/s13347-017-0279-x","volume":"31","author":"B Lepri","year":"2018","unstructured":"Lepri B, Oliver N, Letouz\u00e9 E, Pentland A, Vinck P (2018) Fair, transparent, and accountable algorithmic decision-making processes: the premise, the proposed solutions, and the open challenges. Philos Technol 31(4):611\u2013627. https:\/\/doi.org\/10.1007\/s13347-017-0279-x","journal-title":"Philos Technol"},{"key":"1286_CR1","doi-asserted-by":"publisher","unstructured":"Leslie D (2019) Understanding artificial intelligence ethics and safety. https:\/\/doi.org\/10.5281\/zenodo.3240529","DOI":"10.5281\/zenodo.3240529"},{"key":"1286_CR99","volume-title":"Engineering a safer world: Systems thinking applied to safety","author":"N Leveson","year":"2011","unstructured":"Leveson N (2011) Engineering a safer world: Systems thinking applied to safety. MIT Press"},{"issue":"5","key":"1286_CR100","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3342355","volume":"52","author":"M Luckcuck","year":"2019","unstructured":"Luckcuck M, Farrell M, Dennis L, Dixon C, Fisher M (2019) Formal specification and verification of autonomous robotic systems: a survey. ACM Comput Surv 52(5):1\u201341. https:\/\/doi.org\/10.1145\/3342355","journal-title":"ACM Comput Surv"},{"key":"1286_CR101","unstructured":"Lurie E, Mustafaraj E (2019) Opening up the black box: auditing Google\u2019s top stories algorithm. In: The Thirty-Second International Flairs Conference. The Thirty-Second International Flairs Conference. https:\/\/aaai.org\/ocs\/index.php\/FLAIRS\/FLAIRS19\/paper\/view\/18316"},{"key":"1286_CR102","doi-asserted-by":"publisher","unstructured":"Matthews J, Babaeianjelodar M, Lorenz S, Matthews A, Njie M, Adams N, Krane D, Goldthwaite J, Hughes C (2019) The right to confront your accusers: opening the black box of forensic DNA Software. In: Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, pp 321\u2013327. https:\/\/doi.org\/10.1145\/3306618.3314279","DOI":"10.1145\/3306618.3314279"},{"key":"1286_CR103","doi-asserted-by":"publisher","unstructured":"Mcnamara A, Smith J, Murphy-Hill E (2018) Does ACM\u2019s code of ethics change ethical decision making in software development? 729\u2013733. https:\/\/doi.org\/10.1145\/3236024.3264833","DOI":"10.1145\/3236024.3264833"},{"key":"1286_CR104","doi-asserted-by":"publisher","unstructured":"Mitchell M, Wu S, Zaldivar A, Barnes P, Vasserman L, Hutchinson B, Spitzer E, Raji ID, Gebru T (2019) Model cards for model reporting. In: Proceedings of the Conference on Fairness, Accountability, and Transparency\u2014FAT*\u201919, pp 220\u2013229. https:\/\/doi.org\/10.1145\/3287560.3287596","DOI":"10.1145\/3287560.3287596"},{"key":"1286_CR105","unstructured":"Mittelstadt B (2016) Auditing for transparency in content personalization systems. Int J Commun"},{"key":"1286_CR106","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-021-09557-8","author":"J M\u00f6kander","year":"2021","unstructured":"M\u00f6kander J, Floridi L (2021) Ethics-based auditing to develop trustworthy AI. Mind Mach. https:\/\/doi.org\/10.1007\/s11023-021-09557-8","journal-title":"Mind Mach"},{"issue":"4","key":"1286_CR107","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1007\/s11948-021-00319-4","volume":"27","author":"J M\u00f6kander","year":"2021","unstructured":"M\u00f6kander J, Morley J, Taddeo M, Floridi L (2021a) Ethics-based auditing of automated decision-making systems: nature, scope, and limitations. Sci Eng Ethics 27(4):44. https:\/\/doi.org\/10.1007\/s11948-021-00319-4","journal-title":"Sci Eng Ethics"},{"key":"1286_CR108","doi-asserted-by":"crossref","unstructured":"M\u00f6kander J, Axente M, Casolari F, Floridi L (2021b) Conformity assessments and post-market monitoring: a guide to the role of auditing in the proposed European AI Regulation. Minds Mach (Forthcoming)","DOI":"10.1007\/s11023-021-09577-4"},{"issue":"4","key":"1286_CR109","doi-asserted-by":"publisher","first-page":"2141","DOI":"10.1007\/s11948-019-00165-5","volume":"26","author":"J Morley","year":"2020","unstructured":"Morley J, Floridi L, Kinsey L, Elhalal A (2020) From what to how: an initial review of publicly available AI ethics tools, methods and research to translate principles into practices. Sci Eng Ethics 26(4):2141\u20132141. https:\/\/doi.org\/10.1007\/s11948-019-00165-5","journal-title":"Sci Eng Ethics"},{"issue":"2","key":"1286_CR110","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1007\/s11023-021-09563-w","volume":"31","author":"J Morley","year":"2021","unstructured":"Morley J, Elhalal A, Garcia F, Kinsey L, M\u00f6kander J, Floridi L (2021) Ethics as a service: a pragmatic operationalisation of AI ethics. Mind Mach 31(2):239\u2013256. https:\/\/doi.org\/10.1007\/s11023-021-09563-w","journal-title":"Mind Mach"},{"key":"1286_CR111","doi-asserted-by":"crossref","unstructured":"Muhlenbach F (2020) A methodology for ethics-by-design AI systems: dealing with human value conflicts. http:\/\/arxiv.org\/abs\/2010.07610","DOI":"10.1109\/SMC42975.2020.9283185"},{"key":"1286_CR112","volume-title":"Weapons of math destruction: how big data increases inequality and threatens democracy","author":"C O\u2019Neil","year":"2016","unstructured":"O\u2019Neil C (2016) Weapons of math destruction: how big data increases inequality and threatens democracy. Allen Lane"},{"key":"1286_CR113","doi-asserted-by":"publisher","unstructured":"OECD (2015) Industry self regulation: role and use in supporting consumer interests (OECD Digital Economy Papers No. 247; OECD Digital Economy Papers, vol. 247). https:\/\/doi.org\/10.1787\/5js4k1fjqkwh-en","DOI":"10.1787\/5js4k1fjqkwh-en"},{"key":"1286_CR114","unstructured":"OECD (2019) Recommendation of\u00a0the Council on Artificial Intelligence. OECD\/LEGAL\/0449"},{"key":"1286_CR115","unstructured":"OECD (2020) OECD framework for the classification of AI systems\u2014interim report on the activities of the One AI Working Group on the Classification of AI Systems. DSTI\/CDEP(2020)13, November."},{"key":"1286_CR116","unstructured":"Pedreschi D, Giannotti F, Guidotti R, Monreale A, Pappalardo L, Ruggieri S, Turini F (2018) Open the black box data-driven explanation of black box decision systems. ArXiv: 1806.09936 [Cs]. http:\/\/arxiv.org\/abs\/1806.09936"},{"key":"1286_CR04","unstructured":"Power M (1997) The audit society: rituals of verification. Oxford University Press"},{"key":"1286_CR117","unstructured":"PwC (2019) A practical guide to responsible artificial intelligence (AI) (p. 20)"},{"issue":"1","key":"1286_CR118","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s10676-017-9430-8","volume":"20","author":"I Rahwan","year":"2018","unstructured":"Rahwan I (2018) Society-in-the-loop: programming the algorithmic social contract. Ethics Inf Technol 20(1):5\u201314. https:\/\/doi.org\/10.1007\/s10676-017-9430-8","journal-title":"Ethics Inf Technol"},{"key":"1286_CR119","doi-asserted-by":"publisher","unstructured":"Raji ID, Buolamwini J (2019) Actionable auditing: investigating the impact of publicly naming biased performance results of commercial AI products. In: Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, pp 429\u2013435. https:\/\/doi.org\/10.1145\/3306618.3314244","DOI":"10.1145\/3306618.3314244"},{"key":"1286_CR120","doi-asserted-by":"publisher","unstructured":"Raji ID, Smart A, White RN, Mitchell M, Gebru T, Hutchinson B, Smith-Loud J, Theron D, Barnes P (2020) Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In: FAT* 2020\u2014Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pp 33\u201344. https:\/\/doi.org\/10.1145\/3351095.3372873","DOI":"10.1145\/3351095.3372873"},{"key":"1286_CR121","unstructured":"Regulation (EU) No 1168\/2011. (2011). Regulation (EU) No 1168\/2011 of the European Parliament and of the Council of Europe [Legislation]. EU"},{"key":"1286_CR122","doi-asserted-by":"publisher","unstructured":"Robertson R, Jiang S, Joseph K, Friedland L, Lazer D, Wilson C (2018) Auditing partisan audience bias within google search. In: Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1\u201322. https:\/\/doi.org\/10.1145\/3274417","DOI":"10.1145\/3274417"},{"key":"1286_CR123","unstructured":"Rushby J (1988) Quality measures and assurance for AI software\/John Rushby. National Aeronautics and Space Administration, Scientific and Technical Information Division. https:\/\/hdl.handle.net\/2027\/uiug.30112106738088"},{"key":"1286_CR124","unstructured":"Saleiro P, Kuester B, Hinkson L, London J, Stevens A, Anisfeld A, Rodolfa KT, Ghani R (2019) Aequitas: a bias and fairness audit toolkit. ArXiv: 1811.05577 [Cs]. http:\/\/arxiv.org\/abs\/1811.05577"},{"issue":"3429","key":"1286_CR125","first-page":"741","volume":"132","author":"AL Samuel","year":"1960","unstructured":"Samuel AL (1960) Some moral and technical consequences of automation\u2014a refutation. Am Assoc Adv Sci 132(3429):741\u2013742","journal-title":"Am Assoc Adv Sci"},{"key":"1286_CR126","unstructured":"Sandvig C, Hamilton K, Karahalios K, Langbort C (2014) Auditing algorithms: research methods for detecting discrimination on internet platforms. 23"},{"key":"1286_CR127","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3453632","author":"J Schuett","year":"2019","unstructured":"Schuett J (2019) A legal definition of AI. Arxiv. https:\/\/doi.org\/10.2139\/ssrn.3453632","journal-title":"Arxiv"},{"key":"1286_CR128","unstructured":"Schulam P, Saria S (2019) Can you trust this prediction? Auditing pointwise reliability after learning. ArXiv: 1901.00403 [Cs, Stat]. http:\/\/arxiv.org\/abs\/1901.00403"},{"key":"1286_CR129","unstructured":"Selbst AD (2021) An institutional view of algorithmic impact assessments (SSRN Scholarly Paper ID 3867634). Social Science Research Network. https:\/\/papers.ssrn.com\/abstract=3867634"},{"key":"1286_CR130","doi-asserted-by":"publisher","unstructured":"Stegh\u00f6fer JP, Knauss E, Horkoff J, Wohlrab R (2019) Challenges of scaled agile for safety-critical systems. In: Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11915 LNCS, pp 350\u2013366. https:\/\/doi.org\/10.1007\/978-3-030-35333-9_26","DOI":"10.1007\/978-3-030-35333-9_26"},{"key":"1286_CR131","unstructured":"Suresh H, Guttag JV (2019) A framework for understanding unintended consequences of machine learning. ArXiv"},{"issue":"6404","key":"1286_CR132","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1126\/science.aat5991","volume":"361","author":"M Taddeo","year":"2018","unstructured":"Taddeo M, Floridi L (2018) How AI can be a force for good. Science 361(6404):751\u2013752. https:\/\/doi.org\/10.1126\/science.aat5991","journal-title":"Science"},{"issue":"1","key":"1286_CR133","doi-asserted-by":"publisher","first-page":"61","DOI":"10.2139\/ssrn.3172840","volume":"7","author":"J Tasioulas","year":"2018","unstructured":"Tasioulas J (2018) First steps towards an ethics of robots and artificial intelligence. SSRN Electron J 7(1):61\u201395. https:\/\/doi.org\/10.2139\/ssrn.3172840","journal-title":"SSRN Electron J"},{"key":"1286_CR134","unstructured":"The Institute of Internal Auditors (2017) The IIA\u2019s artificial intelligence auditing framework: practical applications Part A. Global Perspectives and Insights. www.theiia.org\/gpi"},{"issue":"9","key":"1286_CR135","doi-asserted-by":"publisher","first-page":"1396","DOI":"10.1016\/j.techfore.2008.04.004","volume":"75","author":"TA Tran","year":"2008","unstructured":"Tran TA, Daim T (2008) A taxonomic review of methods and tools applied in technology assessment. Technol Forecast Soc Change 75(9):1396\u20131405. https:\/\/doi.org\/10.1016\/j.techfore.2008.04.004","journal-title":"Technol Forecast Soc Change"},{"key":"1286_CR136","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.3662302","author":"A Tsamados","year":"2020","unstructured":"Tsamados A, Aggarwal N, Cowls J, Morley J, Roberts H, Taddeo M, Floridi L (2020) The ethics of algorithms: key problems and solutions. SSRN Electron J. https:\/\/doi.org\/10.2139\/ssrn.3662302","journal-title":"SSRN Electron J"},{"issue":"1","key":"1286_CR137","first-page":"171","volume":"175","author":"M Turilli","year":"2008","unstructured":"Turilli M (2008) Ethics and the practice of software design. Front Artif Intell Appl 175(1):171\u2013183","journal-title":"Front Artif Intell Appl"},{"key":"1286_CR138","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.2747994","author":"A Tutt","year":"2016","unstructured":"Tutt A (2016) An FDA for algorithms. SSRN Electron J. https:\/\/doi.org\/10.2139\/ssrn.2747994","journal-title":"SSRN Electron J"},{"issue":"3","key":"1286_CR139","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1007\/s11023-020-09537-4","volume":"30","author":"I Van de Poel","year":"2020","unstructured":"Van de Poel I (2020) Embedding values in artificial intelligence (AI) systems. Mind Mach 30(3):385\u2013409. https:\/\/doi.org\/10.1007\/s11023-020-09537-4","journal-title":"Mind Mach"},{"key":"1286_CR140","doi-asserted-by":"crossref","unstructured":"Vincent N, Johnson I, Sheehan P, Hecht B (2019) Measuring the importance of user-generated content to search engines. https:\/\/arxiv.org\/abs\/1906.08576","DOI":"10.1609\/icwsm.v13i01.3248"},{"issue":"4","key":"1286_CR141","doi-asserted-by":"publisher","first-page":"39","DOI":"10.2307\/248959","volume":"4","author":"IR Weiss","year":"1980","unstructured":"Weiss IR (1980) Auditability of software: a survey of techniques and costs. MIS Quar Manag Inform Syst 4(4):39\u201350. https:\/\/doi.org\/10.2307\/248959","journal-title":"MIS Quar Manag Inform Syst"},{"key":"1286_CR142","doi-asserted-by":"publisher","unstructured":"Whittlestone J, Alexandrova A, Nyrup R, Cave S (2019a) The role and limits of principles in AI ethics: towards a focus on tensions. In: AIES 2019\u2014proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, 195\u2013200. https:\/\/doi.org\/10.1145\/3306618.3314289","DOI":"10.1145\/3306618.3314289"},{"key":"1286_CR143","unstructured":"Whittlestone J, Nyrup R, Alexandrova A, Dihal K (2019b) Ethical and societal implications of algorithms, data, and artificial intelligence: a roadmap for research"},{"issue":"14","key":"1286_CR144","doi-asserted-by":"publisher","first-page":"eaar7650","DOI":"10.1126\/scirobotics.aar7650","volume":"3","author":"G-Z Yang","year":"2018","unstructured":"Yang G-Z, Bellingham J, Dupont PE, Fischer P, Floridi L, Full R, Jacobstein N, Kumar V, McNutt M, Merrifield R, Nelson BJ, Scassellati B, Taddeo M, Taylor R, Veloso M, Wang ZL, Wood R (2018) The grand challenges of Science Robotics. Sci Robots 3(14):eaar7650. https:\/\/doi.org\/10.1126\/scirobotics.aar7650","journal-title":"Sci Robots"},{"key":"1286_CR145","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-018-0330-6","author":"J Zerilli","year":"2018","unstructured":"Zerilli J, Knott A, Maclaurin J, Gavaghan C (2018) Transparency in algorithmic and human decision-making: is there a double standard? Philos Technol. https:\/\/doi.org\/10.1007\/s13347-018-0330-6","journal-title":"Philos Technol"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-021-01286-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-021-01286-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-021-01286-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,4]],"date-time":"2023-02-04T10:15:32Z","timestamp":1675505732000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-021-01286-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,27]]},"references-count":149,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,2]]}},"alternative-id":["1286"],"URL":"https:\/\/doi.org\/10.1007\/s00146-021-01286-x","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,27]]},"assertion":[{"value":"18 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 October 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"MA works for a professional services company that helps clients manage technology-related risks.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}