{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T13:37:47Z","timestamp":1774445867048,"version":"3.50.1"},"reference-count":118,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T00:00:00Z","timestamp":1708300800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T00:00:00Z","timestamp":1708300800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000275","name":"Leverhulme Trust","doi-asserted-by":"publisher","award":["RC-2015-067"],"award-info":[{"award-number":["RC-2015-067"]}],"id":[{"id":"10.13039\/501100000275","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["AI &amp; Soc"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Responsible AI (RAI) guidelines aim to ensure that AI systems respect democratic values. While a step in the right direction, they currently fail to impact practice. Our work discusses reasons for this lack of impact and clusters them into five areas: (1) the abstract nature of RAI guidelines, (2) the problem of selecting and reconciling values, (3) the difficulty of operationalising RAI success metrics, (4) the fragmentation of the AI pipeline, and (5) the lack of internal advocacy and accountability. Afterwards, we introduce a number of approaches to RAI from a range of disciplines, exploring their potential as solutions to the identified challenges. We anchor these solutions in practice through concrete examples, bridging the gap between the theoretical considerations of RAI and on-the-ground processes that currently shape how AI systems are built. Our work considers the socio-technical nature of RAI limitations and the resulting necessity of producing socio-technical solutions.<\/jats:p>","DOI":"10.1007\/s00146-024-01880-9","type":"journal-article","created":{"date-parts":[[2024,2,19]],"date-time":"2024-02-19T19:02:34Z","timestamp":1708369354000},"page":"199-215","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":34,"title":["Challenges of responsible AI in practice: scoping review and recommended actions"],"prefix":"10.1007","volume":"40","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8284-890X","authenticated-orcid":false,"given":"Malak","family":"Sadek","sequence":"first","affiliation":[]},{"given":"Emma","family":"Kallina","sequence":"additional","affiliation":[]},{"given":"Thomas","family":"Bohn\u00e9","sequence":"additional","affiliation":[]},{"given":"C\u00e9line","family":"Mougenot","sequence":"additional","affiliation":[]},{"given":"Rafael A.","family":"Calvo","sequence":"additional","affiliation":[]},{"given":"Stephen","family":"Cave","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,2,19]]},"reference":[{"issue":"7","key":"1880_CR1","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.5267\/j.ac.2020.8.022","volume":"6","author":"M Abuhusain","year":"2020","unstructured":"Abuhusain M (2020) The role of artificial intelligence and big data on loan decisions. Accounting 6(7):1291\u20131296","journal-title":"Accounting"},{"issue":"3","key":"1880_CR2","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1257\/aer.20190658","volume":"110","author":"H Allcott","year":"2020","unstructured":"Allcott H, Braghieri L, Eichmeyer S, Gentzkow M (2020) The welfare effects of social media. Am Econ Rev 110(3):629\u201376. https:\/\/doi.org\/10.1257\/aer.20190658","journal-title":"Am Econ Rev"},{"key":"1880_CR3","unstructured":"Antonic J (2021) How to foster advocacy for digital transformations through collaboration. Presentation at World Interaction Design Day (IxDD) and can be accessed at: https:\/\/vimeo.com\/619232039"},{"issue":"3","key":"1880_CR4","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1007\/s43681-021-00084-x","volume":"2","author":"J Ayling","year":"2021","unstructured":"Ayling J, Chapman A (2021) Putting ai ethics to work: are the tools fit for purpose? AI Ethics 2(3):405\u201329","journal-title":"AI Ethics"},{"key":"1880_CR5","unstructured":"Bacciarelli A, Westby J, Mass\u00e9 E, Mitnick D, Hidvegi F, Adegoke B, Kaltheuner F, Jayaram M, C\u00f3rdova Y, Barocas S, Isaac W (2018) The toronto declaration: Protecting the rights to equality and non-discrimination in machine learning systems - amnesty international. https:\/\/www.amnesty.org\/en\/documents\/pol30\/8447\/2018\/en\/. Accessed on 18 Aug 2023"},{"key":"1880_CR6","doi-asserted-by":"crossref","unstructured":"Barker M, Kallina E, Ashok D, Collins K, Casovan A, Weller A, Talwalkar A, Chen V, Bhatt U (2023) Feedbacklogs: Recording and incorporating stakeholder feedback into machine learning pipelines. In: Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pp 1\u201315","DOI":"10.1145\/3617694.3623239"},{"key":"1880_CR7","unstructured":"Bednar K, Spiekermann S (2021) On the power of ethics: how value-based thinking fosters creative and sustainable it innovation. WU Vienna University of Economics and Business (Working Papers\/Institute for IS & Society)"},{"issue":"2","key":"1880_CR8","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/s10506-020-09270-4","volume":"29","author":"A Bibal","year":"2021","unstructured":"Bibal A, Lognoul M, De Streel A, Fr\u00e9nay B (2021) Legal requirements on explainability in machine learning. Artif Intell Law 29(2):149\u2013169","journal-title":"Artif Intell Law"},{"issue":"2","key":"1880_CR9","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1038\/s42256-020-0151-z","volume":"2","author":"RA Calvo","year":"2020","unstructured":"Calvo RA, Peters D, Cave S (2020) Advancing impact assessment for intelligent systems. Nat Mach Intell 2(2):89\u201391","journal-title":"Nat Mach Intell"},{"issue":"4","key":"1880_CR10","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1007\/s00766-020-00333-1","volume":"25","author":"L Chazette","year":"2020","unstructured":"Chazette L, Schneider K (2020) Explainability as a non-functional requirement: challenges and recommendations. Requir Eng 25(4):493\u2013514","journal-title":"Requir Eng"},{"key":"1880_CR11","unstructured":"Chen J, Storchan V, Kurshan E (2021) Beyond fairness metrics: roadblocks and challenges for ethical ai in practice. arXiv preprint arXiv:2108.06217"},{"key":"1880_CR12","unstructured":"Cherubini M (n.d.) Ethical autonomous algorithms. Retrieved from https:\/\/medium.com\/@mchrbn\/ethical-autonomous-algorithms-5ad07c311bcc"},{"key":"1880_CR13","unstructured":"Chouldechova A, Benavides-Prado D, Fialko O, Vaithianathan R (2018) A case study of algorithm-assisted decision making in child maltreatment hotline screening decisions. In: Conference on Fairness, Accountability and Transparency, pp 134\u2013148"},{"key":"1880_CR14","volume-title":"The alignment problem - machine learning and human values","author":"B Christian","year":"2020","unstructured":"Christian B (2020) The alignment problem - machine learning and human values. W. W. Norton & Company"},{"key":"1880_CR15","unstructured":"Cognilytica (2021) Comprehensive ethical ai framework. (Tech. Rep.), Cognilytica"},{"key":"1880_CR16","unstructured":"Corporation M (2022) Responsible ai principles from microsoft. https:\/\/www.microsoft.com\/en-us\/ai\/responsible-ai?activetab=pivot1%3aprimaryr6. Accessed 02 Aug 2022"},{"key":"1880_CR17","unstructured":"Costanza-Chock S (2018) Design justice: Towards an intersectional feminist framework for design theory and practice. In: Proceedings of the Design Research Society"},{"key":"1880_CR18","volume-title":"The atlas of AI: Power, politics, and the planetary costs of artificial intelligence","author":"K Crawford","year":"2021","unstructured":"Crawford K (2021) The atlas of AI: Power, politics, and the planetary costs of artificial intelligence. Yale University Press"},{"issue":"7625","key":"1880_CR19","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1038\/538311a","volume":"538","author":"K Crawford","year":"2016","unstructured":"Crawford K, Calo R (2016) There is a blind spot in ai research. Nature 538(7625):311\u2013313","journal-title":"Nature"},{"key":"1880_CR20","unstructured":"Custis C (2021) Operationalizing ai ethics through documentation: About ml in 2020 and beyond. Retrieved from https:\/\/partnershiponai.org\/about-ml-2021\/"},{"key":"1880_CR21","unstructured":"Davies W (2017) How statistics lost their power - and why we should fear what comes next. The Guardian"},{"key":"1880_CR22","unstructured":"Delgado F, Yang S, Madaio M, Yang Q (2021) Stakeholder participation in ai: Beyond\" add diverse stakeholders and stir\". arXiv preprint arXiv:2111.01122"},{"key":"1880_CR23","doi-asserted-by":"crossref","unstructured":"Delgado F, Yang S, Madaio M, Yang Q (2023) The participatory turn in ai design: theoretical foundations and the current state of practice. In: Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pp 1\u201323","DOI":"10.1145\/3617694.3623261"},{"key":"1880_CR24","unstructured":"DigitalCatapult (2020) Lessons in practical ai ethics. Tech Rep, Digital Catapult"},{"key":"1880_CR25","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/11805.001.0001","volume-title":"Data feminism","author":"C D\u2019ignazio","year":"2020","unstructured":"D\u2019ignazio C, Klein LF (2020) Data feminism. MIT press"},{"key":"1880_CR26","unstructured":"Drage E, Kanta D, Stephen C, Kerry M (2022) Who makes AI? gender and portrayals of ai scientists in popular film 1920\u20132020"},{"issue":"1","key":"1880_CR27","doi-asserted-by":"crossref","DOI":"10.1126\/sciadv.aao5580","volume":"4","author":"J Dressel","year":"2018","unstructured":"Dressel J, Farid H (2018) The accuracy, fairness, and limits of predicting recidivism. Sci Adv 4(1):eaao5580","journal-title":"Sci Adv"},{"key":"1880_CR28","doi-asserted-by":"crossref","unstructured":"Edwards L, McAuley D, Diver L (2016) From privacy impact assessment to social impact assessment. In: Proceedings of the IEEE Computer Society Conference","DOI":"10.1109\/SPW.2016.19"},{"key":"1880_CR29","unstructured":"Elam M, Reich R (2022) Stanford hai artificial intelligence bill of rights. https:\/\/hai.stanford.edu\/white-paper-stanford-hai-artificial-intelligence-bill-rights. Accessed on 30 Nov 2023"},{"issue":"3","key":"1880_CR30","doi-asserted-by":"crossref","first-page":"699","DOI":"10.1007\/s10796-021-10230-9","volume":"24","author":"E Elshan","year":"2022","unstructured":"Elshan E, Zierau N, Engel C, Janson A, Leimeister JM (2022) Understanding the design elements affecting user acceptance of intelligent agents: past, present and future. Inf Syst Front 24(3):699\u2013730","journal-title":"Inf Syst Front"},{"key":"1880_CR31","unstructured":"Fair G (2021) Data based science: fair becomes the new normal. Retrieved from https:\/\/www.go-fair.org\/2021\/01\/21\/data-based-science-fair-becomes-the-new-normal\/"},{"key":"1880_CR32","unstructured":"Falk B, Gautam J, Srinivasan P, Alanoca S, Bora A, Jain A, Lannquist Y (2020) Ey & the future society report: Bridging ai\u2019s trust gap. (Tech. Rep.), EY. Accessed 13 Apr 2022"},{"issue":"6","key":"1880_CR33","doi-asserted-by":"crossref","first-page":"3333","DOI":"10.1007\/s11948-020-00276-4","volume":"26","author":"H Felzmann","year":"2020","unstructured":"Felzmann H, Fosch-Villaronga E, Lutz C, Tam\u00f2-Larrieux A (2020) Towards transparency by design for artificial intelligence. Sci Eng Ethics 26(6):3333\u20133361","journal-title":"Sci Eng Ethics"},{"key":"1880_CR34","doi-asserted-by":"crossref","unstructured":"Fjeld J, Achten N, Hilligoss H, Nagy A, Srikumar M (2020) Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI. Berkman Klein Center Research Publication 1(2020-1)","DOI":"10.2139\/ssrn.3518482"},{"key":"1880_CR35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s13347-018-0303-9","volume":"31","author":"L Floridi","year":"2018","unstructured":"Floridi L (2018) Soft ethics and the governance of the digital. Philos Technol 31:1\u20138","journal-title":"Philos Technol"},{"key":"1880_CR36","unstructured":"For Coordination CEB (2019) A united nations system-wide strategic approach and road map for supporting capacity development on ai. https:\/\/unsceb.org\/sites\/default\/files\/2020-09\/CEB_2019_1_Add-3-EN_0.pdf. Accessed 18 Aug 2023"},{"key":"1880_CR37","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/7585.001.0001","volume-title":"Value sensitive design: shaping technology with moral imagination","author":"B Friedman","year":"2019","unstructured":"Friedman B, Hendry D (2019) Value sensitive design: shaping technology with moral imagination. MIT Press"},{"key":"1880_CR38","unstructured":"Friedman B, Kahn P, Borning A (2002) Value sensitive design: theory and methods. University of Washington. Tech Rep 2:12"},{"key":"1880_CR39","doi-asserted-by":"crossref","unstructured":"Fuchsberger V, Moser C, Tscheligi M (2012) Values in action (via): Combining usability, user experience and user acceptance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI)","DOI":"10.1145\/2212776.2223711"},{"issue":"3","key":"1880_CR40","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1007\/s11023-020-09539-2","volume":"30","author":"I Gabriel","year":"2020","unstructured":"Gabriel I (2020) Artificial intelligence, values, and alignment. Minds Mach 30(3):411\u2013437","journal-title":"Minds Mach"},{"key":"1880_CR41","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/B978-0-12-801851-4.00018-5","volume-title":"Emotions and affect in human factors and human-computer interaction","author":"A Gaggioli","year":"2017","unstructured":"Gaggioli A, Riva G, Peters D, Calvo RA (2017) Positive technology, computing, and design: shaping a future in which technology promotes psychological well-being. Emotions and affect in human factors and human-computer interaction. Elsevier, pp 477\u2013502"},{"issue":"2","key":"1880_CR42","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1080\/23299460.2022.2069994","volume":"9","author":"J Garst","year":"2022","unstructured":"Garst J, Blok V, Jansen L, Omta O (2022) From value sensitive design to values absorption - building an instrument to analyze organizational capabilities for value-sensitive innovation. J Responsible Innov 9(2):196\u2013223","journal-title":"J Responsible Innov"},{"issue":"12","key":"1880_CR43","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1145\/3458723","volume":"64","author":"T Gebru","year":"2021","unstructured":"Gebru T, Morgenstern J, Vecchione B, Vaughan JW, Wallach H, Iii HD, Crawford K (2021) Datasheets for datasets. Commun ACM 64(12):86\u201392","journal-title":"Commun ACM"},{"key":"1880_CR44","unstructured":"Google (unknown). Responsible ai practices - google ai. https:\/\/ai.google\/responsibilities\/responsible-ai-practices\/. Accessed 02 Aug 2022"},{"key":"1880_CR45","unstructured":"Gupta A, Wright C, Ganapini MB, Sweidan M, Butalid R (2022) State of AI ethics report (volume 6, february 2022). arXiv preprint arXiv:2202.07435"},{"key":"1880_CR46","doi-asserted-by":"crossref","unstructured":"Gutierrez CI, Marchant GE (2021) A global perspective of soft law programs for the governance of artificial intelligence. Available at SSRN 3855171","DOI":"10.2139\/ssrn.3855171"},{"issue":"1","key":"1880_CR47","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s11023-020-09517-8","volume":"30","author":"T Hagendorff","year":"2020","unstructured":"Hagendorff T (2020) The ethics of AI ethics: an evaluation of guidelines. Minds Mach 30(1):99\u2013120","journal-title":"Minds Mach"},{"key":"1880_CR48","doi-asserted-by":"publisher","unstructured":"Harbers M, Overdiek A (2022) Towards a living lab for responsible applied ai. In: Proceedings of the DRS 2022. Retrieved from https:\/\/doi.org\/10.21606\/drs.2022.422","DOI":"10.21606\/drs.2022.422"},{"key":"1880_CR49","doi-asserted-by":"crossref","unstructured":"Hemment D, Aylett R, Belle V, Murray-Rust D, Luger E, Hillston J, Rovatsos M, Broz F (2019) Experiential AI. Computing Research Repository (CoRR). Retrieved from arXiv:1908.02619","DOI":"10.1145\/3320254.3320264"},{"issue":"4","key":"1880_CR50","doi-asserted-by":"crossref","first-page":"263","DOI":"10.2471\/BLT.19.234732","volume":"98","author":"CW Ho","year":"2020","unstructured":"Ho CW, Ali J, Caals K (2020) Ensuring trustworthy use of artificial intelligence and big data analytics in health insurance. Bull World Health Org 98(4):263","journal-title":"Bull World Health Org"},{"key":"1880_CR51","unstructured":"Holden J (2018) Improving nursing student empathy with experiential learning. In: Proceedings of the Nursing Education Research Conference"},{"issue":"4","key":"1880_CR52","doi-asserted-by":"crossref","first-page":"1663","DOI":"10.1007\/s00146-021-01267-0","volume":"37","author":"JC Ib\u00e1\u00f1ez","year":"2021","unstructured":"Ib\u00e1\u00f1ez JC, Olmeda MV (2021) Operationalising ai ethics: how are companies bridging the gap between practice and principles? an exploratory study. AI Soc 37(4):1663\u201387","journal-title":"AI Soc"},{"key":"1880_CR53","unstructured":"IBM (2021) AI ethics IBM. https:\/\/www.ibm.com\/cloud\/learn\/ai-ethics. Accessed 02 Aug 2022"},{"key":"1880_CR54","unstructured":"IEEE standard model process for addressing ethical concerns during system design (Standard) (2021) The Institute of Electrical and Electronics Engineers"},{"key":"1880_CR55","unstructured":"Jacobs AZ (2021) Measurement as governance in and for responsible AI. arXiv preprint arXiv:2109.05658"},{"key":"1880_CR56","doi-asserted-by":"crossref","unstructured":"Jakesch M, Bu\u00e7inca Z, Amershi S, Olteanu A (2022) How different groups prioritize ethical values for responsible AI. In: Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT)","DOI":"10.1145\/3531146.3533097"},{"issue":"9","key":"1880_CR57","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1038\/s42256-019-0088-2","volume":"1","author":"A Jobin","year":"2019","unstructured":"Jobin A, Ienca M, Vayena E (2019) The global landscape of AI ethics guidelines. Nat Mach Intell 1(9):389\u2013399","journal-title":"Nat Mach Intell"},{"issue":"4","key":"1880_CR58","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1007\/s10551-022-05055-8","volume":"178","author":"J-M John-Mathews","year":"2022","unstructured":"John-Mathews J-M, Cardon D, Balagu\u00e9 C (2022) From reality to world. A critical perspective on ai fairness. J Bus Ethics 178(4):945\u201359","journal-title":"J Bus Ethics"},{"key":"1880_CR59","doi-asserted-by":"crossref","unstructured":"Kawakami A, Sivaraman V, Cheng H-F, Stapleton L, Cheng Y, Qing D, Perer A, Wu ZS, Zhu H, Holstein K (2022) Improving human-ai partnerships in child welfare: Understanding worker practices, challenges, and desires for algorithmic decision support. In: CHI Conference on Human Factors in Computing Systems, pp 1\u201318","DOI":"10.1145\/3491102.3517439"},{"key":"1880_CR60","doi-asserted-by":"crossref","unstructured":"Keyes O, Hutson J, Durbin M (2019) A mulching proposal: Analysing and improving an algorithmic system for turning the elderly into high-nutrient slurry. In: Extended abstracts of the 2019 CHI conference on human factors in computing systems, pp 1\u201311","DOI":"10.1145\/3290607.3310433"},{"key":"1880_CR61","unstructured":"Keyes O, Drage E, Mackereth K (2022) Podcast \u2019the good robot\u2019 - os keyes on avoiding universalism and \u2019silver bullets\u2019 in tech design. https:\/\/thegoodrobotpodcast.buzzsprout.com\/1786427\/10692660-os-keyes-on-avoiding-universalism-and-silver-bullets-in-tech-design. Accessed 18 Aug 2022"},{"key":"1880_CR62","doi-asserted-by":"crossref","DOI":"10.1016\/j.ijhcs.2020.102562","volume":"146","author":"L Koesten","year":"2021","unstructured":"Koesten L, Gregory K, Groth P, Simperl E (2021) Talking datasets-understanding data sensemaking behaviours. Int J Hum-Comput Stud 146:102562","journal-title":"Int J Hum-Comput Stud"},{"issue":"4","key":"1880_CR63","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1007\/s00146-021-01256-3","volume":"37","author":"J Krijger","year":"2021","unstructured":"Krijger J (2021) Enter the metrics: critical theory and organizational operationalization of ai ethics. AI Soc 37(4):1427\u201337","journal-title":"AI Soc"},{"key":"1880_CR64","doi-asserted-by":"publisher","unstructured":"Lane G, Angus A, Murdoch A (2018) Unbias fairness toolkit. Zenodo. Retrieved from https:\/\/doi.org\/10.5281\/zenodo.2667808","DOI":"10.5281\/zenodo.2667808"},{"key":"1880_CR65","doi-asserted-by":"crossref","unstructured":"Lee MK, Kusbit D, Kahng A, Kim JT, Yuan X, Chan A, See D, Noothigattu R, Lee S, Psomas A, et al (2019) Webuildai: Participatory framework for algorithmic governance. In: Proceedings of the ACM on Human-Computer Interaction 3(CSCW):1\u201335","DOI":"10.1145\/3359283"},{"issue":"4","key":"1880_CR66","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1007\/s43681-021-00067-y","volume":"1","author":"MSA Lee","year":"2021","unstructured":"Lee MSA, Floridi L, Singh J (2021) Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics. AI Ethics 1(4):529\u2013544","journal-title":"AI Ethics"},{"key":"1880_CR67","unstructured":"Leslie D (2019) Understanding artificial intelligence ethics and safety. https:\/\/www.turing.ac.uk\/sites\/default\/files\/2019-06\/understanding_artificial_intelligence_ethics_and_safety.pdf. Accessed 23 Aug 2023"},{"key":"1880_CR68","first-page":"143","volume-title":"Ethics data and analytics","author":"S Lohr","year":"2018","unstructured":"Lohr S (2018) Facial recognition is accurate, if you\u2019re a white guy. Ethics data and analytics. Auerbach Publications, NY, pp 143\u2013147"},{"key":"1880_CR69","first-page":"37","volume":"10","author":"N Manders-Huits","year":"2009","unstructured":"Manders-Huits N, Zimmer M (2009) Values and pragmatic action: the challenges of introducing ethical intelligence in technical design communities. Int Rev Inf Ethics 10:37\u201344","journal-title":"Int Rev Inf Ethics"},{"key":"1880_CR70","doi-asserted-by":"crossref","unstructured":"McCradden M, Mazwi M, Joshi S, Anderson JA (2020) When your only tool is a hammer: ethical limitations of algorithmic fairness solutions in healthcare machine learning. In: Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, pp 109\u2013109","DOI":"10.1145\/3375627.3375824"},{"key":"1880_CR71","doi-asserted-by":"publisher","unstructured":"McNamara A, Smith J, Murphy-Hill E (2018) Does acm\u2019s code of ethics change ethical decision making in software development? In: Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC\/FSE 2018, New York, NY, USA. Association for Computing Machinery. pp 729-733. Retrieved from https:\/\/doi.org\/10.1145\/3236024.3264833","DOI":"10.1145\/3236024.3264833"},{"key":"1880_CR72","doi-asserted-by":"crossref","unstructured":"Metcalf J, Moss E, Watkins E, Singh R, Elish M (2021) Algorithmic impact assessments and accountability: the co-construction of impacts. In: Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)","DOI":"10.1145\/3442188.3445935"},{"issue":"11","key":"1880_CR73","doi-asserted-by":"crossref","first-page":"501","DOI":"10.1038\/s42256-019-0114-4","volume":"1","author":"B Mittelstadt","year":"2019","unstructured":"Mittelstadt B (2019) Principles alone cannot guarantee ethical AI. Nat Mach Intell 1(11):501\u2013507","journal-title":"Nat Mach Intell"},{"issue":"2","key":"1880_CR74","doi-asserted-by":"crossref","first-page":"205395171667967","DOI":"10.1177\/2053951716679679","volume":"3","author":"B Mittelstadt","year":"2016","unstructured":"Mittelstadt B, Allo P, Taddeo M, Wachter S, Floridi L (2016) The ethics of algorithms: mapping the debate. Big Data Soc 3(2):2053951716679679","journal-title":"Big Data Soc"},{"key":"1880_CR75","doi-asserted-by":"crossref","DOI":"10.1016\/j.eswa.2020.113725","volume":"161","author":"S Modha","year":"2020","unstructured":"Modha S, Majumder P, Mandl T, Mandalia C (2020) Detecting and visualizing hate speech in social media: a cyber watchdog for surveillance. Expert Syst Appl 161:113725","journal-title":"Expert Syst Appl"},{"issue":"2","key":"1880_CR76","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s11023-021-09563-w","volume":"31","author":"J Morley","year":"2021","unstructured":"Morley J, Elhalal A, Garcia F, Kinsey L, M\u00f6kander J, Floridi L (2021) Ethics as a service: a pragmatic operationalisation of ai ethics. Minds Mach 31(2):239\u2013256","journal-title":"Minds Mach"},{"key":"1880_CR77","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1007\/s00146-021-01308-8","volume":"38","author":"J Morley","year":"2021","unstructured":"Morley J, Kinsey L, Elhalal A, Garcia F, Ziosi M, Floridi L (2021) Operationalising ai ethics: barriers, enablers and next steps. AI Soc 38:411\u2013423","journal-title":"AI Soc"},{"key":"1880_CR78","doi-asserted-by":"crossref","unstructured":"Moss E, Watkins E, Metcalf J, Singh R, Elish M (2021) Governing with algorithmic impact assessments: six observations. In: Proceedings of the ACM Conference on Artificial Intelligence, Ethics and Society (AIES)","DOI":"10.2139\/ssrn.3584818"},{"key":"1880_CR79","doi-asserted-by":"crossref","unstructured":"Mujtaba DF, Mahapatra NR (2019) Ethical considerations in ai-based recruitment. In: 2019 IEEE International Symposium on Technology and Society (ISTAS), pp 1\u20137","DOI":"10.1109\/ISTAS48451.2019.8937920"},{"issue":"3","key":"1880_CR80","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1007\/s43681-022-00209-w","volume":"3","author":"L Munn","year":"2023","unstructured":"Munn L (2023) The uselessness of AI ethics. AI Ethics 3(3):869\u2013877","journal-title":"AI Ethics"},{"key":"1880_CR81","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-018-0611-x","author":"Z Munn","year":"2018","unstructured":"Munn Z, Peters M, Stern C (2018) Systematic review or scoping review? guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. https:\/\/doi.org\/10.1186\/s12874-018-0611-x","journal-title":"BMC Med Res Methodol"},{"key":"1880_CR82","unstructured":"Narayanan A (2018) Translation tutorial: 21 fairness definitions and their politics. In: Proc. conf. fairness accountability transp., New York, USA, vol. 1170,p 3"},{"key":"1880_CR83","unstructured":"Observatory OP (2018) Inclusive growth, sustainable development and well-being (OECD AI principle). https:\/\/oecd.ai\/en\/dashboards\/ai-principles\/P5. Accessed 18 Aug 2023"},{"key":"1880_CR84","unstructured":"OECD (2019) Recommendation of the council on artificial intelligence - oecd\/legal\/0449. https:\/\/legalinstruments.oecd.org\/en\/instruments\/OECD-LEGAL-0449. Accessed 11 July 2022"},{"key":"1880_CR85","unstructured":"Office ATP (2022) Statement on prinicples for responsible algorithmic systems. https:\/\/www.acm.org\/binaries\/content\/assets\/public-policy\/final-joint-ai-statement-update.pdf. Accessed 23 Aug 2023"},{"key":"1880_CR86","unstructured":"Organization WH et al (2021) Ethics and governance of artificial intelligence for health: WHO guidance"},{"key":"1880_CR87","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1162\/DESI_a_00401","volume":"32","author":"D Ozkaramanli","year":"2016","unstructured":"Ozkaramanli D, Desmet P, \u00d6zcan E (2016) Beyond resolving dilemmas: three design directions for addressing intrapersonal concern conflicts. Des Issues 32:78\u201391. https:\/\/doi.org\/10.1162\/DESI_a_00401","journal-title":"Des Issues"},{"key":"1880_CR88","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1007\/s00146-020-01097-6","volume":"36","author":"M Pasquinelli","year":"2021","unstructured":"Pasquinelli M, Joler V (2021) The nooscope manifested: AI as instrument of knowledge extractivism. AI Soc 36:1263\u20131280","journal-title":"AI Soc"},{"key":"1880_CR89","unstructured":"Pesenti J (2021) Facebook\u2019s five pillars of responsible AI. https:\/\/ai.facebook.com\/blog\/facebooks-five-pillars-of-responsible-ai\/. Accessed 02 Aug 2022"},{"issue":"1","key":"1880_CR90","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/TTS.2020.2974991","volume":"1","author":"D Peters","year":"2020","unstructured":"Peters D, Vold K, Robinson D, Calvo RA (2020) Responsible ai-two frameworks for ethical design practice. IEEE Trans Technol Soc 1(1):34\u201347","journal-title":"IEEE Trans Technol Soc"},{"issue":"1","key":"1880_CR91","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10676-017-9430-8","volume":"20","author":"I Rahwan","year":"2018","unstructured":"Rahwan I (2018) Society-in-the-loop: programming the algorithmic social contract. Ethics Inf Technol 20(1):5\u201314","journal-title":"Ethics Inf Technol"},{"key":"1880_CR92","doi-asserted-by":"crossref","unstructured":"Rakova B, Yang J, Cramer H, Chowdhury R (2021) Where responsible ai meets reality: Practitioner perspectives on enablers for shifting organizational practices. In: Proceedings of the ACM on Human-Computer Interaction 5(CSCW1):1\u201323","DOI":"10.1145\/3449081"},{"key":"1880_CR93","unstructured":"Reisman D, Schultz J, Crawford K, Whittaker M (2018) Algorithmic impact assessments: a practical framework for public agency accountability"},{"key":"1880_CR94","doi-asserted-by":"crossref","DOI":"10.1016\/j.destud.2023.101230","volume":"89","author":"M Sadek","year":"2023","unstructured":"Sadek M, Calvo R, Mougenot C (2023) Co-designing conversational agents: a comprehensive review and recommendations for best practices. Des Stud 89:101230","journal-title":"Des Stud"},{"key":"1880_CR95","doi-asserted-by":"publisher","DOI":"10.1007\/s43681-023-00373-7","author":"M Sadek","year":"2023","unstructured":"Sadek M, Calvo R, Mougenot C (2023) Designing value-sensitive ai: a critical review and recommendations for socio-technical design processes. AI Ethics. https:\/\/doi.org\/10.1007\/s43681-023-00373-7","journal-title":"AI Ethics"},{"key":"1880_CR96","unstructured":"Sadek M, Calvo R, Mougenot C (2023c) Why codesigning ai is different and difficult. ACM Interactions. Retrieved from https:\/\/interactions.acm.org\/blog\/view\/why-codesigning-ai-is-differentand-difficult"},{"key":"1880_CR97","unstructured":"Scantamburlo T, Cort\u00e9s A, Schacht M (2020) Progressing towards responsible ai. arXiv preprint arXiv:2008.07326"},{"key":"1880_CR98","unstructured":"Schiff D, Rakova B, Ayesh A, Fanti A, Lennon M (2020) Principles to practices for responsible AI: closing the gap. arXiv preprint arXiv:2006.04707"},{"key":"1880_CR99","unstructured":"Simonite T (2021) What really happened when google ousted timnit gebru - wired. https:\/\/www.wired.com\/story\/google-timnit-gebru-ai-what-really-happened\/. Accessed 04 Apr 2023"},{"key":"1880_CR100","doi-asserted-by":"crossref","unstructured":"Spiekermann S (2021) From value-lists to value-based engineering with IEEE 7000$$^{{\\rm TM}}$$. In: IEEE International Symposium on Technology and Society (ISTAS)","DOI":"10.1109\/ISTAS52410.2021.9629134"},{"key":"1880_CR101","doi-asserted-by":"crossref","unstructured":"Spiekermann S, Winkler T (2020) Value-based engineering for ethics by design. Computing Research Repository (CoRR)","DOI":"10.2139\/ssrn.3598911"},{"issue":"4","key":"1880_CR102","doi-asserted-by":"crossref","first-page":"443","DOI":"10.1007\/s42413-020-00086-3","volume":"3","author":"J Stray","year":"2020","unstructured":"Stray J (2020) Aligning ai optimization to community well-being. Int J Commun Well-Being 3(4):443\u2013463","journal-title":"Int J Commun Well-Being"},{"key":"1880_CR103","unstructured":"Stray J, Vendrov I, Nixon J, Adler S, Hadfield-Menell D (2021) What are you optimizing for? aligning recommender systems with human values. arXiv preprint arXiv:2107.10939"},{"key":"1880_CR104","unstructured":"Tasioulas J (2021) The role of the arts and humanities in thinking about artificial intelligence (ai) ada lovelace institute. https:\/\/www.adalovelaceinstitute.org\/blog\/role-arts-humanities-thinking-artificial-intelligence-ai\/. Accessed 04 Aug 2022"},{"issue":"2","key":"1880_CR105","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1162\/daed_a_01912","volume":"151","author":"J Tasioulas","year":"2022","unstructured":"Tasioulas J (2022) Artificial intelligence, humanistic ethics. Daedalus 151(2):232\u2013243. https:\/\/doi.org\/10.1162\/daed_a_01912","journal-title":"Daedalus"},{"key":"1880_CR106","unstructured":"Thomas R, Uminsky D (2020) The problem with metrics is a fundamental problem for ai. arXiv preprint arXiv:2002.08512"},{"issue":"5","key":"1880_CR107","doi-asserted-by":"crossref","DOI":"10.1016\/j.patter.2022.100476","volume":"3","author":"RL Thomas","year":"2022","unstructured":"Thomas RL, Uminsky D (2022) Reliance on metrics is a fundamental challenge for ai. Patterns 3(5):100476","journal-title":"Patterns"},{"key":"1880_CR108","unstructured":"UNESCO (2021) Recommendation on the ethics of artificial intelligence. https:\/\/unesdoc.unesco.org\/ark:\/48223\/pf0000380455. Accessed 18 Aug 2023"},{"key":"1880_CR109","doi-asserted-by":"crossref","unstructured":"Varanasi R, Goyal N (2023) It is currently hodgepodge\u201d: Examining ai\/ml practitioners\u2019 challenges during co-production of responsible ai values. In: Proceedings of the CHI Conference on Human Factors in Computing Systems","DOI":"10.1145\/3544548.3580903"},{"key":"1880_CR110","doi-asserted-by":"crossref","unstructured":"Wachter S, Mittelstadt B (2019) A right to reasonable inferences: re-thinking data protection law in the age of big data and ai. Colum Bus L Rev, p 494","DOI":"10.31228\/osf.io\/mu2kf"},{"key":"1880_CR111","unstructured":"Wagner S (2022) Ai in government featuring stuart wagner, chief digital transformation officer, us air force & us space force. Presentation at Cognilytica. AI and can be accessed at: https:\/\/www.cognilytica.com\/session\/july-2022-ai-in-government\/?hash=62dfc4b9b79db"},{"key":"1880_CR112","unstructured":"West S, Whittaker M, Crawford K (2019) Discriminating systems: Gender, race, and power. Tech Rep. AI Now Institute"},{"key":"1880_CR113","doi-asserted-by":"crossref","unstructured":"Whittlestone J, Nyrup R, Alexandrova A, Cave S (2019) The role and limits of principles in ai ethics: towards a focus on tensions. In: Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society, pp 195\u2013200","DOI":"10.1145\/3306618.3314289"},{"key":"1880_CR114","doi-asserted-by":"publisher","unstructured":"Widder DG, Nafus D, Dabbish L, Herbsleb J (2022) Limits and possibilities for \u201cethical ai\u201d in open source: A study of deepfakes. In: Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT \u201922, New York, NY, USA. Association for Computing Machinery. pp 2035\u20132046. Retrieved from https:\/\/doi.org\/10.1145\/3531146.3533779","DOI":"10.1145\/3531146.3533779"},{"key":"1880_CR115","doi-asserted-by":"crossref","unstructured":"Yang Q, Steinfeld A, Ros\u00e9 C, Zimmerman J (2020) Re-examining whether, why, and how human-ai interaction is uniquely difficult to design. In: Proceedings of the 2020 chi conference on human factors in computing systems, pp 1\u201313","DOI":"10.1145\/3313831.3376301"},{"key":"1880_CR116","doi-asserted-by":"publisher","unstructured":"Yildirim N, Pushkarna M, Goyal N, Wattenberg M, Vi\u00e9gas F (2023) Investigating how practitioners use human-ai guidelines: a case study on the people + ai guidebook. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI \u201923, New York, NY, USA. Association for Computing Machinery. Retrieved from https:\/\/doi.org\/10.1145\/3544548.3580900","DOI":"10.1145\/3544548.3580900"},{"key":"1880_CR117","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/s10676-019-09497-z","volume":"12","author":"M Young","year":"2019","unstructured":"Young M, Magassa L, Friedman B (2019) Toward inclusive tech policy design: a method for underrepresented voices to strengthen tech policy documents. Ethics Inf Technol 12:89\u2013103","journal-title":"Ethics Inf Technol"},{"issue":"2","key":"1880_CR118","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1109\/TTS.2021.3066209","volume":"2","author":"RV Zicari","year":"2021","unstructured":"Zicari RV, Brodersen J, Brusseau J, D\u00fcdder B, Eichhorn T, Ivanov T, Kararigas G, Kringen P, McCullough M, M\u00f6slein F et al (2021) Z-inspection\u00ae: a process to assess trustworthy ai. IEEE Trans Technol Soc 2(2):83\u201397","journal-title":"IEEE Trans Technol Soc"}],"container-title":["AI &amp; SOCIETY"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-024-01880-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00146-024-01880-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00146-024-01880-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T11:45:03Z","timestamp":1739879103000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00146-024-01880-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,19]]},"references-count":118,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["1880"],"URL":"https:\/\/doi.org\/10.1007\/s00146-024-01880-9","relation":{},"ISSN":["0951-5666","1435-5655"],"issn-type":[{"value":"0951-5666","type":"print"},{"value":"1435-5655","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,19]]},"assertion":[{"value":"18 April 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 February 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that there are no conflicts of interest present while producing this work.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of interest"}}]}}