{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T12:33:27Z","timestamp":1776083607936,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":105,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,7,21]],"date-time":"2021-07-21T00:00:00Z","timestamp":1626825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Aarhus University and Aarhus University Research Foundation","award":["AUFF-E-2015-FLS-8-55"],"award-info":[{"award-number":["AUFF-E-2015-FLS-8-55"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,7,21]]},"DOI":"10.1145\/3461702.3462564","type":"proceedings-article","created":{"date-parts":[[2021,7,31]],"date-time":"2021-07-31T01:21:38Z","timestamp":1627694498000},"page":"574-585","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Situated Accountability: Ethical Principles, Certification Standards, and Explanation Methods in Applied AI"],"prefix":"10.1145","author":[{"given":"Anne","family":"Henriksen","sequence":"first","affiliation":[{"name":"Aarhus University, Aarhus, Denmark"}]},{"given":"Simon","family":"Enni","sequence":"additional","affiliation":[{"name":"Aarhus University, Aarhus, Denmark"}]},{"given":"Anja","family":"Bechmann","sequence":"additional","affiliation":[{"name":"Aarhus University, Aarhus, Denmark"}]}],"member":"320","published-online":{"date-parts":[[2021,7,30]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1162\/002081800551280"},{"key":"e_1_3_2_1_2_1","volume-title":"Bridging the Great Divide: Social Science, Technical Systems, and Cooperative Work, edited by G. C. Bowker","author":"Agre P. E.","unstructured":"Agre , P. E. 1997. Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI . In Bridging the Great Divide: Social Science, Technical Systems, and Cooperative Work, edited by G. C. Bowker ; L. Gasser, S.L. Star; and B. Turner . USA : L. Erlbaum Associates Inc . Agre, P. E. 1997. Toward a Critical Technical Practice: Lessons Learned in Trying to Reform AI. In Bridging the Great Divide: Social Science, Technical Systems, and Cooperative Work, edited by G. C. Bowker; L. Gasser, S.L. Star; and B. Turner. USA: L. Erlbaum Associates Inc."},{"issue":"3","key":"e_1_3_2_1_3_1","first-page":"973","article-title":"Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability","volume":"20","author":"Ananny M.","year":"2018","unstructured":"Ananny , M. ; and Crawford , K. 2018 . Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability . Big Data & Society 20 ( 3 ): 973 -- 989 . doi.org\/10.1177\/1461444816676645. Ananny, M.; and Crawford, K. 2018. Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. Big Data & Society 20(3): 973--989. doi.org\/10.1177\/1461444816676645.","journal-title":"Big Data & Society"},{"key":"e_1_3_2_1_4_1","volume-title":"Ethics Guidelines for Trustworthy AI. Report for the European Commission by the High-Level Expert Group on Artificial Intelligence. Report no. B-1049","unstructured":"AI HLEG. 2019. Ethics Guidelines for Trustworthy AI. Report for the European Commission by the High-Level Expert Group on Artificial Intelligence. Report no. B-1049 . Brussels, Belgien . Available at: ec.europa.eu\/digital-single-market\/en\/news\/ethics-guidelines-trustworthy-ai. AI HLEG. 2019. Ethics Guidelines for Trustworthy AI. Report for the European Commission by the High-Level Expert Group on Artificial Intelligence. Report no. B-1049. Brussels, Belgien. Available at: ec.europa.eu\/digital-single-market\/en\/news\/ethics-guidelines-trustworthy-ai."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 648--57","author":"Bhatt U.","unstructured":"Bhatt , U. ; Xiang , A. ; Sharma , S. ; Weller , A. ; Taly , A. ; Jia , Y. ; Ghosh , J. Puri , R. ; Moura , J. M. F. ; and Eckersley , P . 2020. Explainable Machine Learning in Deployment . In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 648--57 . FAT* '20. Barcelona, Spain : Association for Computing Machinery. doi.org\/10.1145\/3351095.3375624. Bhatt, U.; Xiang, A.; Sharma, S.; Weller, A.; Taly, A.; Jia, Y.; Ghosh, J. Puri, R.; Moura, J. M. F.; and Eckersley, P. 2020. Explainable Machine Learning in Deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 648--57. FAT* '20. Barcelona, Spain: Association for Computing Machinery. doi.org\/10.1145\/3351095.3375624."},{"key":"e_1_3_2_1_7_1","first-page":"325","article-title":"The New Role for Assurance Services in Global Commerce","volume":"33","author":"Blair M. M.","year":"2008","unstructured":"Blair , M. M. 2008 . The New Role for Assurance Services in Global Commerce . Journal of Corporation Law 33 : 325 -- 360 . Available at: scholar-ship.law.vanderbilt.edu\/faculty-publications\/30. Blair, M. M. 2008. The New Role for Assurance Services in Global Commerce. Journal of Corporation Law 33: 325--360. Available at: scholar-ship.law.vanderbilt.edu\/faculty-publications\/30.","journal-title":"Journal of Corporation Law"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1080\/01402382.2010.486119"},{"issue":"1","key":"e_1_3_2_1_9_1","first-page":"1","article-title":"Public Accountability. The Oxford Handbook of Public Accountability, edited by M. Bovens; and R","volume":"1","author":"Bovens M.","year":"2014","unstructured":"Bovens , M. ; Schillemans , T. ; and Goodin , R. E. 2014 . Public Accountability. The Oxford Handbook of Public Accountability, edited by M. Bovens; and R . E. Goodin 1 ( 1 ): 1 -- 22 . doi: 10.1093\/oxfordhb\/9780199641253.001.0001. Bovens, M.; Schillemans, T.; and Goodin, R. E. 2014. Public Accountability. The Oxford Handbook of Public Accountability, edited by M. Bovens; and R. E. Goodin 1(1): 1--22. doi: 10.1093\/oxfordhb\/9780199641253.001.0001.","journal-title":"E. Goodin"},{"key":"e_1_3_2_1_10_1","volume-title":"Artificial intelligence and responsible innovation. Fundamental Issues of Artificial Intelligence, edited by V. E. M\u00fcller, 543--555","author":"Brundage M.","unstructured":"Brundage , M. 2016. Artificial intelligence and responsible innovation. Fundamental Issues of Artificial Intelligence, edited by V. E. M\u00fcller, 543--555 . Switzerland : Springer International Publishing . Brundage, M. 2016. Artificial intelligence and responsible innovation. Fundamental Issues of Artificial Intelligence, edited by V. E. M\u00fcller, 543--555. Switzerland: Springer International Publishing."},{"key":"e_1_3_2_1_11_1","volume-title":"et al","author":"Brundage M.","year":"2020","unstructured":"Brundage , M. ; Avin , S. ; Wang , J. ; Belfield , H. ; Krueger , G. ; Hadfield , G. ; Khlaaf , H. et al . 2020 . Toward Trust-worthy AI Development: Mechanisms for Supporting Verifiable Claims. arXiv preprint. ArXiv :2004.07213 [Cs.CY]. Available at: arxiv.org\/abs\/2004.07213. Brundage, M.; Avin, S.; Wang, J.; Belfield, H.; Krueger, G.; Hadfield, G.; Khlaaf, H. et al. 2020. Toward Trust-worthy AI Development: Mechanisms for Supporting Verifiable Claims. arXiv preprint. ArXiv:2004.07213 [Cs.CY]. Available at: arxiv.org\/abs\/2004.07213."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10676-018-9448-6"},{"key":"e_1_3_2_1_13_1","volume-title":"No One Should Trust AI","author":"Bryson J.","year":"2018","unstructured":"Bryson , J. 2018b. AI & Global Governance : No One Should Trust AI . United Nations University Centre for Policy Research ( blog). 2018 . Available at: cpr.unu.edu\/ai-global-governance-no-one-should-trust-ai.html. Bryson, J. 2018b. AI & Global Governance: No One Should Trust AI. United Nations University Centre for Policy Research (blog). 2018. Available at: cpr.unu.edu\/ai-global-governance-no-one-should-trust-ai.html."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2017.154"},{"key":"e_1_3_2_1_15_1","first-page":"77","article-title":"Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of Machine Learning Research, edited by A. F. Sorelle; and C","volume":"81","author":"Buolamwini J.","year":"2018","unstructured":"Buolamwini , J. ; and Gebru , T. 2018 . Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of Machine Learning Research, edited by A. F. Sorelle; and C . Wilson , 81 : 77 -- 91 . New York, NY, USA: PMLR. proceedings.mlr.press\/v81\/buolamwini18a.html. Buolamwini, J.; and Gebru, T. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of Machine Learning Research, edited by A. F. Sorelle; and C. Wilson, 81: 77--91. New York, NY, USA: PMLR. proceedings.mlr.press\/v81\/buolamwini18a.html.","journal-title":"Wilson"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715622512"},{"key":"e_1_3_2_1_17_1","volume-title":"Princeton, N.J.: Princeton University Press.","author":"B\u00fcthe T.","year":"2011","unstructured":"B\u00fcthe , T. ; and Mattli , W . 2011 . The New Global Rulers: The Privatization of Regulation in the World Economy. Oxford ; Princeton, N.J.: Princeton University Press. B\u00fcthe, T.; and Mattli, W. 2011. The New Global Rulers: The Privatization of Regulation in the World Economy. Oxford; Princeton, N.J.: Princeton University Press."},{"key":"e_1_3_2_1_18_1","volume-title":"Cham: Springer International Publishing. doi.org\/10.1007\/978--3--319--90503--7_10.","author":"Cabitza F.","year":"2019","unstructured":"Cabitza , F. ; Ciucci , D. ; and Rasoini , R . 2019 . A Giant with Feet of Clay: On the Validity of the Data That Feed Machine Learning in Medicine. Organizing for the Digital World, edited by F. Cabitza; C. Batini; and M. Magni, 121--36. Lecture Notes in Information Systems and Organisation . Cham: Springer International Publishing. doi.org\/10.1007\/978--3--319--90503--7_10. Cabitza, F.; Ciucci, D.; and Rasoini, R. 2019. A Giant with Feet of Clay: On the Validity of the Data That Feed Machine Learning in Medicine. Organizing for the Digital World, edited by F. Cabitza; C. Batini; and M. Magni, 121--36. Lecture Notes in Information Systems and Organisation. Cham: Springer International Publishing. doi.org\/10.1007\/978--3--319--90503--7_10."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2017.7797"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3430368"},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15: 1721--30","author":"Caruana R.","unstructured":"Caruana , R. ; Lou , Y. ; Gehrke , J. ; Koch , P. ; Sturm , M. ; and Elhadad , N . 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Read-mission . In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15: 1721--30 . Sydney, NSW, Australia : Association for Computing Machinery. doi.org\/10.1145\/2783258.2788613. Caruana, R.; Lou, Y.; Gehrke, J.; Koch, P.; Sturm, M.; and Elhadad, N. 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Read-mission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '15: 1721--30. Sydney, NSW, Australia: Association for Computing Machinery. doi.org\/10.1145\/2783258.2788613."},{"key":"e_1_3_2_1_22_1","unstructured":"CEN-CENELEC. 2020. CEN-CENELEC Focus Group Report: Road Map on Artificial Intelligence (AI). CEN-CENELEC Road Map Report on AI version 2020-09. Available at: ftp.cencenelec.eu\/EN\/EuropeanStandardization\/Sectors\/AI\/CEN-CLC_FGR_RoadMapAI.pdf.  CEN-CENELEC. 2020. CEN-CENELEC Focus Group Report: Road Map on Artificial Intelligence (AI). CEN-CENELEC Road Map Report on AI version 2020-09. Available at: ftp.cencenelec.eu\/EN\/EuropeanStandardization\/Sectors\/AI\/CEN-CLC_FGR_RoadMapAI.pdf."},{"key":"e_1_3_2_1_23_1","first-page":"11","article-title":"The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Robotics and Well-Being, edited by M. I. A. Ferreira; J. S. Sequeira; G. S. Virk; M. O. Tokhi; and E","volume":"95","author":"Chatila R.","year":"2019","unstructured":"Chatila , R. ; and Havens , J. C. 2019 . The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Robotics and Well-Being, edited by M. I. A. Ferreira; J. S. Sequeira; G. S. Virk; M. O. Tokhi; and E . E. Kadar , 95 : 11 -- 16 . Cham: Springer International Publishing. doi.org\/10.1007\/978--3-030--12524-0_2. Chatila, R.; and Havens, J. C. 2019. The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems. Robotics and Well-Being, edited by M. I. A. Ferreira; J. S. Sequeira; G. S. Virk; M. O. Tokhi; and E. E. Kadar, 95: 11--16. Cham: Springer International Publishing. doi.org\/10.1007\/978--3-030--12524-0_2.","journal-title":"E. Kadar"},{"key":"e_1_3_2_1_24_1","volume-title":"review. AI Certification: Advancing Practice by Reducing Information Asymmetries","author":"Cihon P.","unstructured":"Cihon , P. ; Kleinaltenkamp , M. J. ; Schuett , J. ; and Baum , S. D . In review. AI Certification: Advancing Practice by Reducing Information Asymmetries . IEEE Transactions on Technology and Society . Cihon, P.; Kleinaltenkamp, M. J.; Schuett, J.; and Baum, S. D. In review. AI Certification: Advancing Practice by Reducing Information Asymmetries. IEEE Transactions on Technology and Society."},{"key":"e_1_3_2_1_25_1","volume-title":"Reflexive Ethnography -- A Guide to Researching Selves and Others. London &","author":"Davies C. A.","unstructured":"Davies , C. A. 2008. Reflexive Ethnography -- A Guide to Researching Selves and Others. London & New York : Routledge . Davies, C. A. 2008. Reflexive Ethnography -- A Guide to Researching Selves and Others. London & New York: Routledge."},{"key":"e_1_3_2_1_26_1","volume-title":"The Law, Economics and Politics of International Standardisation","author":"Delimatsis P.","unstructured":"Delimatsis , P. 2015. The Law, Economics and Politics of International Standardisation . Cambridge International Trade and Economic Law. Cambridge , United Kingdom : Cambridge University Press . Delimatsis, P. 2015. The Law, Economics and Politics of International Standardisation. Cambridge International Trade and Economic Law. Cambridge, United Kingdom: Cambridge University Press."},{"key":"e_1_3_2_1_27_1","volume-title":"The SAGE Handbook of Qualitative Data Analysis, edited by U. Flick, 569--584","author":"Denzin N. K.","unstructured":"Denzin , N. K. 2013. Writing and\/as Analysis or Performing the World. The SAGE Handbook of Qualitative Data Analysis, edited by U. Flick, 569--584 . London, UK : Sage Publications . Denzin, N. K. 2013. Writing and\/as Analysis or Performing the World. The SAGE Handbook of Qualitative Data Analysis, edited by U. Flick, 569--584. London, UK: Sage Publications."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1161\/CIRCULATIONAHA.115.001593"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1080\/21670811.2014.976411"},{"key":"e_1_3_2_1_30_1","volume-title":"et al","author":"Diakopoulos N.","year":"2017","unstructured":"Diakopoulos , N. ; Friedler , S. ; Arenas , M. ; Barocas , S. ; Hay , M. ; Howe , B. ; Jagadish , H. V. et al . 2017 . Principles for Accountable Algorithms and a Social Impact Statement for Algorithms. FAT\/ML. Available at: fatml.org\/resources\/principles-for-accountable-algorithms. Diakopoulos, N.; Friedler, S.; Arenas, M.; Barocas, S.; Hay, M.; Howe, B.; Jagadish, H. V. et al. 2017. Principles for Accountable Algorithms and a Social Impact Statement for Algorithms. FAT\/ML. Available at: fatml.org\/resources\/principles-for-accountable-algorithms."},{"key":"e_1_3_2_1_31_1","volume-title":"Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way","author":"Dignum V.","unstructured":"Dignum , V. 2019. Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way . Cham, Switzerland : Springer . doi.org\/10.1007\/978--3-030--30371--6. Dignum, V. 2019. Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way. Cham, Switzerland: Springer. doi.org\/10.1007\/978--3-030--30371--6."},{"key":"e_1_3_2_1_32_1","volume-title":"et al","author":"Doshi-Velez F.","year":"2019","unstructured":"Doshi-Velez , F. , Kortz , M. ; Budish , R. ; Bavitz , C. ; Gershman , S. ; O'Brien , D. ; Scott , K. et al . 2019 . Accountability of AI Under the Law : The Role of Explana-tion. arXiv preprint. ArXiv :1711.01134 [Cs.AI]. Available at: arxiv.org\/abs\/1711.01134. Doshi-Velez, F., Kortz, M.; Budish, R.; Bavitz, C.; Gershman, S.; O'Brien, D.; Scott, K. et al. 2019. Accountability of AI Under the Law: The Role of Explana-tion. arXiv preprint. ArXiv:1711.01134 [Cs.AI]. Available at: arxiv.org\/abs\/1711.01134."},{"key":"e_1_3_2_1_33_1","first-page":"160","article-title":"Runaway Feedback Loops in Predictive Policing. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency","volume":"81","author":"Ensign D.","year":"2018","unstructured":"Ensign , D. ; Friedler , S. A. ; Neville , S. ; Scheidegger , C. ; and Venkatasubramanian , S. 2018 . Runaway Feedback Loops in Predictive Policing. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency , PMLR 81 : 160 -- 171 . proceedings.mlr.press\/v81\/ensign18a.html. Ensign, D.; Friedler, S. A.; Neville, S.; Scheidegger, C.; and Venkatasubramanian, S. 2018. Runaway Feedback Loops in Predictive Policing. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR 81: 160--71. proceedings.mlr.press\/v81\/ensign18a.html.","journal-title":"PMLR"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature21056"},{"key":"e_1_3_2_1_35_1","volume-title":"Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor.","author":"Eubanks V.","year":"2018","unstructured":"Eubanks , V. 2018 . Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York, NY, USA : St. Martin's Press . Eubanks, V. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York, NY, USA: St. Martin's Press."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-017-0291-1"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Floridi L. 2018. Soft Ethics the Governance of the Digi-tal and the General Data Protection Regulation. Philosophical Transactions of the Royal Society A: Mathematical Physical and Engineering Sciences 376(2133). doi.org\/10.1098\/rsta.2018.0081.  Floridi L. 2018. Soft Ethics the Governance of the Digi-tal and the General Data Protection Regulation. Philosophical Transactions of the Royal Society A: Mathematical Physical and Engineering Sciences 376(2133). doi.org\/10.1098\/rsta.2018.0081.","DOI":"10.1098\/rsta.2018.0081"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-019-00354-x"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-018-9482-5"},{"key":"e_1_3_2_1_40_1","volume-title":"Stanford, CA: Stanford University Press.","author":"Forsythe D.","year":"2001","unstructured":"Forsythe , D. ; and Hess , D. J . 2001 . Studying Those Who Study Us: An Anthropologist in the World of Artificial Intelligence. Writing Science . Stanford, CA: Stanford University Press. Forsythe, D.; and Hess, D. J. 2001. Studying Those Who Study Us: An Anthropologist in the World of Artificial Intelligence. Writing Science. Stanford, CA: Stanford University Press."},{"key":"e_1_3_2_1_41_1","volume-title":"The Interpretation of Cultures","author":"Geertz C.","unstructured":"Geertz , C. 1973. The Interpretation of Cultures . New York, NY, USA : Basic Books . Geertz, C. 1973. The Interpretation of Cultures. New York, NY, USA: Basic Books."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013681"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v38i3.2741"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Guidotti R.; Monreale A.; Ruggieri S.; Turini F.; Giannotti F.; and Pedreschi D. 2018. A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys 51(5) Article 93: 1--42. doi.org\/10.1145\/3236009.  Guidotti R.; Monreale A.; Ruggieri S.; Turini F.; Giannotti F.; and Pedreschi D. 2018. A Survey of Methods for Explaining Black Box Models. ACM Computing Surveys 51(5) Article 93: 1--42. doi.org\/10.1145\/3236009.","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2016.17216"},{"key":"e_1_3_2_1_46_1","unstructured":"DARPA. 2017. Broad Agency Announcement: Explain-able Artificial Intelligence (XAI) DARPA-BAA-16--53. Available at: darpa.mil\/attachments\/DARPA-BAA-16--53.pdf.  DARPA. 2017. Broad Agency Announcement: Explain-able Artificial Intelligence (XAI) DARPA-BAA-16--53. Available at: darpa.mil\/attachments\/DARPA-BAA-16--53.pdf."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-020-09517-8"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1177\/1077800411423194"},{"key":"e_1_3_2_1_49_1","unstructured":"ISO. 2013. ISO\/IEC 27001:2013. International Organization for Standardization. Available at: iso.org\/isoiec-27001-information-security.html.  ISO. 2013. ISO\/IEC 27001:2013. International Organization for Standardization. Available at: iso.org\/isoiec-27001-information-security.html."},{"key":"e_1_3_2_1_50_1","unstructured":"ISO. 2016. ISO 13485:2016 Medical Devices - Quality Management Systems - Requirements for Regulatory Purposes. International Organization for Standardization. Available at: iso.org\/standard\/59752.html.  ISO. 2016. ISO 13485:2016 Medical Devices - Quality Management Systems - Requirements for Regulatory Purposes. International Organization for Standardization. Available at: iso.org\/standard\/59752.html."},{"key":"e_1_3_2_1_51_1","volume-title":"Consumer Standards: Partnership for a Better World","author":"ISO.","year":"2021","unstructured":"ISO. 2021 a. Consumer Standards: Partnership for a Better World . International Organization for Standardization . Available at: iso.org\/sites\/ConsumersStandards\/index.html#top ISO. 2021a. Consumer Standards: Partnership for a Better World. International Organization for Standardization. Available at: iso.org\/sites\/ConsumersStandards\/index.html#top"},{"key":"e_1_3_2_1_52_1","unstructured":"ISO. 2021b. Popular Standards. International Organization for Standardization. Available at: iso.org\/popular-standards.html.  ISO. 2021b. Popular Standards. International Organization for Standardization. Available at: iso.org\/popular-standards.html."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1177\/0306312717730428"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0088-2"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1161\/01.CIR.51.4.606"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1080\/1369118X.2018.1477967"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2016.07.007"},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2018.0084"},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Kroll J. A. 2020. Accountability in Computer Systems. The Oxford Handbook of Ethics of AI 181.  Kroll J. A. 2020. Accountability in Computer Systems. The Oxford Handbook of Ethics of AI 181.","DOI":"10.1093\/oxfordhb\/9780190067397.013.10"},{"key":"e_1_3_2_1_60_1","first-page":"74","article-title":"Ac-countable Algorithms","volume":"165","author":"Kroll J. A.","year":"2017","unstructured":"Kroll , J. A. ; Huey , J. ; Barocas , S. ; Felten , E. W. ; Reidenberg , J. R. ; Robinson , D. G. ; and Yu , H.. 2017 . Ac-countable Algorithms . University of Pennsylvania Law Review 165 : 74 . Kroll, J. A.; Huey, J.; Barocas, S.; Felten, E. W.; Reidenberg, J. R.; Robinson, D. G.; and Yu, H.. 2017. Ac-countable Algorithms. University of Pennsylvania Law Review 165: 74.","journal-title":"University of Pennsylvania Law Review"},{"key":"e_1_3_2_1_61_1","volume-title":"Doing interviews","author":"Kvale S.","year":"2008","unstructured":"Kvale , S. ( 2008 ). Doing interviews . London, UK : Sage Publications . Kvale, S. (2008). Doing interviews. London, UK: Sage Publications."},{"key":"e_1_3_2_1_62_1","volume-title":"Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, 79--85. AIES '20","author":"Lakkaraju H.","unstructured":"Lakkaraju , H. ; and Bastani , O . 2020. 'How Do I Fool You?': Manipulating User Trust via Misleading Black Box Explanations . In Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, 79--85. AIES '20 . New York, NY, USA: Association for Computing Machinery. doi.org\/10.1145\/3375627.3375833. Lakkaraju, H.; and Bastani, O. 2020. 'How Do I Fool You?': Manipulating User Trust via Misleading Black Box Explanations. In Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, 79--85. AIES '20. New York, NY, USA: Association for Computing Machinery. doi.org\/10.1145\/3375627.3375833."},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1017\/als.2020.19"},{"key":"e_1_3_2_1_64_1","volume-title":"Science in action: How to follow scientists and engineers through society","author":"Latour B.","unstructured":"Latour , B. 1987. Science in action: How to follow scientists and engineers through society . Harvard University Press . Latour, B. 1987. Science in action: How to follow scientists and engineers through society. Harvard University Press."},{"key":"e_1_3_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13347-017-0279-x"},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236386.3241340"},{"key":"e_1_3_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1177\/1403494812465030"},{"key":"e_1_3_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10551-018-3921-3"},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2018.07.007"},{"key":"e_1_3_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0114-4"},{"key":"e_1_3_2_1_71_1","volume-title":"Proceedings of the Con-ference on Fairness, Accountability, and Transparency - FAT* '19","author":"Mittelstadt B.","unstructured":"Mittelstadt , B. ; Russell , C. ; and Wachter , S . 2019. Ex-plaining Explanations in AI . In Proceedings of the Con-ference on Fairness, Accountability, and Transparency - FAT* '19 , 279--88. Atlanta, GA, USA: ACM Press. doi.org\/10.1145\/3287560.3287574. Mittelstadt, B.; Russell, C.; and Wachter, S. 2019. Ex-plaining Explanations in AI. In Proceedings of the Con-ference on Fairness, Accountability, and Transparency - FAT* '19, 279--88. Atlanta, GA, USA: ACM Press. doi.org\/10.1145\/3287560.3287574."},{"key":"e_1_3_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.11.008"},{"key":"e_1_3_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(19)30079-2"},{"key":"e_1_3_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/175222.175228"},{"key":"e_1_3_2_1_75_1","unstructured":"OECD. 2020. OECD Recommendation of the Council on Artificial Intelligence OECD\/LEGAL\/0449. Organisation for Economic Co-operation and Development.  OECD. 2020. OECD Recommendation of the Council on Artificial Intelligence OECD\/LEGAL\/0449. Organisation for Economic Co-operation and Development."},{"key":"e_1_3_2_1_76_1","volume-title":"Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy","author":"O'Neil C.","unstructured":"O'Neil , C. 2017. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy . London : Penguin Books . O'Neil, C. 2017. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. London: Penguin Books."},{"key":"e_1_3_2_1_77_1","volume-title":"The Audit Society: Rituals of Verification","author":"Power M.","unstructured":"Power , M. 1997. The Audit Society: Rituals of Verification . Oxford, [England]; New York : Oxford University Press . Power, M. 1997. The Audit Society: Rituals of Verification. Oxford, [England]; New York: Oxford University Press."},{"key":"e_1_3_2_1_78_1","doi-asserted-by":"crossref","unstructured":"Raji I. D.; Smart A.; White R. N.; Mitchell M.; Gebru T.; Hutchinson B.; Smith-Loud J.; Theron D.; and Barnes P.. 2020. Closing the AI Accountability Gap: De-fining an End-to-End Framework for Internal Algorithmic Auditing. ArXiv:2001.00973 [Cs] January. Available at: arxiv.org\/abs\/2001.00973.  Raji I. D.; Smart A.; White R. N.; Mitchell M.; Gebru T.; Hutchinson B.; Smith-Loud J.; Theron D.; and Barnes P.. 2020. Closing the AI Accountability Gap: De-fining an End-to-End Framework for Internal Algorithmic Auditing. ArXiv:2001.00973 [Cs] January. Available at: arxiv.org\/abs\/2001.00973.","DOI":"10.1145\/3351095.3372873"},{"key":"e_1_3_2_1_79_1","unstructured":"Ress\u00e9guier A.; and Rodrigues R. 2020. AI ethics should not remain toothless! A call to bring back the teeth of ethics. Big Data & Society 1--5. doi.org\/10.1177\/2053951720942541  Ress\u00e9guier A.; and Rodrigues R. 2020. AI ethics should not remain toothless! A call to bring back the teeth of ethics. Big Data & Society 1--5. doi.org\/10.1177\/2053951720942541"},{"key":"e_1_3_2_1_80_1","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135--44. KDD '16","author":"Ribeiro M. T.","unstructured":"Ribeiro , M. T. ; Singh , S. ; and Guestrin , C . 2016. ?Why Should I Trust You?': Explaining the Predictions of Any Classifier . In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135--44. KDD '16 . New York, NY, USA: Association for Computing Machinery. doi.org\/10.1145\/2939672.2939778. Ribeiro, M. T.; Singh, S.; and Guestrin, C. 2016. ?Why Should I Trust You?': Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135--44. KDD '16. New York, NY, USA: Association for Computing Machinery. doi.org\/10.1145\/2939672.2939778."},{"key":"e_1_3_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0048-x"},{"key":"e_1_3_2_1_82_1","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell S. J.","year":"2016","unstructured":"Russell , S. J. ; and Norvig , P . 2016 . Artificial Intelligence: A Modern Approach , 3 rd Edition. Pearson Education , 2009. Russell, S. J.; and Norvig, P. 2016. Artificial Intelligence: A Modern Approach, 3rd Edition. Pearson Education, 2009.","edition":"3"},{"key":"e_1_3_2_1_83_1","volume-title":"Machine Learning: The High Interest Credit Card of Technical Debt. In SE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop).","author":"Sculley D.","year":"2014","unstructured":"Sculley , D. ; Holt , G. ; Golovin , D. ; Davydov , E. ; Phil -lips, T.; Ebner , D. ; Chaudhary , V. ; and Young , M . 2014 . Machine Learning: The High Interest Credit Card of Technical Debt. In SE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop). Sculley, D.; Holt, G.; Golovin, D.; Davydov, E.; Phil-lips, T.; Ebner, D.; Chaudhary, V.; and Young, M. 2014. Machine Learning: The High Interest Credit Card of Technical Debt. In SE4ML: Software Engineering for Machine Learning (NIPS 2014 Workshop)."},{"key":"e_1_3_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2017.2767063"},{"key":"e_1_3_2_1_85_1","unstructured":"Shwartz-Ziv R.; and Tishby N. 2017. Opening the Black Box of Deep Neural Networks via Information. ArXiv:1703.00810 [Cs]. Available at: arxiv.org\/abs\/1703.00810.  Shwartz-Ziv R.; and Tishby N. 2017. Opening the Black Box of Deep Neural Networks via Information. ArXiv:1703.00810 [Cs]. Available at: arxiv.org\/abs\/1703.00810."},{"key":"e_1_3_2_1_86_1","volume-title":"The Chameleon of Accountability: Forms and Discourses. Accounting","author":"Sinclair A.","unstructured":"Sinclair , A. 1995. The Chameleon of Accountability: Forms and Discourses. Accounting , Organizations and Society 20 (2--3): 219--37. doi.org\/10.1016\/0361--3682(93)E0003-Y. Sinclair, A. 1995. The Chameleon of Accountability: Forms and Discourses. Accounting, Organizations and Society 20 (2--3): 219--37. doi.org\/10.1016\/0361--3682(93)E0003-Y."},{"key":"e_1_3_2_1_87_1","volume-title":"Clinical AI: Opacity, Accountability, Responsibility and Liability. AI & SOCIETY, July. doi.org\/10.1007\/s00146-020-01019--6.","author":"Smith H.","year":"2020","unstructured":"Smith , H. 2020 . Clinical AI: Opacity, Accountability, Responsibility and Liability. AI & SOCIETY, July. doi.org\/10.1007\/s00146-020-01019--6. Smith, H. 2020. Clinical AI: Opacity, Accountability, Responsibility and Liability. AI & SOCIETY, July. doi.org\/10.1007\/s00146-020-01019--6."},{"key":"e_1_3_2_1_88_1","doi-asserted-by":"crossref","unstructured":"Sossin L.; and Smith C. W. 2003. Hard Choices and Soft Law: Ethical Codes Policy Guidelines and the Role of the Courts in Regulating Government. Alberta Law Re-view: 867--89. doi.org\/10.29173\/alr1344.  Sossin L.; and Smith C. W. 2003. Hard Choices and Soft Law: Ethical Codes Policy Guidelines and the Role of the Courts in Regulating Government. Alberta Law Re-view: 867--89. doi.org\/10.29173\/alr1344.","DOI":"10.29173\/alr1344"},{"key":"e_1_3_2_1_89_1","volume-title":"Participant Observation","author":"Spradley J. P.","unstructured":"Spradley , J. P. 1980. Participant Observation . New York : Holt, Rinehart and Winston . Spradley, J. P. 1980. Participant Observation. New York: Holt, Rinehart and Winston."},{"key":"e_1_3_2_1_90_1","volume-title":"The SAGE Handbook of Qualitative Research","author":"St. Pierre E. A.","unstructured":"St. Pierre , E. A. 2011. Post qualitative research: The critique and the coming after . In The SAGE Handbook of Qualitative Research , edited by A. K. Denzin and Y. S. Lincoln, 4 th Edition, 11--25. London, UK : Sage Publication . St. Pierre, E. A. 2011. Post qualitative research: The critique and the coming after. In The SAGE Handbook of Qualitative Research, edited by A. K. Denzin and Y. S. Lincoln, 4th Edition, 11--25. London, UK: Sage Publication.","edition":"4"},{"key":"e_1_3_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.5555\/38407"},{"key":"e_1_3_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1145\/2460276.2460278"},{"key":"e_1_3_2_1_93_1","unstructured":"the European Commission. 2020. Commission Implementing Decision (EU) 2020\/437. Available at: eur-lex.europa.eu\/eli\/dec_impl\/2020\/437\/oj.  the European Commission. 2020. Commission Implementing Decision (EU) 2020\/437. Available at: eur-lex.europa.eu\/eli\/dec_impl\/2020\/437\/oj."},{"key":"e_1_3_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-019-0136-y"},{"key":"e_1_3_2_1_95_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951720907107"},{"key":"e_1_3_2_1_96_1","volume-title":"An FDA for Algorithms. 69 Admin. L. Rev. 83","author":"Tutt A.","unstructured":"Tutt , A. 2016. An FDA for Algorithms. 69 Admin. L. Rev. 83 . Rochester, NY : Social Science Research Network . Available at: papers.ssrn.com\/abstract=2747994. Tutt, A. 2016. An FDA for Algorithms. 69 Admin. L. Rev. 83. Rochester, NY: Social Science Research Network. Available at: papers.ssrn.com\/abstract=2747994."},{"key":"e_1_3_2_1_97_1","doi-asserted-by":"crossref","unstructured":"Wachter S.; Mittelstadt B.; and Floridi L. 2017. Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation. International Data Privacy Law (7): 76--99. Ox-ford University Press. doi.org\/10.1609\/aimag.v38i3.2741.  Wachter S.; Mittelstadt B.; and Floridi L. 2017. Why a Right to Explanation of Automated Decision-Making Does Not Exist in the General Data Protection Regulation. International Data Privacy Law (7): 76--99. Ox-ford University Press. doi.org\/10.1609\/aimag.v38i3.2741.","DOI":"10.1093\/idpl\/ipx005"},{"key":"e_1_3_2_1_98_1","volume-title":"Transparency: Motivations and Chal-lenges. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"Weller A.","year":"2019","unstructured":"Weller , A. 2019 . Transparency: Motivations and Chal-lenges. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning , edited by Samek, W.; Montavon, G.; Vedaldi, A.; Hansen, L.; M\u00fcller, K. R. 23--40. Lecture Notes in Computer Science . Cham : Springer In-ternational Publishing . doi.org\/10.1007\/978--3-030--28954--6_2. Weller, A. 2019. Transparency: Motivations and Chal-lenges. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, edited by Samek, W.; Montavon, G.; Vedaldi, A.; Hansen, L.; M\u00fcller, K. R. 23--40. Lecture Notes in Computer Science. Cham: Springer In-ternational Publishing. doi.org\/10.1007\/978--3-030--28954--6_2."},{"key":"e_1_3_2_1_99_1","unstructured":"Whittaker M.; Crawford K.; Dobbe R.; Fried G.; Kazi-unas E.; Mathur V.; West S. M.; Richardson R.; Schultz J.; and Schwartz O. 2018. AI Now Report 2018. AI Now Institute at New York University New York.  Whittaker M.; Crawford K.; Dobbe R.; Fried G.; Kazi-unas E.; Mathur V.; West S. M.; Richardson R.; Schultz J.; and Schwartz O. 2018. AI Now Report 2018. AI Now Institute at New York University New York."},{"key":"e_1_3_2_1_100_1","volume-title":"Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society (AIES '19)","author":"Whittlestone J.","unstructured":"Whittlestone , J. ; Nyrup , R. ; Alexandrova , A. ; and Cave , S . 2019. The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions , In Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society (AIES '19) : 195--200. Association for Computing Machinery, New York, NY, USA. doi.org\/10.1145\/3306618.3314289 Whittlestone, J.; Nyrup, R.; Alexandrova, A.; and Cave, S. 2019. The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions, In Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society (AIES '19): 195--200. Association for Computing Machinery, New York, NY, USA. doi.org\/10.1145\/3306618.3314289"},{"key":"e_1_3_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372833"},{"key":"e_1_3_2_1_102_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2018.0085"},{"key":"e_1_3_2_1_103_1","volume-title":"Case Study Research: Design and Methods","author":"Yin R. K.","unstructured":"Yin , R. K. 2009. Case Study Research: Design and Methods , 4 th Edition. Los Angeles, LA, USA : Sage Publications . Yin, R. K. 2009. Case Study Research: Design and Methods, 4th Edition. Los Angeles, LA, USA: Sage Publications.","edition":"4"},{"key":"e_1_3_2_1_104_1","doi-asserted-by":"crossref","unstructured":"Zerilli F. M. 2010. The Rule of Soft Law. Focaal-Journal of Global and Historical Anthropology (56): 3--18. doi.org\/10.3167\/fcl.2010.560101.  Zerilli F. M. 2010. The Rule of Soft Law. Focaal-Journal of Global and Historical Anthropology (56): 3--18. doi.org\/10.3167\/fcl.2010.560101.","DOI":"10.3167\/fcl.2010.560101"},{"key":"e_1_3_2_1_105_1","article-title":"Ma-chine Learning Testing: Survey, Landscapes and Horizons","volume":"10","author":"Zhang J. M.","year":"2019","unstructured":"Zhang , J. M. ; Harman , M. ; Ma , L. ; and Liu , Y. 2019 . Ma-chine Learning Testing: Survey, Landscapes and Horizons . IEEE Transactions on Software Engineering. doi : 10 .1109\/TSE.2019.2962027. Zhang, J. M.; Harman, M.; Ma, L.; and Liu, Y. 2019. Ma-chine Learning Testing: Survey, Landscapes and Horizons. IEEE Transactions on Software Engineering. doi: 10.1109\/TSE.2019.2962027.","journal-title":"IEEE Transactions on Software Engineering. doi"}],"event":{"name":"AIES '21: AAAI\/ACM Conference on AI, Ethics, and Society","location":"Virtual Event USA","acronym":"AIES '21","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","AAAI"]},"container-title":["Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462564","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461702.3462564","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:49:06Z","timestamp":1750193346000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461702.3462564"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,21]]},"references-count":105,"alternative-id":["10.1145\/3461702.3462564","10.1145\/3461702"],"URL":"https:\/\/doi.org\/10.1145\/3461702.3462564","relation":{},"subject":[],"published":{"date-parts":[[2021,7,21]]},"assertion":[{"value":"2021-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}