{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:34:07Z","timestamp":1784392447595,"version":"3.55.0"},"reference-count":71,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,8,17]],"date-time":"2023-08-17T00:00:00Z","timestamp":1692230400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>The purpose of the Stakeholder Playbook is to enable the developers of explainable AI systems to take into account the different ways in which different stakeholders or role-holders need to \u201clook inside\u201d the AI\/XAI systems.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>We conducted structured cognitive interviews with senior and mid-career professionals who had direct experience either developing or using AI and\/or autonomous systems.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The results show that role-holders need access to others (e.g., trusted engineers and trusted vendors) for them to be able to develop satisfying mental models of AI systems. They need to know how it fails and misleads as much as they need to know how it works. Some stakeholders need to develop an understanding that enables them to explain the AI to someone else and not just satisfy their own sense-making requirements. Only about half of our interviewees said they always wanted explanations or even needed better explanations than the ones that were provided. Based on our empirical evidence, we created a \u201cPlaybook\u201d that lists explanation desires, explanation challenges, and explanation cautions for a variety of stakeholder groups and roles.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>This and other findings seem surprising, if not paradoxical, but they can be resolved by acknowledging that different role-holders have differing skill sets and have different sense-making desires. Individuals often serve in multiple roles and, therefore, can have different immediate goals. The goal of the Playbook is to help XAI developers by guiding the development process and creating explanations that support the different roles.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fcomp.2023.1117848","type":"journal-article","created":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T07:45:30Z","timestamp":1692344730000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Explainable AI: roles and stakeholders, desirements and challenges"],"prefix":"10.3389","volume":"5","author":[{"given":"Robert R.","family":"Hoffman","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shane T.","family":"Mueller","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gary","family":"Klein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammadreza","family":"Jalaeian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Connor","family":"Tate","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,8,17]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3233\/AAC-181005","article-title":"Noise induced hearing loss: Building an application using the ANGELIC methodology","volume":"10","author":"Al-Abdulakarim","year":"2019","journal-title":"Argu. Comput."},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10506-016-9178-1","article-title":"A methodology for designing systems to reason with legal cases using abstract dialectical frameworks","volume":"24","author":"Al-Abdulkarim","year":"2016","journal-title":"Artif. Intell. Law"},{"key":"B3","article-title":"On the importance of application-grounded experimental design for evaluating explainable ml methods","author":"Amarasinghe","year":"2022","journal-title":"arXiv:"},{"key":"B4","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.eswa.2017.03.009","article-title":"Explanatory dialogs with argumentative faculties over inconsistent knowledge bases","volume":"80","author":"Arioua","year":"2017","journal-title":"J. Expert Syst. Applic."},{"key":"B5","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"B6","article-title":"One explanation does not fit all: A toolkit and Taxonomy of AI explainability concepts","author":"Arya","year":"2019","journal-title":"arXiv"},{"key":"B7","doi-asserted-by":"publisher","first-page":"103387","DOI":"10.1016\/j.artint.2020.103387","article-title":"Explanation in AI and law: Past, present and future","volume":"22","author":"Atkinson","year":"2020","journal-title":"Artif. Intell."},{"key":"B8","article-title":"Machine learning explainability for external stakeholders","author":"Bhatt","year":"2020","journal-title":"arXiv"},{"key":"B9","first-page":"454","article-title":"\u201cProxy tasks and subjective measures can be misleading in evaluating explainable AI systems,\u201d","volume-title":"Proceedings of the ACM International Conference on Intelligent User Interfaces","author":"Bu\u00e7inca","year":"2020"},{"key":"B10","doi-asserted-by":"publisher","first-page":"118888","DOI":"10.1016\/j.eswa.2022.118888","article-title":"Quod erat demonstrandum? - Toward a typology of the concept of explanation for the design of explainable AI","volume":"313","author":"Cabitza","year":"2023","journal-title":"Expert Syst. Applic."},{"key":"B11","unstructured":"\u201cInterpretable narrative explanation for ML predictors with LP: A case study for XAI,\u201d\n            CalegariR.\n            CiattoG.\n            DellaluceJ.\n            OmiciniA.\n          Workshop 'From Objects to Agents' (WOA 2019).2019"},{"key":"B12","first-page":"228","article-title":"\u201cExplanation ontology: A model of explanations for user-centered AI,\u201d","volume-title":"International Semantic Web Conference","author":"Chari","year":"2020"},{"key":"B13","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1080\/03640210701863396","article-title":"Observing tutorial dialogues collaboratively: Insights about human tutoring effectiveness from vicarious learning","volume":"32","author":"Chi","year":"2008","journal-title":"Cogn. Sci."},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/7304.001.0001","volume-title":"Working Minds: A Practitioner's Guide to Cognitive Task Analysis.","author":"Crandall","year":"2006"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1282","DOI":"10.3389\/fpsyg.2017.01282","article-title":"Identifying the machine translation error types with the greatest impact on post-editing effort","volume":"8","author":"Daems","year":"2017","journal-title":"Front. Psychol"},{"key":"B16","doi-asserted-by":"crossref","unstructured":"DahanS.\n          AU-powered trademark dispute resolution. Report to the European Union Intellectual Property Office (EUIPO)2020","DOI":"10.2139\/ssrn.3786069"},{"key":"B17","doi-asserted-by":"publisher","first-page":"e36","DOI":"10.1002\/ail2.36","article-title":"From \u201cno clear winner\u201d to an effective explainable Artificial Intelligence process: An empirical journey","volume":"2","author":"Dodge","year":"2021","journal-title":"Appl. AI Lett."},{"key":"B18","article-title":"Towards a rigorous science of interpretable machine learning","author":"Doshi-Velez","year":"2017","journal-title":"arXiv"},{"key":"B19","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1145\/3172944.3172961","article-title":"\u201cBringing transparency design into practice,\u201d","author":"Eiband","year":"2018","journal-title":"23rd International Conference on Intelligent User Interfaces"},{"key":"B20","unstructured":"\u201cGeneral Data Protection Regulation Article 22, Recital 71.\u201d2016"},{"key":"B21","unstructured":"FazelpourS.\n          Disciplining deliberation: Interpreting machine learning trade-offs in sociotechnical systems. AI Metrology Colloquia Series. National Institute of Standards and Technology2023"},{"key":"B22","doi-asserted-by":"publisher","first-page":"2053951719860542","DOI":"10.1177\/2053951719860542","article-title":"Transparency you can trust: Transparency requirements for artificial intelligence between legal norms and contextual concerns","volume":"6","author":"Felzmann","year":"2019","journal-title":"Big Data Soc."},{"key":"B23","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","article-title":"AI4people\u2014An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations","volume":"28","author":"Floridi","year":"2018","journal-title":"Minds Mach."},{"key":"B24","volume-title":"Doing Grounded Theory - Issues and Discussions","author":"Glaser","year":"1998"},{"key":"B25","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1609\/aimag.v38i3.2741","article-title":"European Union regulations on algorithmic decision-making and a \u201cright to explanation.\u201d","volume":"38","author":"Goodman","year":"2017","journal-title":"AI Mag."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1727","DOI":"10.22541\/au.163699841.19031727\/v1","article-title":"DARPA's explainable AI Program: A retrospective","volume":"19","author":"Gunning","year":"2021","journal-title":"Appl. AI Lett."},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-50353-6_20","article-title":"\u201cExplainable artificial intelligence: What do you need to know?\u201d","volume-title":"Augmented Cognition. Theoretical and Technological Approaches","author":"Hepenstal","year":"2020"},{"key":"B28","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1145\/3306618.3314273","article-title":"\u201cTED: Teaching AI to explain its decisions,\u201d","volume-title":"Proceedings of the 2019 AAAI\/ACM Conference on AI, Ethics, and Society","author":"Hind","year":"2019"},{"key":"B29","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1109\/MIS.2006.9","article-title":"\u201cHCC implications for the procurement process","volume":"21","author":"Hoffman","year":"2006","journal-title":"IEEE Intell. Syst."},{"key":"B30","doi-asserted-by":"publisher","first-page":"397","DOI":"10.3233\/IKS-2012-0203","article-title":"Naturalistic investigations and models of reasoning about complex indeterminate causation","volume":"10","author":"Hoffman","year":"2011","journal-title":"Inf. Knowl. Syst. Manag."},{"key":"B31","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MIS.2013.108","article-title":"Envisioning desirements","volume":"26","author":"Hoffman","year":"2013","journal-title":"IEEE Intell. Syst."},{"key":"B32","doi-asserted-by":"publisher","first-page":"1096257","DOI":"10.3389\/fcomp.2023.1096257","article-title":"Measures for explainable AI: Explanation goodness, User satisfaction, mental models, curiosity, trust and human-AI Performance","volume":"5","author":"Hoffman","year":"2023","journal-title":"Front. Comput. Sci"},{"key":"B33","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1109\/MIS.2017.3121544","article-title":"Explaining Explanation, Part 2: Empirical Foundations","volume":"34","author":"Hoffman","year":"2017","journal-title":"IEEE Intell. Syst."},{"key":"B34","article-title":"\u201cCognitive ethnography,\u201d","volume":"25","author":"Hutchins","year":"2003","journal-title":"Proceedings of the Annual Meeting of the Cognitive Science Society"},{"key":"B35","unstructured":"IBM Research Trusted AI2021"},{"key":"B36","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3411764.3445385","article-title":"\u201cDesigning AI for trust and collaboration in time-constrained medical decisions: a sociotechnical lens,\u201d","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Jacobs","year":"2021"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.22541\/au.163284810.09140868\/v1","article-title":"Qualitative investigation in explainable Artificial Intelligence: A bit more insight from social science","author":"Johs","year":"2020","journal-title":"arXiv"},{"key":"B38","first-page":"1","article-title":"\u201cInterpreting Interpretability: Understanding data scientists' use of interpretability tools for machine learning,\u201d","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Kaur","year":"2020"},{"key":"B39","doi-asserted-by":"publisher","first-page":"103459","DOI":"10.1016\/j.artint.2021.103459","article-title":"Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies","volume":"294","author":"Kenny","year":"2021","journal-title":"Artif. Intell."},{"key":"B40","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1177\/15553434211045154","article-title":"Modeling the process by which people try to explain things to others","volume":"15","author":"Klein","year":"2021","journal-title":"J. Cogn. Eng. Deci. Mak."},{"key":"B41","article-title":"An evaluation of the human-interpretability of explanation","author":"Lage","year":"2019","journal-title":"arXiv"},{"key":"B42","doi-asserted-by":"publisher","first-page":"103473","DOI":"10.1016\/j.artint.2021.103473","article-title":"What do we want from explainable artificial intelligence (XAI): A stakeholder perspective on XAI and a conceptual model guiding interdisciplinary research","volume":"296","author":"Langer","year":"2021","journal-title":"Artif. Intell."},{"key":"B43","doi-asserted-by":"crossref","DOI":"10.1145\/3313831.3376590","article-title":"\u201cQuestioning the AI informing design practices for explainable AI user experiences,\u201d","volume-title":"Proceedings of CHI 2020","author":"Liao","year":"2020"},{"key":"B44","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","article-title":"The mythos of model interpretability","volume":"16","author":"Lipton","year":"2018","journal-title":"Queue"},{"key":"B45","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1016\/j.tics.2016.08.001","article-title":"Explanatory preferences shape learning and inference","volume":"20","author":"Lombrozo","year":"2016","journal-title":"Trends Cogn. Sci."},{"key":"B46","doi-asserted-by":"publisher","first-page":"154096","DOI":"10.1109\/ACCESS.2019.2949286","article-title":"Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view","volume":"7","author":"Loyola-Gonzalez","year":"2019","journal-title":"IEEE Access"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","article-title":"Explanation in Artificial Intelligence: Insights from the social sciences","volume":"267","author":"Miller","year":"2017","journal-title":"Artif. Intell."},{"key":"B48","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1145\/3287560.3287574","article-title":"\u201cExplaining explanations in AI,\u201d","volume-title":"Proceedings of the 2019 Conference on Fairness, Accountability, and Transparency","author":"Mittelstadt","year":"2019"},{"key":"B49","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3387166","article-title":"A multidisciplinary survey and framework for design and evaluation of explainable AI Systems","volume":"11","author":"Mohseni","year":"2020","journal-title":"ACM Trans. Inter. Intell. Syst."},{"key":"B50","unstructured":"MuellerS. T.\n            HoffmanR. R.\n            ClanceyW.\n            EmreyA.\n            KleinG.\n          Explanation in human-AI systems: a literature meta-review, synopsis of key ideas and publications and bibliography for explainable AI. Technical Report from Task Area 2 to the DARPA Explainable AI Program2019"},{"key":"B51","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1109\/MIS.2011.32","article-title":"Improving users' mental models of intelligent software tools","volume":"26","author":"Mueller","year":"2011","journal-title":"IEEE Intell. Syst."},{"key":"B52","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1007\/978-3-030-45691-7_49","article-title":"\u201cPersonalizing explainable recommendations: literature and conceptualization,\u201d","volume-title":"Trends and Innovations in Information Systems and Technologies","author":"Naiseh","year":"2020"},{"key":"B53","unstructured":"\u201cComparing automatic and human evaluation of local explanations for text classification,\u201d10691078\n            NguyenD.\n          Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies2018"},{"key":"B54","article-title":"Stakeholders in explainable AI","author":"Preece","year":"2018","journal-title":"arXiv preprint arXiv"},{"key":"B55","article-title":"\u201cCan we do better explanations? A proposal of user-centered AI,\u201d","volume-title":"Proceedings of the ACM IUI 2019 Workshop","author":"Ribera","year":"2019"},{"key":"B56","unstructured":"\u201cBetter metrics for evaluating explainable artificial intelligence,\u201d\n            RosenfeldA.\n          33780444New YorkAssociation for Computing MachineryAAMAS '21: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems2021"},{"key":"B57","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"B58","doi-asserted-by":"publisher","first-page":"149","DOI":"10.1007\/978-3-030-77283-3_8","article-title":"\u201cRe-orienting towards the science of the artificial: Engineering AI systems,\u201d","author":"Russell","year":"2021","journal-title":"Systems Engineering and Artificial Intelligence"},{"key":"B59","volume-title":"Introduction to Cognitive Ethnography and Systematic Field Work","author":"Schoepfle","year":"2021"},{"key":"B60","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1007\/s13218-018-0559-3","article-title":"Defining explainable AI for requirements analysis","volume":"32","author":"Sheh","year":"2018","journal-title":"KI - K\u00fcnstliche Intell."},{"key":"B61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3538882.3542790","article-title":"\u201cHuman-centered ai: ensuring human control while increasing automation.\u201d","author":"Shneiderman","year":"2023","journal-title":"Proceedings of the 5th Workshop on Human Factors in Hypertext"},{"key":"B62","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/3351095.3372870","article-title":"\u201cA framework for systematic assessment of explainable approaches,\u201d","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","author":"Sokol","year":"2020"},{"key":"B63","first-page":"56","article-title":"\u201cDo human rationales improve machine explanations?\u201d","volume-title":"Proceedings of the Second BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP","author":"Strout","year":"2019"},{"key":"B64","unstructured":"TateD. M.\n            GrierR. A.\n            MartinC. A.\n            MosesF. L.\n            SparrowD. A.\n          A Framework For Evidence-Based Licensure Of Adaptive Autonomous Systems. Alexandria, VA: Institute for Defense Analysis2016"},{"key":"B65","unstructured":"\u201cA survey on explainable Artificial Intelligence (XAI): Toward medical XAI,\u201d\n            TjoaE.\n            GuanC.\n          33079674IEEE Transactions on Neural Networks and Learning Systems.2020"},{"key":"B66","article-title":"\u201cInterpretable to whom? A role-based model for analyzing interpretable machine learning systems,\u201d","volume-title":"Proceedings of the 2018 ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)","author":"Tomsett","year":"2018"},{"key":"B67","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-93736-2_39","article-title":"\u201cHow to choose an explainability method? Towards a methodical implementation of XAI in practice,\u201d","volume-title":"Machine Learning and Principles and Practice of Knowledge Discovery in Databases. (ECML PKDD 2021)","author":"Vermeire","year":"2021"},{"key":"B68","doi-asserted-by":"publisher","first-page":"76","DOI":"10.2139\/ssrn.2903469","article-title":"Why a right to explanation of automated decision-making does not exist in the general data protection regulation","volume":"72","author":"Wachter","year":"2016","journal-title":"Int. Data Priv. Law"},{"key":"B69","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/978-3-030-28954-6_2","article-title":"\u201cTransparency: Motivations and challenges,\u201d","volume-title":"Explainable AI: Interpreting, explaining and visualizing deep learning","author":"Weller","year":"2019"},{"key":"B70","unstructured":"\u201cUsing annotator rationales to improve machine learning for text categorization,\u201d260267\n            ZaidanO.\n            EisnerJ.\n            PiatkoC.\n          36754129Proceedings of Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference2007"},{"key":"B71","unstructured":"ZhangY.\n            MarshallI.\n            WallaceB. C.\n          28191551Rationale-Augmented Convolutional Networks for Text Classification2016"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1117848\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T07:45:52Z","timestamp":1692344752000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1117848\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,17]]},"references-count":71,"alternative-id":["10.3389\/fcomp.2023.1117848"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2023.1117848","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,17]]},"article-number":"1117848"}}