{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T14:44:07Z","timestamp":1777905847048,"version":"3.51.4"},"reference-count":59,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T00:00:00Z","timestamp":1736294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>A growing consensus emphasizes the efficacy of user-centered and personalized approaches within the field of explainable artificial intelligence (XAI). The proliferation of diverse explanation strategies in recent years promises to improve the interaction between humans and explainable agents. This poses the challenge of assessing the goodness and efficacy of the proposed explanation, which so far has primarily relied on indirect measures, such as the user's task performance. We introduce an assessment task designed to objectively and quantitatively measure the goodness of XAI systems, specifically in terms of their \u201cinformation power.\u201d This metric aims to evaluate the amount of information the system provides to non-expert users during the interaction. This work has a three-fold objective: to propose the Information Power assessment task, provide a comparison between our proposal and other XAI goodness measures with respect to eight characteristics, and provide detailed instructions to implement it based on researchers' needs.<\/jats:p>","DOI":"10.3389\/fcomp.2024.1412341","type":"journal-article","created":{"date-parts":[[2025,1,8]],"date-time":"2025-01-08T06:27:43Z","timestamp":1736317663000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["How informative is your XAI? Assessing the quality of explanations through information power"],"prefix":"10.3389","volume":"6","author":[{"given":"Marco","family":"Matarese","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesco","family":"Rea","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katharina J.","family":"Rohlfing","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessandra","family":"Sciutti","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,1,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"52138","DOI":"10.1109\/ACCESS.2018.2870052","article-title":"Peeking inside the black-box: a survey on explainable artificial intelligence (xAI)","volume":"6","author":"Adadi","year":"2018","journal-title":"IEEE Access"},{"key":"B2","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1207\/s15516709cog2602_1","article-title":"An effective metacognitive strategy: learning by doing and explaining with a computer-based cognitive tutor","volume":"26","author":"Aleven","year":"2002","journal-title":"Cogn. Sci"},{"key":"B3","doi-asserted-by":"publisher","first-page":"101805","DOI":"10.1016\/j.inffus.2023.101805","article-title":"What we know and what is left to attain trustworthy artificial intelligence","volume":"99","author":"Ali","year":"2022","journal-title":"Inf. Fusion"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1710.09511","article-title":"Interpnet: neural introspection for interpretable deep learning","author":"Barratt","year":"2017","journal-title":"arXiv"},{"key":"B5","unstructured":"\u201cInterpretability of machine learning models and representations: an introduction,\u201d\n          \n          77\n          82\n          \n            \n              Bibal\n              A.\n            \n            \n              Fr\u00e9nay\n              B.\n            \n          \n          CIACO\n          24th European symposium on artificial neural networks, computational intelligence and machine learning\n          \n          2016"},{"key":"B6","doi-asserted-by":"crossref","DOI":"10.1145\/3648505.3648509","article-title":"\u201cTowards a computational architecture for co-constructive explainable systems,\u201d","volume-title":"Proceedings of the 2024 Workshop on Explainability Engineering (ExEn'24)","author":"Booshehri","year":"2024"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.01540","article-title":"Openai gym","author":"Brockman","year":"2016","journal-title":"arXiv"},{"key":"B8","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/S0304-3975(01)00144-X","article-title":"Complexity measures and decision tree complexity: a survey","volume":"288","author":"Buhrman","year":"2002","journal-title":"Theoret. Comput. Sci"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.08760","article-title":"Forms of understanding of xAI-explanations","author":"Buschmeier","year":"2023","journal-title":"arXiv"},{"key":"B10","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1037\/0022-3514.42.1.116","article-title":"The need for cognition","volume":"42","author":"Cacioppo","year":"1982","journal-title":"J. Pers. Soc. Psychol"},{"key":"B11","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1007\/978-3-319-99740-7_23","article-title":"\u201cEvaluating explanations by cognitive value,\u201d","author":"Chander","year":"2018","journal-title":"Machine Learning and Knowledge Extraction"},{"key":"B12","first-page":"361","article-title":"\u201cTowards real interpretability of student success prediction combining methods of xAI and social science,\u201d","author":"Cohausz","year":"2022","journal-title":"Proceedings of the 15th International Conference on Educational Data Mining"},{"key":"B13","doi-asserted-by":"publisher","first-page":"103503","DOI":"10.1016\/j.artint.2021.103503","article-title":"Toward personalized xAI: a case study in intelligent tutoring systems","volume":"298","author":"Conati","year":"2021","journal-title":"Artif. Intell"},{"key":"B14","doi-asserted-by":"publisher","first-page":"101840","DOI":"10.1016\/j.artmed.2020.101840","article-title":"Explainable AI meets persuasiveness: translating reasoning results into behavioral change advice","volume":"105","author":"Dragoni","year":"2020","journal-title":"Artif. Intell. Med"},{"key":"B15","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1016\/S1071-5819(03)00038-7","article-title":"The role of trust in automation reliance","volume":"58","author":"Dzindolet","year":"2003","journal-title":"Int. J. Hum.-Comput. Stud"},{"key":"B16","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.procs.2022.07.015","article-title":"Internet of behaviour (IOB)-based AI models for personalized smart education systems","volume":"203","author":"Embarak","year":"2022","journal-title":"Procedia Comput. Sci"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2102.05460","article-title":"The human-AI relationship in decision-making: AI explanation to support people on justifying their decisions","author":"Ferreira","year":"2021","journal-title":"arXiv"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1806.00069","article-title":"Explaining explanations: an overview of interpretability of machine learning","author":"Gilpin","year":"2018","journal-title":"arXiv"},{"key":"B19","first-page":"2376","article-title":"\u201cCounterfactual visual explanations,\u201d","volume-title":"Proceedings of the 36th International Conference on Machine Learning","author":"Goyal","year":"2019"},{"key":"B20","unstructured":"\u201cGuidelines for developing explainable cognitive models,\u201d\n          \n          85\n          90\n          \n            \n              Harbers\n              M.\n            \n            \n              Broekens\n              J.\n            \n            \n              Van Den Bosch\n              K.\n            \n            \n              Meyer\n              J.-J.\n            \n          \n          Citeseer\n          Proceedings of ICCM"},{"key":"B21","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1109\/WI-IAT.2010.115","article-title":"\u201cDesign and evaluation of explainable BDI agents,\u201d","author":"Harbers","year":"","journal-title":"2010 IEEE\/WIC\/ACM International Conference on Web Intelligence and Intelligent Agent Technology, Vol"},{"key":"B22","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1007\/978-3-030-50353-6_20","article-title":"\u201cExplainable artificial intelligence: what do you need to know?\u201d","author":"Hepenstal","year":"2020","journal-title":"Augmented Cognition. Theoretical and Technological Approaches"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1812.04608","article-title":"Metrics for explainable AI: Challenges and prospects","author":"Hoffman","year":"2018","journal-title":"arXiv"},{"key":"B24","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1007\/s13218-020-00636-z","article-title":"Measuring the quality of explanations: the system causability scale (SCS)","volume":"34","author":"Holzinger","year":"2020","journal-title":"K\u00fcnstliche Intelligenz"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1002\/widm.1312","article-title":"Causability and explainability of artificial intelligence in medicine","author":"Holzinger","year":"2019","journal-title":"Wiley Interdiscip. Rev.: Data Min. Knowl. Discov"},{"key":"B26","article-title":"\u201cTowards trustable explainable AI,\u201d","volume-title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI'20","author":"Ignatiev","year":"2021"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.3390\/app12031353","article-title":"A systematic review of explainable artificial intelligence in terms of different application domains and tasks","volume":"12","author":"Islam","year":"2022","journal-title":"Appl. Sci"},{"key":"B28","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1177\/0894439320980118","article-title":"Will algorithms blind people? the effect of explainable AI and decision-makers' experience on AI-supported decision-making in government","volume":"40","author":"Janssen","year":"2022","journal-title":"Soc. Sci. Comput. Rev"},{"key":"B29","first-page":"1","article-title":"\u201cInterpreting interpretability: understanding data scientists' use of interpretability tools for machine learning,\u201d","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI '20","author":"Kaur","year":"2020"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1902.00006","article-title":"An evaluation of the human-interpretability of explanation","author":"Lage","year":"2019","journal-title":"arXiv"},{"key":"B31","first-page":"2801","article-title":"\u201cThe dangers of post-hoc interpretability: unjustified counterfactual explanations,\u201d","volume-title":"Proceedings of the 28th International Joint Conference on Artificial Intelligence, IJCAI'19","author":"Laugel","year":"2019"},{"key":"B32","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1016\/j.tics.2016.08.001","article-title":"Explanatory preferences shape learning and inference","volume":"20","author":"Lombrozo","year":"2016","journal-title":"Trends Cogn. Sci"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2109.12912","article-title":"A user-centred framework for explainable artificial intelligence in human-robot interaction","author":"Matarese","year":"2021","journal-title":"arXiv"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.artint.2018.07.007","article-title":"Explanation in artificial intelligence: insights from the social sciences","volume":"267","author":"Miller","year":"2019","journal-title":"Artif. Intell"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.11839.18111","article-title":"A survey of evaluation methods and measures for interpretable machine learning","author":"Mohseni","year":"2018","journal-title":"arXiv"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.08164","article-title":"From anecdotal evidence to quantitative evaluation methods: a systematic review on evaluating explainable AI","author":"Nauta","year":"2022","journal-title":"arXiv"},{"key":"B37","doi-asserted-by":"crossref","DOI":"10.1145\/3411764.3445315","article-title":"\u201cManipulating and measuring model interpretability,\u201d","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, CHI '21","author":"Poursabzi-Sangdeh","year":"2021"},{"key":"B38","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1002\/isaf.1422","article-title":"Asking \u201cwhy\u201d in AI: explainability of intelligent systems-perspectives and challenges","volume":"25","author":"Preece","year":"2018","journal-title":"Intell. Sys. Acc. Fin. Mgmt"},{"key":"B39","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/j.patrec.2020.04.004","article-title":"Understanding the decisions of CNNS: an in-model approach","volume":"133","author":"Rio-Torto","year":"2020","journal-title":"Pattern Recognit. Lett"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1907.01180","article-title":"Conservative q-improvement: reinforcement learning for an interpretable decision-tree policy","author":"Roth","year":"2019","journal-title":"arXiv"},{"key":"B41","doi-asserted-by":"publisher","first-page":"2660","DOI":"10.1109\/TNNLS.2016.2599820","article-title":"Evaluating the visualization of what a deep neural network has learned","volume":"28","author":"Samek","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B42","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1007\/978-3-030-51924-7_6","article-title":"\u201cA situation awareness-based framework for design and evaluation of explainable AI,\u201d","author":"Sanneman","year":"2020","journal-title":"Explainable, Transparent Autonomous Agents and Multi-Agent Systems"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2204.06916","article-title":"Should I follow AI-based advice? Measuring appropriate reliance in human-AI decision-making","author":"Schemmer","year":"","journal-title":"arXiv"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2205.05126","article-title":"A meta-analysis on the utility of explainable artificial intelligence in human-AI decision-making","author":"Schemmer","year":"","journal-title":"arXiv"},{"key":"B45","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/3351095.3372870","article-title":"\u201cExplainability fact sheets: a framework for systematic assessment of explainable approaches,\u201d","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* '20","author":"Sokol","year":"2020"},{"key":"B46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/FUZZ-IEEE55066.2022.9882574","article-title":"\u201cHow to quantify the degree of explainability: experiments and practical implications,\u201d","volume-title":"2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)","author":"Sovrano","year":"2022"},{"key":"B47","doi-asserted-by":"publisher","first-page":"866920","DOI":"10.3389\/frai.2022.866920","article-title":"Self-explaining social robots: an explainable behavior generation architecture for human-robot interaction","volume":"87","author":"Stange","year":"2022","journal-title":"Front. Artif. Intell"},{"key":"B48","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1007\/978-1-4471-0269-4_5","article-title":"\u201cSelecting optimal split-functions for large datasets,\u201d","volume-title":"Research and Development in Intelligent Systems XVII","author":"Stoffel","year":"2001"},{"key":"B49","doi-asserted-by":"publisher","first-page":"103404","DOI":"10.1016\/j.artint.2020.103404","article-title":"Evaluating xAI: a comparison of rule-based and example-based explanations","volume":"291","author":"van der Waa","year":"2021","journal-title":"Artif. Intell"},{"key":"B50","first-page":"194","article-title":"\u201cDecision tree extraction using trained neural network,\u201d","volume-title":"Proceedings of the 9th International Conference on Smart Cities and Green ICT Systems - SMARTGREENS","author":"Vasilev","year":"2020"},{"key":"B51","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.inffus.2021.05.009","article-title":"Notions of explainability and evaluation approaches for explainable artificial intelligence","volume":"76","author":"Vilone","year":"2021","journal-title":"Inf. Fusion"},{"key":"B52","first-page":"8981","article-title":"\u201cScout: self-aware discriminant counterfactual explanations,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Wang","year":"2020"},{"key":"B53","first-page":"318","article-title":"\u201cAre explanations helpful? A comparative study of the effects of explanations in AI-assisted decision-making,\u201d","volume-title":"International Conference on Intelligent User Interfaces","author":"Wang","year":"2021"},{"key":"B54","doi-asserted-by":"crossref","DOI":"10.1145\/3519266","article-title":"\u201cEffects of explanations in AI-assisted decision making: principles and comparisons,\u201d","volume-title":"ACM Transactions on Interactive Intelligent Systems (TiiS)","author":"Wang","year":"2022"},{"key":"B55","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/s12193-020-00332-0","article-title":"\u201cLet me explain!\u201d: exploring the potential of virtual agents in explainable AI interaction design","volume":"15","author":"Weitz","year":"2021","journal-title":"J. Multimodal User Interfaces"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.13140\/RG.2.2.27885.92645","author":"Williams","year":"2021","journal-title":"Towards human-centred explainable AI: A systematic literature review"},{"key":"B57","article-title":"\u201cLearning decision trees with reinforcement learning,\u201d","volume-title":"31st Conference on Neural Information Processing Systems (NIPS 2017)","author":"Xiong","year":"2017"},{"key":"B58","article-title":"\u201cOn the (in)fidelity and sensitivity of explanations,\u201d","author":"Yeh","year":"2019","journal-title":"Advances in Neural Information Processing Systems, Vol. 32"},{"key":"B59","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1631\/FITEE.1700808","article-title":"Visual interpretability for deep learning: a survey","volume":"19","author":"Zhang","year":"2018","journal-title":"Front. Inf. Technol. Electron. Eng"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1412341\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,7]],"date-time":"2025-02-07T07:44:11Z","timestamp":1738914251000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2024.1412341\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,8]]},"references-count":59,"alternative-id":["10.3389\/fcomp.2024.1412341"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2024.1412341","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,8]]},"article-number":"1412341"}}