{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T03:48:55Z","timestamp":1782791335364,"version":"3.54.5"},"publisher-location":"Cham","reference-count":28,"publisher":"Springer Nature Switzerland","isbn-type":[{"value":"9783031440632","type":"print"},{"value":"9783031440649","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T00:00:00Z","timestamp":1698624000000},"content-version":"vor","delay-in-days":302,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The growing importance of Explainable Artificial Intelligence (XAI) has highlighted the need to understand the decision-making processes of black-box models. Surrogation, emulating a black-box model (BB) with a white-box model (WB), is crucial in applications where BBs are unavailable due to security or practical concerns. Traditional fidelity measures only evaluate the similarity of the final predictions, which can lead to a significant limitation: considering a WB faithful even when it has the same prediction as the BB but with a completely different rationale. Addressing this limitation is crucial to develop Trustworthy AI practical applications beyond XAI. To address this issue, we introduce ShapGAP, a novel metric that assesses the faithfulness of surrogate models by comparing their reasoning paths, using SHAP explanations as a proxy. We validate the effectiveness of ShapGAP by applying it to real-world datasets from healthcare and finance domains, comparing its performance against traditional fidelity measures. Our results show that ShapGAP enables better understanding and trust in XAI systems, revealing the potential dangers of relying on models with high task accuracy but unfaithful explanations. ShapGAP serves as a valuable tool for identifying faithful surrogate models, paving the way for more reliable and Trustworthy AI applications.<\/jats:p>","DOI":"10.1007\/978-3-031-44064-9_10","type":"book-chapter","created":{"date-parts":[[2023,10,29]],"date-time":"2023-10-29T04:19:19Z","timestamp":1698553159000},"page":"160-173","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Beyond Prediction Similarity: ShapGAP for\u00a0Evaluating Faithful Surrogate Models in\u00a0XAI"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8764-1174","authenticated-orcid":false,"given":"Ettore","family":"Mariotti","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4460-1671","authenticated-orcid":false,"given":"Adarsa","family":"Sivaprasad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3673-421X","authenticated-orcid":false,"given":"Jose Maria Alonso","family":"Moral","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,10,30]]},"reference":[{"key":"10_CR1","unstructured":"Alaa, A.M., van der Schaar, M.: Demystifying black-box models with symbolic metamodels. In: Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019)"},{"key":"10_CR2","doi-asserted-by":"publisher","unstructured":"Ali, S., et al.: Explainable Artificial Intelligence (XAI): what we know and what is left to attain trustworthy artificial intelligence. Inf. Fusion 101805 (2023). https:\/\/doi.org\/10.1016\/j.inffus.2023.101805. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1566253523001148","DOI":"10.1016\/j.inffus.2023.101805"},{"key":"10_CR3","unstructured":"Alvarez-Melis, D., Jaakkola, T.S.: Towards robust interpretability with self-explaining neural networks. In: Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS 2018, pp. 7786\u20137795. Curran Associates Inc., Red Hook (2018)"},{"key":"10_CR4","unstructured":"Ba, J., Caruana, R.: Do deep nets really need to be deep? In: Advances in Neural Information Processing Systems, vol. 27. Curran Associates, Inc. (2014)"},{"key":"10_CR5","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","volume":"58","author":"A Barredo Arrieta","year":"2020","unstructured":"Barredo Arrieta, A., et al.: Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82\u2013115 (2020). https:\/\/doi.org\/10.1016\/j.inffus.2019.12.012","journal-title":"Inf. Fusion"},{"key":"10_CR6","unstructured":"Bastani, O., Kim, C., Bastani, H.: Interpretability via model extraction (2018). http:\/\/arxiv.org\/abs\/1706.09773 [cs, stat]"},{"key":"10_CR7","doi-asserted-by":"publisher","unstructured":"Burkart, N., Huber, M.F.: A survey on the explainability of supervised machine learning. J. Artif. Intell. Res. 70, 245\u2013317 (2021). https:\/\/doi.org\/10.1613\/jair.1.12228. https:\/\/www.jair.org\/index.php\/jair\/article\/view\/12228","DOI":"10.1613\/jair.1.12228"},{"key":"10_CR8","unstructured":"Craven, M., Shavlik, J.: Extracting tree-structured representations of trained networks. In: Advances in Neural Information Processing Systems, vol. 8. MIT Press (1995)"},{"key":"10_CR9","doi-asserted-by":"publisher","unstructured":"Dai, J., Upadhyay, S., Aivodji, U., Bach, S.H., Lakkaraju, H.: Fairness via explanation quality: evaluating disparities in the quality of post hoc explanations. In: Proceedings of the 2022 AAAI\/ACM Conference on AI, Ethics, and Society, pp. 203\u2013214 (2022). https:\/\/doi.org\/10.1145\/3514094.3534159. http:\/\/arxiv.org\/abs\/2205.07277 [cs]","DOI":"10.1145\/3514094.3534159"},{"key":"10_CR10","unstructured":"Dua, Dheeru and Graff, Casey: UCI Machine Learning Repository (2017). http:\/\/archive.ics.uci.edu\/ml. University of California, Irvine, School of Information and Computer Sciences"},{"issue":"4","key":"10_CR11","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1007\/s11023-018-9482-5","volume":"28","author":"L Floridi","year":"2018","unstructured":"Floridi, L., et al.: AI4People\u2014an ethical framework for a good AI society: opportunities, risks, principles, and recommendations. Mind. Mach. 28(4), 689\u2013707 (2018). https:\/\/doi.org\/10.1007\/s11023-018-9482-5","journal-title":"Mind. Mach."},{"issue":"4","key":"10_CR12","doi-asserted-by":"publisher","DOI":"10.1002\/ail2.61","volume":"2","author":"D Gunning","year":"2021","unstructured":"Gunning, D., Vorm, E., Wang, J.Y., Turek, M.: DARPA\u2019s explainable AI (XAI) program: a retrospective. Appl. AI Lett. 2(4), e61 (2021). https:\/\/doi.org\/10.1002\/ail2.61","journal-title":"Appl. AI Lett."},{"key":"10_CR13","unstructured":"Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network (2015). http:\/\/arxiv.org\/abs\/1503.02531 [cs, stat]"},{"key":"10_CR14","doi-asserted-by":"publisher","unstructured":"Jacovi, A., Goldberg, Y.: Towards faithfully interpretable NLP systems: how should we define and evaluate faithfulness? In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4198\u20134205. Association for Computational Linguistics, Online (2020). https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.386. https:\/\/www.aclweb.org\/anthology\/2020.acl-main.386","DOI":"10.18653\/v1\/2020.acl-main.386"},{"key":"10_CR15","unstructured":"Lakkaraju, H., Kamar, E., Caruana, R., Leskovec, J.: Interpretable & explorable approximations of black box models (2017). http:\/\/arxiv.org\/abs\/1707.01154 [cs]"},{"key":"10_CR16","unstructured":"Lundberg, S.M., Erion, G.G., Lee, S.I.: Consistent Individualized Feature Attribution for Tree Ensembles (2018)"},{"key":"10_CR17","unstructured":"Lundberg, S.M., Lee, S.I.: A unified approach to interpreting model predictions. In: Advances in Neural Information Processing Systems, vol. 30. Curran Associates, Inc. (2017)"},{"key":"10_CR18","doi-asserted-by":"publisher","unstructured":"Mariotti, E., Alonso-Moral, J.M., Gatt, A.: Measuring model understandability by means of shapley additive explanations. In: 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Padua, Italy, pp. 1\u20138. IEEE (2022). https:\/\/doi.org\/10.1109\/FUZZ-IEEE55066.2022.9882773. https:\/\/ieeexplore.ieee.org\/document\/9882773\/","DOI":"10.1109\/FUZZ-IEEE55066.2022.9882773"},{"key":"10_CR19","doi-asserted-by":"publisher","unstructured":"Markus, A.F., Kors, J.A., Rijnbeek, P.R.: The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies. J. Biomed. Inform. 113, 103655 (2021). https:\/\/doi.org\/10.1016\/j.jbi.2020.103655. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1532046420302835","DOI":"10.1016\/j.jbi.2020.103655"},{"key":"10_CR20","doi-asserted-by":"publisher","unstructured":"Mittelstadt, B.D., Allo, P., Taddeo, M., Wachter, S., Floridi, L.: The ethics of algorithms: mapping the debate. Big Data Soc. 3(2), 205395171667967 (2016). https:\/\/doi.org\/10.1177\/2053951716679679. http:\/\/journals.sagepub.com\/doi\/10.1177\/2053951716679679","DOI":"10.1177\/2053951716679679"},{"key":"10_CR21","doi-asserted-by":"publisher","unstructured":"Ribeiro, M.T., Singh, S., Guestrin, C.: \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2016, pp. 1135\u20131144. Association for Computing Machinery, New York (2016). https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"issue":"5","key":"10_CR22","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","volume":"1","author":"C Rudin","year":"2019","unstructured":"Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206\u2013215 (2019). https:\/\/doi.org\/10.1038\/s42256-019-0048-x","journal-title":"Nat. Mach. Intell."},{"key":"10_CR23","unstructured":"Sanchez, I., Rocktaschel, T., Riedel, S., Singh, S.: Towards extracting faithful and descriptive representations of latent variable models (2015)"},{"key":"10_CR24","doi-asserted-by":"publisher","unstructured":"Selbst, A.D., Barocas, S.: The intuitive appeal of explainable machines. SSRN Electron. J. (2018). https:\/\/doi.org\/10.2139\/ssrn.3126971. https:\/\/www.ssrn.com\/abstract=3126971","DOI":"10.2139\/ssrn.3126971"},{"key":"10_CR25","doi-asserted-by":"publisher","unstructured":"Shapley, L.S.: A value for n-person games. In: Kuhn, H.W., Tucker, A.W. (eds.) Contributions to the Theory of Games (AM-28), vol. II, pp. 307\u2013318. Princeton University Press (1953). https:\/\/doi.org\/10.1515\/9781400881970-018","DOI":"10.1515\/9781400881970-018"},{"key":"10_CR26","doi-asserted-by":"publisher","unstructured":"Street, N., Wolberg, W.H., Mangasarian, O.L.: Nuclear feature extraction for breast tumor diagnosis. In: Proceedings of the Conference on Biomedical Image Processing and Biomedical Visualization, vol. 1905 (1993). https:\/\/doi.org\/10.1117\/12.148698","DOI":"10.1117\/12.148698"},{"key":"10_CR27","doi-asserted-by":"publisher","unstructured":"Tan, S., Caruana, R., Hooker, G., Lou, Y.: Distill-and-compare: auditing black-box models using transparent model distillation. In: Proceedings of the 2018 AAAI\/ACM Conference on AI, Ethics, and Society, pp. 303\u2013310. ACM, New Orleans (2018). https:\/\/doi.org\/10.1145\/3278721.3278725. https:\/\/dl.acm.org\/doi\/10.1145\/3278721.3278725","DOI":"10.1145\/3278721.3278725"},{"key":"10_CR28","doi-asserted-by":"publisher","unstructured":"Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: automated decisions and the GDPR. SSRN Electron. J. (2017). https:\/\/doi.org\/10.2139\/ssrn.3063289. https:\/\/www.ssrn.com\/abstract=3063289","DOI":"10.2139\/ssrn.3063289"}],"container-title":["Communications in Computer and Information Science","Explainable Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-44064-9_10","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,29]],"date-time":"2023-10-29T04:25:16Z","timestamp":1698553516000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-44064-9_10"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"ISBN":["9783031440632","9783031440649"],"references-count":28,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-44064-9_10","relation":{},"ISSN":["1865-0929","1865-0937"],"issn-type":[{"value":"1865-0929","type":"print"},{"value":"1865-0937","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]},"assertion":[{"value":"30 October 2023","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}},{"value":"xAI","order":1,"name":"conference_acronym","label":"Conference Acronym","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"World Conference on Explainable Artificial Intelligence","order":2,"name":"conference_name","label":"Conference Name","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Lisbon","order":3,"name":"conference_city","label":"Conference City","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Portugal","order":4,"name":"conference_country","label":"Conference Country","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"2023","order":5,"name":"conference_year","label":"Conference Year","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"26 July 2023","order":7,"name":"conference_start_date","label":"Conference Start Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"28 July 2023","order":8,"name":"conference_end_date","label":"Conference End Date","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"1","order":9,"name":"conference_number","label":"Conference Number","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"xai2023","order":10,"name":"conference_id","label":"Conference ID","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"http:\/\/xaiworldconference.com\/","order":11,"name":"conference_url","label":"Conference URL","group":{"name":"ConferenceInfo","label":"Conference Information"}},{"value":"Single-blind","order":1,"name":"type","label":"Type","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"EasyChair","order":2,"name":"conference_management_system","label":"Conference Management System","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"220","order":3,"name":"number_of_submissions_sent_for_review","label":"Number of Submissions Sent for Review","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"94","order":4,"name":"number_of_full_papers_accepted","label":"Number of Full Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"0","order":5,"name":"number_of_short_papers_accepted","label":"Number of Short Papers Accepted","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"43% - The value is computed by the equation \"Number of Full Papers Accepted \/ Number of Submissions Sent for Review * 100\" and then rounded to a whole number.","order":6,"name":"acceptance_rate_of_full_papers","label":"Acceptance Rate of Full Papers","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":7,"name":"average_number_of_reviews_per_paper","label":"Average Number of Reviews per Paper","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"3","order":8,"name":"average_number_of_papers_per_reviewer","label":"Average Number of Papers per Reviewer","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}},{"value":"No","order":9,"name":"external_reviewers_involved","label":"External Reviewers Involved","group":{"name":"ConfEventPeerReviewInformation","label":"Peer Review Information (provided by the conference organizers)"}}]}}