{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T07:06:24Z","timestamp":1775027184961,"version":"3.50.1"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,12,18]],"date-time":"2024-12-18T00:00:00Z","timestamp":1734480000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-1704189"],"award-info":[{"award-number":["CNS-1704189"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2024,12,18]]},"abstract":"<jats:p>Machine learning (ML) algorithms have advanced significantly in recent years, progressively evolving into artificial intelligence (AI) agents capable of solving complex, human-like intellectual challenges. Despite the advancements, the interpretability of these sophisticated models lags behind, with many ML architectures remaining \"black boxes\" that are too intricate and expansive for human interpretation. Recognizing this issue, there has been a revived interest in the field of explainable AI (XAI) aimed at explaining these opaque ML models. However, XAI tools often suffer from being tightly coupled with the underlying ML models and are inefficient due to redundant computations. We introduce provenance-enabled explainable AI (PXAI). PXAI decouples XAI computation from ML models through a provenance graph that tracks the creation and transformation of all data within the model. PXAI improves XAI computational efficiency by excluding irrelevant and insignificant variables and computation in the provenance graph. Through various case studies, we demonstrate how PXAI enhances computational efficiency when interpreting complex ML models, confirming its potential as a valuable tool in the field of XAI.<\/jats:p>","DOI":"10.1145\/3698826","type":"journal-article","created":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T16:40:35Z","timestamp":1734712835000},"page":"1-27","source":"Crossref","is-referenced-by-count":4,"title":["Provenance-Enabled Explainable AI"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6729-5718","authenticated-orcid":false,"given":"Jiachi","family":"Zhang","sequence":"first","affiliation":[{"name":"Alibaba Cloud, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2689-6020","authenticated-orcid":false,"given":"Wenchao","family":"Zhou","sequence":"additional","affiliation":[{"name":"Alibaba Cloud, Hangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-3433-9972","authenticated-orcid":false,"given":"Benjamin E.","family":"Ujcich","sequence":"additional","affiliation":[{"name":"Georgetown University, Washington, DC, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,12,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Somak Aditya Yezhou Yang and Chitta Baral. 2018. Explicit Reasoning over End-to-End Neural Architectures for Visual Question Answering. In AAAI.","DOI":"10.1609\/aaai.v32i1.11324"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140"},{"key":"e_1_2_1_3_1","article-title":"Hinge-Loss Markov Random Fields and Probabilistic Soft Logic","volume":"18","author":"Bach Stephen H.","year":"2015","unstructured":"Stephen H. Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2015. Hinge-Loss Markov Random Fields and Probabilistic Soft Logic. Journal of Machine Learning Research 18 (2015), 109:1--109:67.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140450"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/645504.656274"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2788613"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376715"},{"key":"e_1_2_1_9_1","volume-title":"Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17)","author":"Crankshaw Daniel","year":"2017","unstructured":"Daniel Crankshaw, Xin Wang, Guilio Zhou, Michael J. Franklin, Joseph E. Gonzalez, and Ion Stoica. 2017. Clipper: A Low-Latency Online Prediction Serving System. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, Boston, MA, 613--627. https:\/\/www.usenix.org\/conference\/nsdi17\/ technical-sessions\/presentation\/crankshaw"},{"key":"e_1_2_1_10_1","volume-title":"PPSN 2020, Leiden, The Netherlands, September 5--9, 2020, Proceedings, Part I. Springer, 448--469","author":"Dandl Susanne","year":"2020","unstructured":"Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. 2020. Multi-objective counterfactual explanations. In Parallel Problem Solving from Nature--PPSN XVI: 16th International Conference, PPSN 2020, Leiden, The Netherlands, September 5--9, 2020, Proceedings, Part I. Springer, 448--469."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2211477"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824039"},{"key":"e_1_2_1_13_1","volume-title":"The Case for Learned Provenance Graph Storage Systems. In 32nd USENIX Security Symposium (USENIX Security 23)","author":"Ding Hailun","year":"2023","unstructured":"Hailun Ding, Juan Zhai, Dong Deng, and Shiqing Ma. 2023. The Case for Learned Provenance Graph Storage Systems. In 32nd USENIX Security Symposium (USENIX Security 23). USENIX Association, Anaheim, CA, 3277--3294. https:\/\/www.usenix.org\/conference\/usenixsecurity23\/presentation\/ding-hailun-provenance"},{"key":"e_1_2_1_14_1","unstructured":"Pedro Domingos Dominik Jain Stanley Kok Daniel Lowd Lily Mihalkova Hoifung Poon Matthew Richardson Parag Singla Marc Sumner and Jue Wang. [n. d.]. Alchemy - Open Source AI. http:\/\/alchemy.cs.washington.edu\/alchemy1. html"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-01549-6"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3241978"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377501"},{"key":"e_1_2_1_18_1","volume-title":"Markov properties for graphical models with cycles and latent variables. arXiv preprint arXiv:1710.08775","author":"Forr\u00e9 Patrick","year":"2017","unstructured":"Patrick Forr\u00e9 and Joris M Mooij. 2017. Markov properties for graphical models with cycles and latent variables. arXiv preprint arXiv:1710.08775 (2017)."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of The 24th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research","volume":"693","author":"Satorras V\u00edctor Garcia","year":"2021","unstructured":"V\u00edctor Garcia Satorras and Max Welling. 2021. Neural Enhanced Belief Propagation on Factor Graphs. In Proceedings of The 24th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 130), Arindam Banerjee and Kenji Fukumizu (Eds.). PMLR, 685--693. https:\/\/proceedings.mlr.press\/v130\/garciasatorras21a. html"},{"key":"e_1_2_1_20_1","volume-title":"Making AI Forget You: Data Deletion in Machine Learning","author":"Ginart Antonio A.","unstructured":"Antonio A. Ginart, Melody Y. Guan, Gregory Valiant, and James Zou. 2019. Making AI Forget You: Data Deletion in Machine Learning. Curran Associates Inc., Red Hook, NY, USA."},{"key":"e_1_2_1_21_1","volume-title":"Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. journal of Computational and Graphical Statistics 24, 1","author":"Goldstein Alex","year":"2015","unstructured":"Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. 2015. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. journal of Computational and Graphical Statistics 24, 1 (2015), 44--65."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1265530.1265535"},{"key":"e_1_2_1_23_1","volume-title":"DARPA's explainable artificial intelligence (XAI) program. AI magazine 40, 2","author":"Gunning David","year":"2019","unstructured":"David Gunning and David Aha. 2019. DARPA's explainable artificial intelligence (XAI) program. AI magazine 40, 2 (2019), 44--58."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2020.24270"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings, Part IV 14","author":"Hendricks Lisa Anne","year":"2016","unstructured":"Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, and Trevor Darrell. 2016. Generating visual explanations. In Computer Vision--ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part IV 14. Springer, 3--19."},{"key":"e_1_2_1_26_1","volume-title":"Held in Conjunction with ICML 2020","author":"Holzinger Andreas","year":"2022","unstructured":"Andreas Holzinger, Randy Goebel, Ruth Fong, Taesup Moon, Klaus-Robert Mueller, and Wojciech Samek. 2022. xxAIbeyond explainable artificial intelligence. In xxAI-Beyond Explainable AI: International Workshop, Held in Conjunction with ICML 2020, July 18, 2020, Vienna, Austria, Revised and Extended Papers. Springer, 3--10."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2907116"},{"key":"e_1_2_1_28_1","volume-title":"Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems 29","author":"Kim Been","year":"2016","unstructured":"Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. 2016. Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems 29 (2016)."},{"key":"e_1_2_1_29_1","volume-title":"Probabilistic graphical models: principles and techniques","author":"Koller Daphne","unstructured":"Daphne Koller and Nir Friedman. 2009. Probabilistic graphical models: principles and techniques. MIT press."},{"key":"e_1_2_1_30_1","volume-title":"20th Annual Network and Distributed System Security Symposium, NDSS 2013","author":"Lee Kyu Hyung","year":"2013","unstructured":"Kyu Hyung Lee, Xiangyu Zhang, and Dongyan Xu. 2013. High Accuracy Attack Provenance via Binary-based Execution Partition. In 20th Annual Network and Distributed System Security Symposium, NDSS 2013, San Diego, California, USA, February 24--27, 2013. The Internet Society. https:\/\/www.ndss-symposium.org\/ndss2013\/high-accuracy-attackprovenance- binary-based-execution-partition"},{"key":"e_1_2_1_31_1","volume-title":"13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)","author":"Lee Yunseong","year":"2018","unstructured":"Yunseong Lee, Alberto Scolari, Byung-Gon Chun, Marco Domenico Santambrogio, Markus Weimer, and Matteo Interlandi. 2018. {PRETZEL}: Opening the black box of machine learning prediction serving systems. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 611--626."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2004.1267047"},{"key":"e_1_2_1_33_1","volume-title":"A unified approach to interpreting model predictions. Advances in neural information processing systems 30","author":"Lundberg Scott M","year":"2017","unstructured":"Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106291"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/3447689.3447712"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE48307.2020.00076"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372850"},{"key":"e_1_2_1_38_1","volume-title":"Loopy belief propagation for approximate inference: An empirical study. arXiv preprint arXiv:1301.6725","author":"Murphy Kevin","year":"2013","unstructured":"Kevin Murphy, Yair Weiss, and Michael I Jordan. 2013. Loopy belief propagation for approximate inference: An empirical study. arXiv preprint arXiv:1301.6725 (2013)."},{"key":"e_1_2_1_39_1","unstructured":"Rohan Paris. [n. d.]. Credit Score Classification. https:\/\/www.kaggle.com\/datasets\/parisrohan\/credit-score-classification"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452788"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137789"},{"key":"e_1_2_1_42_1","article-title":"Improving Reproducibility in Machine Learning Research (a Report from the NeurIPS 2019 Reproducibility Program)","volume":"22","author":"Pineau Joelle","year":"2021","unstructured":"Joelle Pineau, Philippe Vincent-Lamarre, Koustuv Sinha, Vincent Larivi\u00e8re, Alina Beygelzimer, Florence d'Alch\u00e9 Buc, Emily Fox, and Hugo Larochelle. 2021. Improving Reproducibility in Machine Learning Research (a Report from the NeurIPS 2019 Reproducibility Program). J. Mach. Learn. Res. 22, 1, Article 164 (jan 2021), 20 pages.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_1_43_1","volume-title":"Preferred Networks","author":"Inc.","year":"2021","unstructured":"Inc. Preferred Networks. 2021. Overview of PyTorch Autograd Engine. https:\/\/pytorch.org\/blog\/overview-of-pytorchautograd- engine"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAI.2021.3133846"},{"key":"e_1_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Christopher R\u00e9 and Dan Suciu. 2008. Approximate lineage for probabilistic databases. In PVLDB. 797--808.","DOI":"10.14778\/1453856.1453943"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-006-5833-1"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2021.3060483"},{"key":"e_1_2_1_50_1","volume-title":"NeurIPS","author":"Schelter Sebastian","year":"2017","unstructured":"Sebastian Schelter, Joos-Hendrik B\u00f6se, Johannes Kirschnick, Thoralf Klein, and Stephan Seufert. 2017. Automatically tracking metadata and provenance of machine learning experiments. In NeurIPS 2017. https:\/\/www.amazon.science\/ publications\/automatically-tracking-metadata-and-provenance-of-machine-learning-experiments"},{"key":"e_1_2_1_51_1","unstructured":"Jeremy Siek Lie-Quan Lee and Andrew Lumsdaine. [n. d.]. The Boost Graph Library (BGL). https:\/\/www.boost.org\/ doc\/libs\/1_80_0\/libs\/graph\/doc\/index.html"},{"key":"e_1_2_1_52_1","first-page":"1094","article-title":"Lifted First-Order Belief Propagation","volume":"8","author":"Singla Parag","year":"2008","unstructured":"Parag Singla and Pedro M Domingos. 2008. Lifted First-Order Belief Propagation.. In AAAI, Vol. 8. 1094--1099.","journal-title":"AAAI"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330885"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2017.109"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098039"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-57959-7"},{"key":"e_1_2_1_57_1","first-page":"841","article-title":"Counterfactual explanations without opening the black box: Automated decisions and the GDPR","volume":"31","author":"Wachter Sandra","year":"2017","unstructured":"Sandra Wachter, Brent Mittelstadt, and Chris Russell. 2017. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harv. JL & Tech. 31 (2017), 841.","journal-title":"Harv. JL & Tech."},{"key":"e_1_2_1_58_1","volume-title":"Susan B. Davidson, and Chen Chen.","author":"Wang Shaobo","year":"2020","unstructured":"Shaobo Wang, Hui Lyu, Jiachi Zhang, Chenyuan Wu, Xinyi Chen, Wenchao Zhou, Boon Thau Loo, Susan B. Davidson, and Chen Chen. 2020. Provenance for Probabilistic Logic Programs. In Extending Database Technology. 145--156."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3380571"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.14778\/3297753.3297763"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403085"},{"key":"e_1_2_1_62_1","volume-title":"Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint arXiv:1702.04595","author":"Zintgraf Luisa M","year":"2017","unstructured":"Luisa M Zintgraf, Taco S Cohen, Tameem Adel, and Max Welling. 2017. Visualizing deep neural network decisions: Prediction difference analysis. arXiv preprint arXiv:1702.04595 (2017)."}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3698826","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3698826","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T17:46:58Z","timestamp":1774979218000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3698826"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,18]]},"references-count":62,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,18]]}},"alternative-id":["10.1145\/3698826"],"URL":"https:\/\/doi.org\/10.1145\/3698826","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,18]]}}}