{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T02:57:51Z","timestamp":1777431471223,"version":"3.51.4"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T00:00:00Z","timestamp":1675814400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>With complexity of artificial intelligence systems increasing continuously in past years, studies to explain these complex systems have grown in popularity. While much work has focused on explaining artificial intelligence systems in popular domains such as classification and regression, explanations in the area of anomaly detection have only recently received increasing attention from researchers. In particular, explaining singular model decisions of a complex anomaly detector by highlighting which inputs were responsible for a decision, commonly referred to as local <jats:italic>post-hoc<\/jats:italic> feature relevance, has lately been studied by several authors. In this paper, we systematically structure these works based on their access to training data and the anomaly detection model, and provide a detailed overview of their operation in the anomaly detection domain. We demonstrate their performance and highlight their limitations in multiple experimental showcases, discussing current challenges and opportunities for future work in feature relevance XAI for anomaly detection.<\/jats:p>","DOI":"10.3389\/frai.2023.1099521","type":"journal-article","created":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T05:31:28Z","timestamp":1675834288000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":39,"title":["Feature relevance XAI in anomaly detection: Reviewing approaches and challenges"],"prefix":"10.3389","volume":"6","author":[{"given":"Julian","family":"Tritscher","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anna","family":"Krause","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andreas","family":"Hotho","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,2,8]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1109\/HSI.2018.8430788","article-title":"\u201cToward explainable deep neural network based anomaly detection,\u201d","author":"Amarasinghe","year":"2018","journal-title":"2018 11th International Conference on Human System Interaction (HSI)"},{"key":"B2","first-page":"169","article-title":"\u201cGradient-based attribution methods,\u201d","author":"Ancona","year":"2019","journal-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2021.115736","article-title":"Explaining anomalies detected by autoencoders using shapley additive explanations","author":"Antwarg","year":"2021","journal-title":"Expert Syst. Appl"},{"key":"B4","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable artificial intelligence (xai): concepts, taxonomies, opportunities and challenges toward responsible ai","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e0130140","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PLoS ONE"},{"key":"B6","first-page":"9592","article-title":"\u201cMvTEC ad-a comprehensive real-world dataset for unsupervised anomaly detection,\u201d","author":"Bergmann","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B7","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1145\/130385.130401","article-title":"\u201cA training algorithm for optimal margin classifiers,\u201d","author":"Boser","year":"1992","journal-title":"Proceedings of the Fifth Annual Workshop on Computational Learning Theory"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2007.1111","article-title":"Interpretable anomaly detection with diffi: depth-based isolation forest feature importance","author":"Carletti","year":"2020","journal-title":"arXiv preprint"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1541880.1541882","article-title":"Anomaly detection: a survey","volume":"41","author":"Chandola","year":"2009","journal-title":"ACM Comput. Surveys"},{"key":"B10","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/S0893-6080(01)00127-7","article-title":"A methodology to explain neural network classification","volume":"15","author":"F\u00e9raud","year":"2002","journal-title":"Neural networks"},{"key":"B11","author":"Goodfellow","year":"2016","journal-title":"Deep Learning"},{"key":"B12","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1609\/aimag.v38i3.2741","article-title":"European union regulations on algorithmic decision-making and a \u201cright to explanation\u201d'","volume":"38","author":"Goodman","year":"2017","journal-title":"AI Mag"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10618-022-00831-6","article-title":"Counterfactual explanations and how to find them: literature review and benchmarking","volume":"2022","author":"Guidotti","year":"2022","journal-title":"Data Min. Knowl. Discov"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-020-62724-2","article-title":"Resolving challenges in deep learning-based analyses of histopathological images using explanation methods","volume":"10","author":"H\u00e4gele","year":"2020","journal-title":"Sci. Rep"},{"key":"B15","first-page":"770","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B16","first-page":"51","article-title":"\u201cWhy are you weird? infusing interpretability in isolation forest for anomaly detection,\u201d","author":"Kartha","year":"2021","journal-title":"Proceedings of the Explainable Agency in AI Workshop (AAAI 2021)"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107198","article-title":"Towards explaining anomalies: a deep Taylor decomposition of one-class models","author":"Kauffmann","year":"","journal-title":"Pattern Recognit"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.10609","article-title":"The clever hans effect in anomaly detection","author":"Kauffmann","year":"","journal-title":"arXiv preprint"},{"key":"B19","author":"L\u00e9ger","year":"2007","journal-title":"Erpsim"},{"key":"B20","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1109\/ICDM.2008.17","article-title":"\u201cIsolation forest,\u201d","author":"Liu","year":"2008","journal-title":"2008 Eighth IEEE International Conference on Data Mining"},{"key":"B21","first-page":"2461","article-title":"\u201cContextual outlier interpretation,\u201d","author":"Liu","year":"2018","journal-title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI-18"},{"key":"B22","first-page":"4765","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","author":"Lundberg","year":"2017","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B23","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","article-title":"Explaining nonlinear classification decisions with deep taylor decomposition","volume":"65","author":"Montavon","year":"2017","journal-title":"Pattern Recognit"},{"key":"B24","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1145\/3533271.3561667","article-title":"\u201cRESHAPE: explaining accounting anomalies in financial statement audits by enhancing SHapley additive explanations,\u201d","author":"M\u00fcller","year":"2022","journal-title":"3rd ACM International Conference on AI in Finance"},{"key":"B25","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/CNS.2019.8802833","article-title":"\u201cGEE: a gradient-based explainable variational autoencoder for network anomaly detection,\u201d","author":"Nguyen","year":"2019","journal-title":"2019 IEEE Conference on Communications and Network Security (CNS)"},{"key":"B26","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1109\/PRML56267.2022.9882256","article-title":"\u201cTell Me Why-a systematic literature review on outlier explanation for tabular data,\u201d","author":"Nonnenmacher","year":"2022","journal-title":"2022 3rd International Conference on Pattern Recognition and Machine Learning (PRML)"},{"key":"B27","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1007\/978-3-642-45494-3_7","article-title":"\u201cValues of games with a priori unions,\u201d","author":"Owen","year":"1977","journal-title":"Mathematical Economics And game Theory"},{"key":"B28","doi-asserted-by":"publisher","author":"Panjei","year":"2022","DOI":"10.1007\/s00778-021-00721-1"},{"key":"B29","first-page":"1","article-title":"\u201cGeneral frameworks for anomaly detection explainability: comparative study,\u201d","author":"Ravi","year":"2021","journal-title":"2021 IEEE International Conference on Autonomous Systems (ICAS)"},{"key":"B30","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","article-title":"\u201cWhy should i trust you?: explaining the predictions of any classifier,\u201d","author":"Ribeiro","year":"2016","journal-title":"22nd ACM SIGKDD International of Conferences on Knowledge Discovery and Data Mining"},{"key":"B31","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1109\/TKDE.2007.190734","article-title":"Explaining classifications for individual instances","volume":"20","author":"Robnik-\u0160ikonja","year":"2008","journal-title":"IEEE Trans. Knowl. Data Eng"},{"key":"B32","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1214\/aoms\/1177728190","article-title":"\u201cRemarks on some nonparametric estimates of a density function,\u201d","author":"Rosenblatt","year":"1956","journal-title":"The Annals of Mathematical Statistics"},{"key":"B33","first-page":"4393","article-title":"\u201cDeep one-class classification,\u201d","author":"Ruff","year":"2018","journal-title":"Proceedings of the 35th International Conference on Machine Learning"},{"key":"B34","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis"},{"key":"B35","doi-asserted-by":"publisher","first-page":"1443","DOI":"10.1162\/089976601750264965","article-title":"Estimating the support of a high-dimensional distribution","volume":"13","author":"Sch\u00f6lkopf","year":"2001","journal-title":"Neural Comput"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1016\/j.mlwa.2021.100172","article-title":"Explainable outlier detection: what, for whom and why?","author":"Sejr","year":"2021","journal-title":"Mach. Learn. Appl"},{"key":"B37","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1023\/A:1008307919726","article-title":"Fernn: an algorithm for fast extraction of rules from neural networks","volume":"12","author":"Setiono","year":"2000","journal-title":"Appl. Intell"},{"key":"B38","doi-asserted-by":"publisher","first-page":"12","DOI":"10.2307\/j.ctv173f1fh.12","article-title":"A value for n-person games","volume":"69","author":"Shapley","year":"1997","journal-title":"Classics Game Theory"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1605.01713","article-title":"Not just a black box: learning important features through propagating activation differences","author":"Shrikumar","year":"2016","journal-title":"arXiv preprint"},{"key":"B40","article-title":"\u201cDeep inside convolutional networks: Visualising image classification models and saliency maps,\u201d","author":"Simonyan","year":"2014","journal-title":"ICLR (Workshop Poster)"},{"key":"B41","first-page":"9016","article-title":"\u201cInterpretable, multidimensional, multimodal anomaly detection with negative sampling for detection of device failure,\u201d","author":"Sipple","year":"2020","journal-title":"Proceedings of the 37th International Conference on Machine Learning"},{"key":"B42","first-page":"162","article-title":"\u201cA general-purpose method for applying explainable ai for anomaly detection,\u201d","author":"Sipple","year":"2022","journal-title":"International Symposium on Methodologies for Intelligent Systems"},{"key":"B43","first-page":"3319","article-title":"\u201cAxiomatic attribution for deep networks,\u201d","author":"Sundararajan","year":"2017","journal-title":"34th International Conferences on Machine Learning-Volume 70"},{"key":"B44","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1109\/ICDMW.2019.00117","article-title":"\u201cShapley values of reconstruction errors of pca for explaining anomaly detection,\u201d","author":"Takeishi","year":"2019","journal-title":"2019 International Conference on Data Mining Workshops (ICDMW)"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.04464","article-title":"On anomaly interpretation via shapley values","author":"Takeishi","year":"2020","journal-title":"arXiv preprint"},{"key":"B46","doi-asserted-by":"publisher","first-page":"1236","DOI":"10.1214\/aos\/1176348768","article-title":"Variable kernel density estimation","volume":"20","author":"Terrell","year":"1992","journal-title":"Ann. Stat"},{"key":"B47","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1111\/1467-9868.00196","article-title":"Probabilistic principal component analysis","volume":"61","author":"Tipping","year":"1999","journal-title":"J. R. Stat. Soc. B"},{"key":"B48","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","article-title":"A survey on explainable artificial intelligence (XAI): towards medical XAI","volume":"32","author":"Tjoa","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2206.04460","article-title":"Open ERP system data for occupational fraud detection","author":"Tritscher","year":"","journal-title":"arXiv preprint"},{"key":"B50","article-title":"\u201cTowards explainable occupational fraud detection,\u201d","author":"Tritscher","year":"","journal-title":"Workshop on Mining Data for Financial Applications"},{"key":"B51","article-title":"\u201cStudent-teacher feature pyramid matching for unsupervised anomaly detection,\u201d","author":"Wang","year":"2021","journal-title":"The 32nd British Machine Vision Conference BMVC 2021"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1016\/j.datak.2021.101946","article-title":"Anomaly explanation: a review","author":"Yepmo","year":"2022","journal-title":"Data Knowl. Eng"},{"key":"B53","doi-asserted-by":"crossref","first-page":"1991","DOI":"10.1109\/BigData47090.2019.9005989","article-title":"\u201cACE-an anomaly contribution explainer for cyber-security applications,\u201d","author":"Zhang","year":"2019","journal-title":"2019 IEEE International Conference on Big Data (Big Data)"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1099521\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,8]],"date-time":"2023-02-08T05:32:06Z","timestamp":1675834326000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1099521\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,8]]},"references-count":53,"alternative-id":["10.3389\/frai.2023.1099521"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1099521","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,8]]},"article-number":"1099521"}}