{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T22:48:37Z","timestamp":1785192517971,"version":"3.55.0"},"reference-count":63,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T00:00:00Z","timestamp":1691452800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002341","name":"Academy of Finland","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002341","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>In recent years the use of complex machine learning has increased drastically. These complex black box models trade interpretability for accuracy. The lack of interpretability is troubling for, e.g., socially sensitive, safety-critical, or knowledge extraction applications. In this paper, we propose a new explanation method, SLISE, for interpreting predictions from black box models. SLISE can be used with any black box model (model-agnostic), does not require any modifications to the black box model (<jats:italic>post-hoc<\/jats:italic>), and explains individual predictions (local). We evaluate our method using real-world datasets and compare it against other model-agnostic, local explanation methods. Our approach solves shortcomings in other related explanation methods by only using existing data instead of sampling new, artificial data. The method also generates more generalizable explanations and is usable without modification across various data domains.<\/jats:p>","DOI":"10.3389\/fcomp.2023.1143904","type":"journal-article","created":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T08:33:11Z","timestamp":1691483591000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Explaining any black box model using real data"],"prefix":"10.3389","volume":"5","author":[{"given":"Anton","family":"Bj\u00f6rklund","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Henelius","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emilia","family":"Oikarinen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kimmo","family":"Kallonen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai","family":"Puolam\u00e4ki","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,8,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1007\/s10115-017-1116-3","article-title":"Auditing black-box models for indirect influence","volume":"54","author":"Adler","year":"2018","journal-title":"Knowledge Inform. Syst."},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1806.08049","article-title":"On the robustness of interpretability methods","author":"Alvarez-Melis","year":"2018","journal-title":"arXiv preprint arXiv:1806.08049"},{"key":"B3","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/0304-3975(94)00254-G","article-title":"The complexity and approximability of finding maximum feasible subsystems of linear relations","volume":"147","author":"Amaldi","year":"1995","journal-title":"Theoret. Comput. Sci."},{"key":"B4","doi-asserted-by":"publisher","first-page":"e479","DOI":"10.7717\/peerj-cs.479","article-title":"To trust or not to trust an explanation: using LEAF to evaluate local linear XAI methods","volume":"7","author":"Amparore","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"B5","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-58412-1","volume-title":"Complexity and Approximation","author":"Ausiello","year":"1999"},{"key":"B6","doi-asserted-by":"publisher","first-page":"e0130140","DOI":"10.1371\/journal.pone.0130140","article-title":"On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation","volume":"10","author":"Bach","year":"2015","journal-title":"PLoS ONE"},{"key":"B7","unstructured":"How to explain individual classification decisions18031831 BaehrensD. SchroeterT. HarmelingS. KawanabeM. HansenK. M\u00fcllerK.-R. J. Mach. Learn. Res.112010"},{"key":"B8","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1007\/978-3-030-33778-0_27","article-title":"\u201cSparse robust regression for explaining classifiers,\u201d","author":"Bj\u00f6rklund","year":"2019","journal-title":"Discovery Science"},{"key":"B9","doi-asserted-by":"publisher","first-page":"781","DOI":"10.1007\/s10618-022-00819-2","article-title":"Robust regression via error tolerance","volume":"36","author":"Bj\u00f6rklund","year":"2022","journal-title":"Data Mining Knowledge Discov."},{"key":"B10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10994-022-06261-1","article-title":"SLISEMAP: supervised dimensionality reduction through local explanations","volume":"112","author":"Bj\u00f6rklund","year":"2023","journal-title":"Mach. Learn."},{"key":"B11","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"B12","author":"Breiman","year":"2002","journal-title":"Manual on Setting Up, Using, and Understanding Random Forests v3.1"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00024","article-title":"Thread: circuits","author":"Cammarata","year":"2020","journal-title":"Distill"},{"key":"B14","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1145\/2783258.2788613","article-title":"\u201cIntelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission,\u201d","volume-title":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Caruana","year":"2015"},{"key":"B15","first-page":"1383","article-title":"\u201cConcise explanations of neural networks using adversarial training,\u201d","volume-title":"Proceedings of the 37th International Conference on Machine Learning","author":"Chalasani","year":"2020"},{"key":"B16","volume-title":"Performance of Quark\/Gluon Discrimination in 8 TeV pp Data","year":"2013"},{"key":"B17","year":"2017","journal-title":"Simulated Dataset {QCD\\_Pt\\-15to3000\\_TuneZ2star\\_Flat\\_8TeV\\_pythia6} in {AODSIM} Format for 2012 Collision Data"},{"key":"B18","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1007\/JHEP02(2015)118","article-title":"Jet-images: computer vision inspired techniques for jet tagging","volume":"2015","author":"Cogan","year":"2015","journal-title":"J. High Energy Phys."},{"key":"B19","doi-asserted-by":"crossref","first-page":"2921","DOI":"10.1109\/IJCNN.2017.7966217","article-title":"\u201cEMNIST: Extending MNIST to handwritten letters,\u201d","volume-title":"2017 International Joint Conference on Neural Networks (IJCNN)","author":"Cohen","year":"2017"},{"key":"B20","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1109\/SP.2016.42","article-title":"\u201cAlgorithmic transparency via quantitative input influence: theory and experiments with learning systems,\u201d","volume-title":"2016 IEEE Symposium on Security and Privacy (SP)","author":"Datta","year":"2016"},{"key":"B21","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1007\/JHEP07(2016)069","article-title":"Jet-images \u2014 deep learning edition","volume":"2016","author":"de Oliveira","year":"2016","journal-title":"J. High Energy Phys."},{"key":"B22","volume-title":"Visualizing Higher-Layer Features of a Deep Network","author":"Erhan","year":"2009"},{"key":"B23","doi-asserted-by":"crossref","first-page":"3449","DOI":"10.1109\/ICCV.2017.371","article-title":"\u201cInterpretable explanations of black boxes by meaningful perturbation,\u201d","volume-title":"2017 IEEE International Conference on Computer Vision (ICCV)","author":"Fong","year":"2017"},{"key":"B24","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1609\/aimag.v38i3.2741","article-title":"European Union Regulations on algorithmic decision-making and a \u201cright to explanation\u201d","volume":"38","author":"Goodman","year":"2017","journal-title":"AI Mag."},{"key":"B25","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/978-3-030-46150-8_12","article-title":"\u201cBlack box explanation by learning image exemplars in the latent feature space,\u201d","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"Guidotti","year":"2020"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1805.10820","article-title":"Local rule-based explanations of black box decision systems","author":"Guidotti","year":"2018","journal-title":"arXiv preprint arXiv:1805.10820"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3236009","article-title":"A survey of methods for explaining black box models","volume":"51","author":"Guidotti","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"B28","doi-asserted-by":"crossref","first-page":"391","DOI":"10.2307\/2530428","article-title":"Building multiple regression models interactively","volume":"37","author":"Henderson","year":"1981","journal-title":"Biometrics"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1503","DOI":"10.1007\/s10618-014-0368-8","article-title":"A peek into the black box: exploring classifiers by randomization","volume":"28","author":"Henelius","year":"2014","journal-title":"Data Mining Knowledge Discov."},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1707.07576","article-title":"Interpreting classifiers through attribute interactions in datasets","author":"Henelius","year":"2017","journal-title":"arXiv preprint arXiv:1707.07576"},{"key":"B31","volume-title":"Helsinki OpenData Tuples","year":"2018"},{"key":"B32","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1007\/s11222-021-10057-z","article-title":"Unrestricted permutation forces extrapolation: variable importance requires at least one more model, or there is no free variable importance","volume":"31","author":"Hooker","year":"2021","journal-title":"Stat. Comput."},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.11626","article-title":"Bridging adversarial robustness and gradient interpretability","author":"Kim","year":"2019","journal-title":"arXiv preprint arXiv:1903.11626"},{"key":"B34","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1007\/JHEP01(2017)110","article-title":"Deep learning in color: towards automated Quark\/Gluon jet discrimination","volume":"2017","author":"Komiske","year":"2017","journal-title":"J. High Energy Phys."},{"key":"B35","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1007\/978-3-030-64949-4_8","article-title":"\u201cSurvey of explainable machine learning with visual and granular methods beyond quasi-explanations,\u201d","volume-title":"Interpretable Artificial Intelligence: A Perspective of Granular Computing","author":"Kovalerchuk","year":"2021"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1811.10799","article-title":"What is interpretable? Using machine learning to design interpretable decision-support systems","author":"Lahav","year":"2019","journal-title":"arXiv preprint arXiv:1811.10799"},{"key":"B37","doi-asserted-by":"crossref","first-page":"1675","DOI":"10.1145\/2939672.2939874","article-title":"\u201cInterpretable decision sets: a joint framework for description and prediction,\u201d","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Lakkaraju","year":"2016"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.1038\/s41467-019-08987-4","article-title":"Unmasking Clever Hans predictors and assessing what machines really learn","volume":"10","author":"Lapuschkin","year":"2019","journal-title":"Nat. Commun."},{"key":"B39","author":"Laugel","year":"2018"},{"key":"B40","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1145\/3236386.3241340","article-title":"The mythos of model interpretability: in machine learning, the concept of interpretability is both important and slippery","volume":"16","author":"Lipton","year":"2018","journal-title":"Queue"},{"key":"B41","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 30","author":"Lundberg","year":"2017"},{"key":"B42","first-page":"142","article-title":"\u201cLearning word vectors for sentiment analysis,\u201d","volume-title":"Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies","author":"Maas","year":"2011"},{"key":"B43","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/978-3-319-14612-6_4","article-title":"\u201cOn the link between Gaussian homotopy continuation and convex envelopes,\u201d","volume-title":"Energy Minimization Methods in Computer Vision and Pattern Recognition","author":"Mobahi","year":"2015"},{"key":"B44","volume-title":"Interpretable Machine Learning: A Guide for Making Black Box Models Interpretable","author":"Molnar","year":"2019"},{"key":"B45","doi-asserted-by":"crossref","first-page":"7992","DOI":"10.1109\/ICPR48806.2021.9412511","article-title":"\u201cA multilinear sampling algorithm to estimate Shapley values,\u201d","author":"Okhrati","year":"2021","journal-title":"2020 25th International Conference on Pattern Recognition (ICPR)"},{"key":"B46","doi-asserted-by":"publisher","first-page":"e00024","DOI":"10.23915\/distill.00024.003","article-title":"An overview of early vision in inceptionV1","volume":"5","author":"Olah","year":"2020","journal-title":"Distill"},{"key":"B47","doi-asserted-by":"publisher","first-page":"149","DOI":"10.3934\/mfc.2018008","article-title":"How convolutional neural networks see the world \u2014 A survey of convolutional neural network visualization methods","volume":"1","author":"Qin","year":"2018","journal-title":"Math. Found. Comput."},{"key":"B48","first-page":"1135","article-title":"\u201c\u201cWhy should I trust you?\u201d: explaining the predictions of any classifier,\u201d","author":"Ribeiro","year":"2016"},{"key":"B49","first-page":"1527","article-title":"\u201cAnchors: high-precision model-agnostic explanations,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Ribeiro","year":"2018"},{"key":"B50","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1002\/widm.2","article-title":"Robust statistics for outlier detection","volume":"1","author":"Rousseeuw","year":"2011","journal-title":"WIREs Data Mining Knowledge Discov."},{"key":"B51","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1080\/01621459.1990.10474920","article-title":"Unmasking multivariate outliers and leverage points","volume":"85","author":"Rousseeuw","year":"1990","journal-title":"J. Am. Stat. Assoc."},{"key":"B52","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1111\/0272-4332.00040","article-title":"Sensitivity analysis for importance assessment","volume":"22","author":"Saltelli","year":"2002","journal-title":"Risk Anal."},{"key":"B53","first-page":"456","article-title":"\u201cOptimizing costly functions with simple constraints: a limited-memory projected quasi-newton algorithm,\u201d","volume-title":"Proceedings of the Twelth International Conference on Artificial Intelligence and Statistics","author":"Schmidt","year":"2009"},{"key":"B54","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1145\/3375627.3375830","article-title":"\u201cFooling LIME and SHAP: adversarial attacks on post hoc explanation methods,\u201d","volume-title":"Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society","author":"Slack","year":"2020"},{"key":"B55","doi-asserted-by":"crossref","first-page":"2818","DOI":"10.1109\/CVPR.2016.308","article-title":"\u201cRethinking the inception architecture for computer vision,\u201d","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Szegedy","year":"2016"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1801.08640","article-title":"Considerations when learning additive explanations for black-box models","author":"Tan","year":"2021","journal-title":"arXiv preprint arXiv:1801.08640"},{"key":"B57","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the Lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. B Methodol."},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1306.6677","article-title":"Supersparse linear integer models for interpretable classification","author":"Ustun","year":"2014","journal-title":"arXiv preprint arXiv:1306.6677"},{"key":"B59","doi-asserted-by":"publisher","first-page":"841","DOI":"10.2139\/ssrn.3063289","article-title":"Counterfactual explanations without opening the black box: automated decisions and the GDPR","volume":"31","author":"Wachter","year":"2017","journal-title":"Harvard J. Law Technol."},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1606.05718","article-title":"Deep learning for identifying metastatic breast cancer","author":"Wang","year":"2016","journal-title":"arXiv preprint arXiv:1606.05718"},{"key":"B61","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/s11229-022-03485-5","article-title":"Conceptual challenges for interpretable machine learning","volume":"200","author":"Watson","year":"2022","journal-title":"Synthese"},{"key":"B62","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.346","article-title":"Improved heterogeneous distance functions","volume":"6","author":"Wilson","year":"1997","journal-title":"J. Artif. Intell. Res."},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.14545","article-title":"Explainable deep learning: a field guide for the uninitiated","author":"Xie","year":"2020","journal-title":"arXiv preprint arXiv:2004.14545"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1143904\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,25]],"date-time":"2024-10-25T21:17:14Z","timestamp":1729891034000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1143904\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,8]]},"references-count":63,"alternative-id":["10.3389\/fcomp.2023.1143904"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2023.1143904","relation":{},"ISSN":["2624-9898"],"issn-type":[{"value":"2624-9898","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,8]]},"article-number":"1143904"}}