{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:20:58Z","timestamp":1740108058917,"version":"3.37.3"},"reference-count":43,"publisher":"Springer Science and Business Media LLC","issue":"14","license":[{"start":{"date-parts":[[2022,10,26]],"date-time":"2022-10-26T00:00:00Z","timestamp":1666742400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,10,26]],"date-time":"2022-10-26T00:00:00Z","timestamp":1666742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"FWF","award":["P 31988"],"award-info":[{"award-number":["P 31988"]}]},{"DOI":"10.13039\/100019180","name":"HORIZON EUROPE European Research Council","doi-asserted-by":"publisher","award":["101019375"],"award-info":[{"award-number":["101019375"]}],"id":[{"id":"10.13039\/100019180","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Given the rise of deep learning and its inherent black-box nature, the desire to interpret these systems and explain their behaviour became increasingly more prominent. The main idea of so-called explainers is to identify which features of particular samples have the most influence on a classifier\u2019s prediction, and present them as explanations. Evaluating explainers, however, is difficult, due to reasons such as a lack of ground truth. In this work, we construct adversarial examples to check the plausibility of explanations, perturbing input deliberately to change a classifier\u2019s prediction. This allows us to investigate whether explainers are able to detect these perturbed regions as the parts of an input that strongly influence a particular classification. Our results from the audio and image domain suggest that the investigated explainers often fail to identify the input regions most relevant for a prediction; hence, it remains questionable whether explanations are useful or potentially misleading.<\/jats:p>","DOI":"10.1007\/s00521-022-07918-7","type":"journal-article","created":{"date-parts":[[2022,10,26]],"date-time":"2022-10-26T16:03:17Z","timestamp":1666800197000},"page":"10011-10029","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Constructing adversarial examples to investigate the plausibility of explanations in deep audio and image classifiers"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7585-7266","authenticated-orcid":false,"given":"Katharina","family":"Hoedt","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Verena","family":"Praher","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arthur","family":"Flexer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gerhard","family":"Widmer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,10,26]]},"reference":[{"issue":"10","key":"7918_CR1","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1145\/3233231","volume":"61","author":"ZC Lipton","year":"2018","unstructured":"Lipton ZC (2018) The mythos of model interpretability. Commun ACM 61(10):36\u201343. https:\/\/doi.org\/10.1145\/3233231","journal-title":"Commun ACM"},{"key":"7918_CR2","unstructured":"Adebayo J, Gilmer J, Muelly M, Goodfellow IJ, Hardt M, Kim B (2018) Sanity checks for saliency maps. In: Advances in neural information processing systems 31: annual conference on neural information processing systems 2018, NeurIPS, pp 9525\u20139536"},{"key":"7918_CR3","doi-asserted-by":"publisher","unstructured":"Neely M, Schouten SF, Bleeker, MJR, Lucic A (2021) Order in the court: explainable AI methods prone to disagreement. CoRR abs\/2105.03287. https:\/\/doi.org\/10.48550\/arXiv.2105.03287","DOI":"10.48550\/arXiv.2105.03287"},{"key":"7918_CR4","doi-asserted-by":"publisher","unstructured":"Krishna S, Han T, Gu A, Pombra J, Jabbari S, Wu S, Lakkaraju H (2022) The disagreement problem in explainable machine learning: a practitioner\u2019s perspective. CoRR abs\/2202.01602. https:\/\/doi.org\/10.48550\/arXiv.2202.01602","DOI":"10.48550\/arXiv.2202.01602"},{"issue":"8","key":"7918_CR5","doi-asserted-by":"publisher","first-page":"832","DOI":"10.3390\/electronics8080832","volume":"8","author":"DV Carvalho","year":"2019","unstructured":"Carvalho DV, Pereira EM, Cardoso JS (2019) Machine learning interpretability: a survey on methods and metrics. Electronics 8(8):832. https:\/\/doi.org\/10.3390\/electronics8080832","journal-title":"Electronics"},{"key":"7918_CR6","doi-asserted-by":"publisher","unstructured":"Praher V, Prinz K, Flexer A, Widmer G (2021) On the veracity of local, model-agnostic explanations in audio classification: targeted investigations with adversarial examples. In: Proceedings of the 22nd international society for music information retrieval conference, ISMIR, pp. 531\u2013538. https:\/\/doi.org\/10.5281\/zenodo.5624471","DOI":"10.5281\/zenodo.5624471"},{"issue":"11","key":"7918_CR7","doi-asserted-by":"publisher","first-page":"2660","DOI":"10.1109\/TNNLS.2016.2599820","volume":"28","author":"W Samek","year":"2017","unstructured":"Samek W, Binder A, Montavon G, Lapuschkin S, M\u00fcller K (2017) Evaluating the visualization of what a deep neural network has learned. IEEE Trans Neural Netw Learn Syst 28(11):2660\u20132673. https:\/\/doi.org\/10.1109\/TNNLS.2016.2599820","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"7918_CR8","unstructured":"Hooker S, Erhan D, Kindermans P, Kim B (2019) A benchmark for interpretability methods in deep neural networks. In: Advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS, pp 9734\u20139745"},{"key":"7918_CR9","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.inffus.2021.11.008","volume":"81","author":"L Arras","year":"2022","unstructured":"Arras L, Osman A, Samek W (2022) CLEVR-XAI: a benchmark dataset for the ground truth evaluation of neural network explanations. Inf Fusion 81:14\u201340. https:\/\/doi.org\/10.1016\/j.inffus.2021.11.008","journal-title":"Inf Fusion"},{"key":"7918_CR10","doi-asserted-by":"publisher","unstructured":"Yona G, Greenfeld D (2021) Revisiting sanity checks for saliency maps. CoRR abs\/2110.14297. https:\/\/doi.org\/10.48550\/arXiv.2110.14297","DOI":"10.48550\/arXiv.2110.14297"},{"key":"7918_CR11","doi-asserted-by":"publisher","unstructured":"Hase P, Bansal M (2020) Evaluating explainable AI: which algorithmic explanations help users predict model behavior? In: Proceedings of the 58th annual meeting of the association for computational linguistics, ACL, pp 5540\u20135552. Association for computational linguistics, Online. https:\/\/doi.org\/10.18653\/v1\/2020.acl-main.491","DOI":"10.18653\/v1\/2020.acl-main.491"},{"key":"7918_CR12","doi-asserted-by":"publisher","unstructured":"Jesus SM, Bel\u00e9m C, Balayan V, Bento J, Saleiro P, Bizarro P, Gama J (2021) How can I choose an explainer?: an application-grounded evaluation of post-hoc explanations. In: Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, FAccT, pp 805\u2013815. ACM, New York, NY, United States. https:\/\/doi.org\/10.1145\/3442188.3445941","DOI":"10.1145\/3442188.3445941"},{"key":"7918_CR13","doi-asserted-by":"publisher","unstructured":"Zhou Y, Booth S, Ribeiro MT, Shah J (2021) Do feature attribution methods correctly attribute features? CoRR abs\/2104.14403. https:\/\/doi.org\/10.48550\/arXiv.2104.14403","DOI":"10.48550\/arXiv.2104.14403"},{"key":"7918_CR14","doi-asserted-by":"publisher","unstructured":"G\u00f6pfert JP, Wersing H, Hammer B (2019) Recovering localized adversarial attacks. In: Artificial neural networks and machine learning - theoretical neural computation - proceedings of the 28th international conference on artificial neural networks, ICANN. Lecture notes in computer science, vol 11727, pp 302\u2013311. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-030-30487-4_24","DOI":"10.1007\/978-3-030-30487-4_24"},{"key":"7918_CR15","unstructured":"Simonyan K, Vedaldi A, Zisserman A (2014) Deep inside convolutional networks: visualising image classification models and saliency maps. In: Workshop track proceedings of the 2nd international conference on learning representations, ICLR"},{"key":"7918_CR16","unstructured":"Springenberg JT, Dosovitskiy A, Brox T, Riedmiller MA (2015) Striving for Simplicity: the all convolutional net. In: Workshop track proceedings of the 3rd international conference on learning representations, ICLR"},{"key":"7918_CR17","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C (2016) \u201cWhy Should I Trust You?\u201d: explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 1135\u20131144. ACM, New York, NY, United States. https:\/\/doi.org\/10.1145\/2939672.2939778","DOI":"10.1145\/2939672.2939778"},{"key":"7918_CR18","doi-asserted-by":"publisher","unstructured":"Schl\u00fcter J, Lehner B (2018) Zero-mean convolutions for level-invariant singing voice detection. In: Proceedings of the 19th international society for music information retrieval conference, ISMIR, pp 321\u2013326. https:\/\/doi.org\/10.5281\/zenodo.1492413","DOI":"10.5281\/zenodo.1492413"},{"key":"7918_CR19","doi-asserted-by":"publisher","unstructured":"Mishra S, Sturm BL, Dixon S (2017) Local interpretable model-agnostic explanations for music content analysis. In: Proceedings of the 18th international society for music information retrieval conference, ISMIR, pp 537\u2013543. https:\/\/doi.org\/10.5281\/zenodo.1417387","DOI":"10.5281\/zenodo.1417387"},{"key":"7918_CR20","doi-asserted-by":"publisher","unstructured":"Chettri B, Sturm BL, Benetos E (2018) Analysing replay spoofing countermeasure performance under varied conditions. In: Proceedings of the 28th IEEE international workshop on machine learning for signal processing, MLSP, pp 1\u20136. https:\/\/doi.org\/10.1109\/MLSP.2018.8516968","DOI":"10.1109\/MLSP.2018.8516968"},{"key":"7918_CR21","doi-asserted-by":"publisher","unstructured":"Haunschmid V, Chowdhury S, Widmer G (2019) Two-level explanations in music emotion recognition. Machine learning for music discovery workshop, ML4MD at ICML2019. https:\/\/doi.org\/10.48550\/arXiv.1905.11760","DOI":"10.48550\/arXiv.1905.11760"},{"key":"7918_CR22","doi-asserted-by":"publisher","unstructured":"Mishra S, Benetos E, Sturm BL, Dixon S (2020) Reliable local explanations for machine listening. In: Proceedings of the 2020 international joint conference on neural networks, IJCNN, pp 1\u20138. https:\/\/doi.org\/10.1109\/IJCNN48605.2020.9207444","DOI":"10.1109\/IJCNN48605.2020.9207444"},{"key":"7918_CR23","doi-asserted-by":"publisher","unstructured":"Smilkov D, Thorat N, Kim B, Vi\u00e9gas FB, Wattenberg M (2017) SmoothGrad: removing noise by adding noise. CoRR abs\/1706.03825. https:\/\/doi.org\/10.48550\/arXiv.1706.03825","DOI":"10.48550\/arXiv.1706.03825"},{"issue":"11","key":"7918_CR24","doi-asserted-by":"publisher","first-page":"2274","DOI":"10.1109\/TPAMI.2012.120","volume":"34","author":"R Achanta","year":"2012","unstructured":"Achanta R, Shaji A, Smith K, Lucchi A, Fua P, S\u00fcsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274\u20132282. https:\/\/doi.org\/10.1109\/TPAMI.2012.120","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"7918_CR25","unstructured":"Molnar C (2022) Interpretable machine learning, 2nd edn. christophm.github.io\/interpretable-ml-book\/"},{"key":"7918_CR26","unstructured":"Sundararajan M, Taly A, Yan Q (2017) Axiomatic Attribution for Deep Networks. In: Proceedings of the 34th international conference on machine learning, ICML. proceedings of machine learning research, vol 70, PMLR, Sydney, NSW, Australia, pp 3319\u20133328"},{"key":"7918_CR27","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. In: Proceedings of the 6th international conference on learning representations, ICLR"},{"issue":"1","key":"7918_CR28","doi-asserted-by":"publisher","first-page":"93","DOI":"10.5334\/tismir.85","volume":"4","author":"K Prinz","year":"2021","unstructured":"Prinz K, Flexer A, Widmer G (2021) On end-to-end white-box adversarial attacks in music information retrieval. Trans Int Soc Music Inf Retr 4(1):93\u2013104. https:\/\/doi.org\/10.5334\/tismir.85","journal-title":"Trans Int Soc Music Inf Retr"},{"key":"7918_CR29","doi-asserted-by":"publisher","unstructured":"Carlini N, Wagner DA (2018) Audio adversarial examples: targeted attacks on speech-to-text. In: Proceedings of the IEEE security and privacy workshops, SP workshops, pp 1\u20137. IEEE computer society, Washington, DC, United States. https:\/\/doi.org\/10.1109\/SPW.2018.00009","DOI":"10.1109\/SPW.2018.00009"},{"key":"7918_CR30","doi-asserted-by":"crossref","unstructured":"Ramona M, Richard G, David B (2008) Vocal detection in music with support vector machines. In: Proceedings of the IEEE international conference on acoustics, speech, and signal processing, ICASSP, pp 1885\u20131888","DOI":"10.1109\/ICASSP.2008.4518002"},{"key":"7918_CR31","doi-asserted-by":"publisher","unstructured":"Schl\u00fcter J, Grill T (2015) Exploring data augmentation for improved singing voice detection with neural networks. In: Proceedings of the 16th international society for music information retrieval conference, ISMIR, pp 121\u2013126. https:\/\/doi.org\/10.5281\/zenodo.1417745","DOI":"10.5281\/zenodo.1417745"},{"issue":"3","key":"7918_CR32","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","volume":"115","author":"O Russakovsky","year":"2015","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS, Berg AC, Fei-Fei L (2015) ImageNet large scale visual recognition challenge. Int J Comput Vision 115(3):211\u2013252. https:\/\/doi.org\/10.1007\/s11263-015-0816-y","journal-title":"Int J Comput Vision"},{"key":"7918_CR33","doi-asserted-by":"publisher","unstructured":"Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition, CVPR, pp 2818\u20132826. IEEE computer society, Washington, DC, United States. https:\/\/doi.org\/10.1109\/CVPR.2016.308","DOI":"10.1109\/CVPR.2016.308"},{"key":"7918_CR34","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition, CVPR, pp 770\u2013778. IEEE computer society, Washington, DC, United States. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"7918_CR35","doi-asserted-by":"publisher","unstructured":"Krizhevsky A (2014) One weird trick for parallelizing convolutional neural networks. CoRR abs\/1404.5997. https:\/\/doi.org\/10.48550\/arXiv.1404.5997","DOI":"10.48550\/arXiv.1404.5997"},{"key":"7918_CR36","unstructured":"Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: Bengio Y, LeCun, Y. (eds) Proceedings of the 3rd international conference on learning representations, ICLR"},{"key":"7918_CR37","doi-asserted-by":"publisher","unstructured":"Huang G, Liu Z, van\u00a0der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition, CVPR, pp 2261\u20132269. IEEE computer society, Washington, DC, United States. https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"},{"key":"7918_CR38","doi-asserted-by":"publisher","unstructured":"Zintgraf LM, Cohen TS, Welling M (2016) A new method to visualize deep neural networks. CoRR abs\/1603.02518. https:\/\/doi.org\/10.48550\/arXiv.1603.02518","DOI":"10.48550\/arXiv.1603.02518"},{"key":"7918_CR39","doi-asserted-by":"publisher","unstructured":"Zhou B, Khosla A, Lapedriza, \u00c0., Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition, CVPR, pp. 2921\u20132929. IEEE computer society, Washington, DC, United States. https:\/\/doi.org\/10.1109\/CVPR.2016.319","DOI":"10.1109\/CVPR.2016.319"},{"key":"7918_CR40","doi-asserted-by":"crossref","unstructured":"Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the 2017 IEEE international conference on computer vision, ICCV, pp 618\u2013626. IEEE computer society, Washington, DC, United States. 10.1109\/ICCV.2017.74","DOI":"10.1109\/ICCV.2017.74"},{"key":"7918_CR41","doi-asserted-by":"publisher","unstructured":"Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Fleet DJ, Pajdla T, Schiele B, Tuytelaars, T. (eds.) Proceedings of the 13th European conference on computer vision, ECCV. Lecture notes in computer science, vol 8689, pp 818\u2013833. Springer, Cham. https:\/\/doi.org\/10.1007\/978-3-319-10590-1_53","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"7918_CR42","doi-asserted-by":"publisher","first-page":"102364","DOI":"10.1016\/j.media.2022.102364","volume":"77","author":"MS Ayhan","year":"2022","unstructured":"Ayhan MS, K\u00fcmmerle LB, K\u00fchlewein L, Inhoffen W, Aliyeva G, Ziemssen F, Berens P (2022) Clinical validation of saliency maps for understanding deep neural networks in ophthalmology. Med Image Anal 77:102364. https:\/\/doi.org\/10.1016\/j.media.2022.102364","journal-title":"Med Image Anal"},{"key":"7918_CR43","doi-asserted-by":"publisher","unstructured":"Gevaert A, Rousseau A, Becker T, Valkenborg D, Bie TD, Saeys Y (2022) Evaluating feature attribution methods in the image domain. CoRR abs\/2202.12270. https:\/\/doi.org\/10.48550\/arXiv.2202.12270","DOI":"10.48550\/arXiv.2202.12270"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07918-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-022-07918-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-022-07918-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,25]],"date-time":"2023-04-25T00:08:09Z","timestamp":1682381289000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-022-07918-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,26]]},"references-count":43,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["7918"],"URL":"https:\/\/doi.org\/10.1007\/s00521-022-07918-7","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"type":"print","value":"0941-0643"},{"type":"electronic","value":"1433-3058"}],"subject":[],"published":{"date-parts":[[2022,10,26]]},"assertion":[{"value":"1 April 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 September 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 October 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 January 2023","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Update","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Missing Open Access funding information has been added in the Funding Note.","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors of the manuscript confirm that there are no conflict of interest to declare.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}