{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T07:32:10Z","timestamp":1749713530686,"version":"3.37.3"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T00:00:00Z","timestamp":1691020800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T00:00:00Z","timestamp":1691020800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001942","name":"CHIST-ERA","doi-asserted-by":"publisher","award":["CHIST-ERA-19-XAI-009"],"award-info":[{"award-number":["CHIST-ERA-19-XAI-009"]}],"id":[{"id":"10.13039\/501100001942","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Universit\u00e0 degli Studi di Roma La Sapienza"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Softw Big Sci"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Neural networks are commonly defined as \u2018black-box\u2019 models, meaning that the mechanism describing how they give predictions and perform decisions is not immediately clear or even understandable by humans. Therefore, Explainable Artificial Intelligence (xAI) aims at overcoming such limitation by providing explanations to Machine Learning (ML) algorithms and, consequently, making their outcomes reliable for users. However, different xAI methods may provide different explanations, both from a quantitative and a qualitative point of view, and the heterogeneity of approaches makes it difficult for a domain expert to select and interpret their result. In this work, we consider this issue in the context of a high-energy physics (HEP) use-case concerning muonic motion. In particular, we explored an array of xAI methods based on different approaches, and we tested their capabilities in our use-case. As a result, we obtained an array of potentially easy-to-understand and human-readable explanations of models\u2019 predictions, and for each of them we describe strengths and drawbacks in this particular scenario, providing an interesting atlas on the convergent application of multiple xAI algorithms in a realistic context.<\/jats:p>","DOI":"10.1007\/s41781-023-00102-z","type":"journal-article","created":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T13:01:41Z","timestamp":1691067701000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Convergent Approaches to AI Explainability for HEP Muonic Particles Pattern Recognition"],"prefix":"10.1007","volume":"7","author":[{"given":"Leandro","family":"Maglianella","sequence":"first","affiliation":[]},{"given":"Lorenzo","family":"Nicoletti","sequence":"additional","affiliation":[]},{"given":"Stefano","family":"Giagu","sequence":"additional","affiliation":[]},{"given":"Christian","family":"Napoli","sequence":"additional","affiliation":[]},{"given":"Simone","family":"Scardapane","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,8,3]]},"reference":[{"key":"102_CR1","doi-asserted-by":"publisher","first-page":"52,138","DOI":"10.1109\/ACCESS.2018.2870052","volume":"6","author":"A Adadi","year":"2018","unstructured":"Adadi A, Berrada M (2018) Peeking inside the black-box: a survey on explainable artificial intelligence (XAI). IEEE Access 6:52,138-52,160","journal-title":"IEEE Access"},{"key":"102_CR2","first-page":"1","volume":"5","author":"G Agarwal","year":"2021","unstructured":"Agarwal G, Hay L, Iashvili I et al (2021) Explainable AI for ML jet taggers using expert variables and layer-wise relevance propagation. J High Energy Phys 5:1\u201336","journal-title":"J High Energy Phys"},{"key":"102_CR3","doi-asserted-by":"crossref","unstructured":"Ahmeto\u011flu A, \u0130rsoy O, Alpayd\u0131n E (2018) Convolutional soft decision trees. In: Artificial Neural Networks and Machine Learning\u2013ICANN 2018: 27th International Conference on Artificial Neural Networks, Springer, pp 134\u2013141","DOI":"10.1007\/978-3-030-01418-6_14"},{"issue":"93","key":"102_CR4","first-page":"1","volume":"20","author":"M Alber","year":"2019","unstructured":"Alber M, Lapuschkin S, Seegerer P et al (2019) Investigate neural networks! J Mach Learn Res 20(93):1\u20138","journal-title":"J Mach Learn Res"},{"issue":"4","key":"102_CR5","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1016\/0925-2312(93)90006-O","volume":"5","author":"S Amari","year":"1993","unstructured":"Amari S (1993) Backpropagation and stochastic gradient descent method. Neurocomputing 5(4):185\u2013196","journal-title":"Neurocomputing"},{"issue":"7","key":"102_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0130140","volume":"10","author":"S Bach","year":"2015","unstructured":"Bach S, Binder A, Montavon G et al (2015) On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7):1\u201346","journal-title":"PLoS ONE"},{"key":"102_CR7","unstructured":"Balestriero R (2017) Neural decision trees. arXiv preprint http:\/\/arxiv.org\/abs\/1702.07360"},{"key":"102_CR8","doi-asserted-by":"crossref","unstructured":"Binder A, Bach S, Montavon G, et\u00a0al (2016) Layer-wise relevance propagation for deep neural network architectures. In: Information science and applications (ICISA) 2016, Springer, pp 913\u2013922","DOI":"10.1007\/978-981-10-0557-2_87"},{"issue":"7","key":"102_CR9","doi-asserted-by":"publisher","first-page":"871","DOI":"10.3390\/electronics10070871","volume":"10","author":"M Bistron","year":"2021","unstructured":"Bistron M, Piotrowski Z (2021) Artificial intelligence applications in military systems and their influence on sense of security of citizens. Electronics 10(7):871","journal-title":"Electronics"},{"issue":"3","key":"102_CR10","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.106.035014","volume":"106","author":"L Bradshaw","year":"2022","unstructured":"Bradshaw L, Chang S, Ostdiek B (2022) Creating simple, interpretable anomaly detectors for new physics in jet substructure. Phys Rev D 106(3):035014","journal-title":"Phys Rev D"},{"key":"102_CR11","first-page":"1","volume":"7","author":"A Chakraborty","year":"2019","unstructured":"Chakraborty A, Lim SH, Nojiri MM (2019) Interpretable deep learning for two-prong jet classification with jet spectra. J High Energy Phys 7:1\u201336","journal-title":"J High Energy Phys"},{"issue":"08","key":"102_CR12","first-page":"S08,003","volume":"3","author":"TA Collaboration","year":"2008","unstructured":"Collaboration TA (2008) The ATLAS experiment at the CERN large hadron collider. J Instrum 3(08):S08,003-S08,003","journal-title":"J Instrum"},{"key":"102_CR13","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevD.103.036020","author":"T Faucett","year":"2021","unstructured":"Faucett T, Thaler J, Whiteson D (2021) Mapping machine-learned physics into a human-readable space. Phys Rev D. https:\/\/doi.org\/10.1103\/PhysRevD.103.036020","journal-title":"Phys Rev D"},{"issue":"11","key":"102_CR14","doi-asserted-by":"publisher","first-page":"969","DOI":"10.1140\/epjc\/s10052-021-09770-w","volume":"81","author":"S Francescato","year":"2021","unstructured":"Francescato S, Giagu S, Riti F et al (2021) Model compression and simplification pipelines for fast deep neural network inference in FPGAS in hep. Eur Phys J C 81(11):969","journal-title":"Eur Phys J C"},{"key":"102_CR15","unstructured":"Frosst N, Hinton G (2017) Distilling a neural network into a soft decision tree. In: CEX Workshop, 16th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017)"},{"key":"102_CR16","doi-asserted-by":"publisher","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","volume":"129","author":"J Gou","year":"2021","unstructured":"Gou J, Yu B, Maybank SJ et al (2021) Knowledge distillation: a survey. Int J Comput Vision 129:1789\u20131819","journal-title":"Int J Comput Vision"},{"key":"102_CR17","unstructured":"Irsoy O, Y\u0131ld\u0131z OT, Alpayd\u0131n E (2012) Soft decision trees. In: 21st International Conference on Pattern Recognition (ICPR2012), IEEE, pp 1819\u20131822"},{"issue":"3","key":"102_CR18","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.3390\/app12031353","volume":"12","author":"MR Islam","year":"2022","unstructured":"Islam MR, Ahmed MU, Barua S et al (2022) A systematic review of explainable artificial intelligence in terms of different application domains and tasks. Appl Sci 12(3):1353","journal-title":"Appl Sci"},{"key":"102_CR19","unstructured":"Khot A, Neubauer MS, Roy A (2022) A detailed study of interpretability of deep neural network based top taggers. arXiv preprint http:\/\/arxiv.org\/abs\/2210.04371"},{"key":"102_CR20","doi-asserted-by":"publisher","first-page":"263","DOI":"10.2501\/JAR-2018-035","volume":"58","author":"J Kietzmann","year":"2018","unstructured":"Kietzmann J, Paschen J, Treen E (2018) Artificial intelligence in advertising: How marketers can leverage artificial intelligence along the consumer journey. J Advert Res 58:263\u2013267","journal-title":"J Advert Res"},{"key":"102_CR21","unstructured":"Kingma DP, Ba J (2015) Adam: A method for stochastic optimization. In: 3rd International Conference for Learning Representations (ICLR)"},{"key":"102_CR22","doi-asserted-by":"crossref","unstructured":"Krishna S, Han T, Gu A, et\u00a0al (2022) The disagreement problem in explainable machine learning: a practitioner\u2019s perspective. arXiv preprint http:\/\/arxiv.org\/abs\/2202.01602","DOI":"10.21203\/rs.3.rs-2963888\/v1"},{"key":"102_CR23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1705.07874","author":"SM Lundberg","year":"2017","unstructured":"Lundberg SM, Lee SI (2017) A unified approach to interpreting model predictions. Adv Neural Inf Process Syst. https:\/\/doi.org\/10.48550\/arXiv.1705.07874","journal-title":"Adv Neural Inf Process Syst"},{"issue":"55","key":"102_CR24","first-page":"999","volume":"9","author":"H Luo","year":"2021","unstructured":"Luo H, Cheng F, Yu H et al (2021) SDTR: soft decision tree regressor for tabular data. IEEE Access 9(55):999\u201356011","journal-title":"IEEE Access"},{"key":"102_CR25","unstructured":"Mokhtar F, Kansal R, Diaz D, et\u00a0al (2021) Explaining machine-learned particle-flow reconstruction. In: Machine Learning for Physical Sciences Workshop, NeurIPS 2021"},{"key":"102_CR26","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","volume":"65","author":"G Montavon","year":"2017","unstructured":"Montavon G, Lapuschkin S, Binder A et al (2017) Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit 65:211\u2013222","journal-title":"Pattern Recognit"},{"key":"102_CR27","doi-asserted-by":"crossref","unstructured":"Montavon G, Binder A, Lapuschkin S, et\u00a0al (2019) Layer-wise relevance propagation: an overview. Explainable AI: interpreting, explaining and visualizing deep learning pp 193\u2013209","DOI":"10.1007\/978-3-030-28954-6_10"},{"key":"102_CR28","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"102_CR29","first-page":"19,920","volume":"33","author":"G Pruthi","year":"2020","unstructured":"Pruthi G, Liu F, Kale S et al (2020) Estimating training data influence by tracing gradient descent. Adv Neural Inf Process Syst 33:19,920-19,930","journal-title":"Adv Neural Inf Process Syst"},{"key":"102_CR30","unstructured":"Samek W, Wiegand T, M\u00fcller KR (2017) Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint http:\/\/arxiv.org\/abs\/1708.08296"},{"key":"102_CR31","unstructured":"Smilkov D, Thorat N, Kim B, et\u00a0al (2017) Smoothgrad: removing noise by adding noise. arXiv preprint http:\/\/arxiv.org\/abs\/1706.03825"},{"key":"102_CR32","unstructured":"Sundararajan M, Taly A, Yan Q (2017) Axiomatic attribution for deep networks. In: 2017 International Conference on Machine Learning (ICML), PMLR, pp 3319\u20133328"},{"key":"102_CR33","unstructured":"Svenmarck P, Luotsinen L, Nilsson M, et\u00a0al (2018) Possibilities and challenges for artificial intelligence in military applications. In: NATO Big Data and Artificial Intelligence for Military Decision Making Specialists\u2019 Meeting, pp 1\u201316"},{"key":"102_CR34","unstructured":"Wang Z, Yang J (2017) Diabetic retinopathy detection via deep convolutional networks for discriminative localization and visual explanation. arXiv preprint http:\/\/arxiv.org\/abs\/1703.10757"},{"key":"102_CR35","unstructured":"Yang Y, Morillo IG, Hospedales TM (2018) Deep neural decision trees. In: ICML Workshop on Human Interpretability in Machine Learning (WHI 2018)"},{"issue":"5","key":"102_CR36","doi-asserted-by":"publisher","first-page":"726","DOI":"10.1109\/TETCI.2021.3100641","volume":"5","author":"Y Zhang","year":"2021","unstructured":"Zhang Y, Ti\u0148o P, Leonardis A et al (2021) A survey on neural network interpretability. IEEE Trans Emerg Top Comput Intell 5(5):726\u2013742","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"key":"102_CR37","doi-asserted-by":"crossref","unstructured":"Zhou B, Khosla A, Lapedriza A, et\u00a0al (2016) Learning deep features for discriminative localization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2921\u20132929","DOI":"10.1109\/CVPR.2016.319"}],"container-title":["Computing and Software for Big Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-023-00102-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41781-023-00102-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41781-023-00102-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,3]],"date-time":"2023-08-03T13:20:08Z","timestamp":1691068808000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41781-023-00102-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,3]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["102"],"URL":"https:\/\/doi.org\/10.1007\/s41781-023-00102-z","relation":{},"ISSN":["2510-2036","2510-2044"],"issn-type":[{"type":"print","value":"2510-2036"},{"type":"electronic","value":"2510-2044"}],"subject":[],"published":{"date-parts":[[2023,8,3]]},"assertion":[{"value":"20 January 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 July 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 August 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical Approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent to Participate"}},{"value":"All authors gave final approval for publication and agreed to be held accountable for the work performed herein.","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"8"}}