{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T06:31:49Z","timestamp":1784615509472,"version":"3.55.0"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T00:00:00Z","timestamp":1732492800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Modern artificial intelligence (AI) solutions often face challenges due to the \u201cblack box\u201d nature of deep learning (DL) models, which limits their transparency and trustworthiness in critical medical applications. In this study, we propose and evaluate a scalable approach based on a transition matrix to enhance the interpretability of DL models in medical signal and image processing by translating complex model decisions into user-friendly and justifiable features for healthcare professionals. The criteria for choosing interpretable features were clearly defined, incorporating clinical guidelines and expert rules to align model outputs with established medical standards. The proposed approach was tested on two medical datasets: electrocardiography (ECG) for arrhythmia detection and magnetic resonance imaging (MRI) for heart disease classification. The performance of the DL models was compared with expert annotations using Cohen\u2019s Kappa coefficient to assess agreement, achieving coefficients of 0.89 for the ECG dataset and 0.80 for the MRI dataset. These results demonstrate strong agreement, underscoring the reliability of the approach in providing accurate, understandable, and justifiable explanations of DL model decisions. The scalability of the approach suggests its potential applicability across various medical domains, enhancing the generalizability and utility of DL models in healthcare while addressing practical challenges and ethical considerations.<\/jats:p>","DOI":"10.3389\/frai.2024.1482141","type":"journal-article","created":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T06:25:20Z","timestamp":1732515920000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Toward explainable deep learning in healthcare through transition matrix and user-friendly features"],"prefix":"10.3389","volume":"7","author":[{"given":"Oleksander","family":"Barmak","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Iurii","family":"Krak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergiy","family":"Yakovlev","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eduard","family":"Manziuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pavlo","family":"Radiuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vladislav","family":"Kuznetsov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,11,25]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"3029","DOI":"10.1109\/EMBC46164.2021.9630169","article-title":"An interpretable object detection-based model for the diagnosis of neonatal lung diseases using ultrasound images","volume":"2021","author":"Bassiouny","year":"2021","journal-title":"Annu. Int. Conf. IEEE Eng. Med. Biol. Soc."},{"key":"ref2","doi-asserted-by":"publisher","first-page":"2514","DOI":"10.1109\/TMI.2018.2837502","article-title":"Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved?","volume":"37","author":"Bernard","year":"2018","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1186\/s12911-022-01817-6","article-title":"Explainable machine learning to predict long-term mortality in critically ill ventilated patients: a retrospective study in Central Taiwan","volume":"22","author":"Chan","year":"2022","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref4","doi-asserted-by":"publisher","first-page":"e1391","DOI":"10.1002\/widm.1391","article-title":"A historical perspective of explainable artificial intelligence","volume":"11","author":"Confalonieri","year":"2020","journal-title":"WIREs Data Mining Knowledge Discov."},{"key":"ref5","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1007\/978-981-10-6349-7_3","article-title":"Completions of operator matrices and generalized inverses","volume-title":"Algebraic Properties of Generalized Inverses","author":"Cvetkovi\u0107 Ili\u0107","year":"2017"},{"key":"ref6","year":"2016"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/s12559-023-10179-8","article-title":"Interpreting black-box models: a review on explainable artificial intelligence","volume":"16","author":"Hassija","year":"2024","journal-title":"Cogn. Comput."},{"key":"ref8","author":"Hinton","year":"2002"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"2","DOI":"10.2307\/2687269","article-title":"A singularly valuable decomposition: the SVD of a matrix","volume":"27","author":"Kalman","year":"2002","journal-title":"Coll. Math. J."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"3713","DOI":"10.1007\/s11042-022-13428-4","article-title":"Natural language processing: state of the art, current trends and challenges","volume":"82","author":"Khurana","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref11","doi-asserted-by":"publisher","first-page":"122343","DOI":"10.1016\/j.techfore.2023.122343","article-title":"How should the results of artificial intelligence be explained to users?\u2014research on consumer preferences in user-centered explainable artificial intelligence","volume":"188","author":"Kim","year":"2023","journal-title":"Technol. Forecast. Soc. Chang."},{"key":"ref12","author":"Kovalchuk","year":"2024"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"282","DOI":"10.1007\/978-3-030-33585-4_28","article-title":"Data classification based on the features reduction and piecewise linear separation","volume":"1072","author":"Krak","year":"2020","journal-title":"Adv. Intellig. Syst. Comput."},{"key":"ref14","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1007\/978-3-031-16203-9_20","article-title":"Analysis of deep learning methods in adaptation to the small data problem solving","volume-title":"Lecture Notes in Data Engineering, Computational Intelligence, and Decision Making","author":"Krak","year":"2023"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"102301","DOI":"10.1016\/j.inffus.2024.102301","article-title":"Explainable artificial intelligence (XAI) 2.0: a manifesto of open challenges and interdisciplinary research directions","volume":"106","author":"Longo","year":"2024","journal-title":"Inform. Fusion"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1186\/s12911-023-02257-6","article-title":"Building a trustworthy AI differential diagnosis application for Crohn\u2019s disease and intestinal tuberculosis","volume":"23","author":"Lu","year":"2023","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"1689","DOI":"10.3758\/s13428-020-01516-y","article-title":"NeuroKit2: a Python toolbox for neurophysiological signal processing","volume":"53","author":"Makowski","year":"2021","journal-title":"Behav. Res. Methods"},{"key":"ref18","author":"Manziuk","year":"2021"},{"key":"ref19","year":"2013"},{"key":"ref20","author":"Moody","year":"2005"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"73","DOI":"10.3390\/bdcc8070073","article-title":"Trustworthy AI guidelines in biomedical decision-making applications: a scoping review","volume":"8","author":"Mora-Cantallops","year":"2024","journal-title":"Big Data Cogn. Comput."},{"key":"ref22","doi-asserted-by":"publisher","first-page":"1219586","DOI":"10.3389\/fcvm.2023.1219586","article-title":"Improvement of a prediction model for heart failure survival through explainable artificial intelligence","volume":"10","author":"Moreno-S\u00e1nchez","year":"2023","journal-title":"Front. Cardiovasc. Med."},{"key":"ref23","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1007\/978-3-031-24628-9_41","article-title":"Explainable artificial intelligence (XAI): motivation, terminology, and taxonomy","volume-title":"Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook","author":"Notovich","year":"2023"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1461","DOI":"10.1007\/s11229-020-02806-w","article-title":"Humanistic interpretation and machine learning","volume":"199","author":"P\u00e4\u00e4kk\u00f6nen","year":"2021","journal-title":"Synthese"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1080\/14786440109462720","article-title":"LIII. On lines and planes of closest fit to systems of points in space","volume":"2","author":"Pearson","year":"1901","journal-title":"London Edinburgh Dublin Philos. Magaz. J. Sci."},{"key":"ref26","doi-asserted-by":"crossref","first-page":"8312","DOI":"10.6028\/NIST.IR.8312","volume-title":"Four Principles of Explainable Artificial Intelligence","author":"Phillips","year":"2021"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"2663","DOI":"10.3390\/electronics12122663","article-title":"Explainable feature extraction and prediction framework for 3D image recognition applied to pneumonia detection","volume":"12","author":"Pintelas","year":"2023","journal-title":"Electronics"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"93","DOI":"10.2174\/1875036202114010093","article-title":"An approach to early diagnosis of pneumonia on individual radiographs based on the CNN information technology","volume":"14","author":"Radiuk","year":"2021","journal-title":"Open Bioinform. J."},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1024","DOI":"10.3390\/math12071024","article-title":"Explainable deep learning: a visual analytics approach with transition matrices","volume":"12","author":"Radiuk","year":"2024","journal-title":"Mathematics"},{"key":"ref30","author":"R\u00e4uker","year":"2023"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"105111","DOI":"10.1016\/j.compbiomed.2021.105111","article-title":"Transparency of deep neural networks for medical image analysis: a review of interpretability methods","volume":"140","author":"Salahuddin","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref32","first-page":"77","author":"Slobodzian","year":"2023"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"103472","DOI":"10.1016\/j.compind.2021.103472","article-title":"The quiet revolution in machine vision\u2014a state-of-the-art survey paper, including historical review, perspectives, and future directions","volume":"130","author":"Smith","year":"2021","journal-title":"Comput. Ind."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"2885","DOI":"10.3390\/app13052885","article-title":"Efficient data preprocessing with ensemble machine learning technique for the early detection of chronic kidney disease","volume":"13","author":"Venkatesan","year":"2023","journal-title":"Appl. Sci."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1111\/jopp.12262","article-title":"The right to explanation","volume":"30","author":"Vredenburgh","year":"2022","journal-title":"J Polit Philos"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1108\/IMDS-07-2021-0419","article-title":"Artificial intelligence in safety-critical systems: a systematic review","volume":"122","author":"Wang","year":"2021","journal-title":"Ind. Manag. Data Syst."},{"key":"ref37","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1007\/s40171-023-00357-w","article-title":"The viability of supply chains with interpretable learning systems: the case of COVID-19 vaccine deliveries","volume":"24","author":"Zaoui","year":"2023","journal-title":"Glob. J. Flex. Syst. Manag."}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1482141\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,25]],"date-time":"2024-11-25T06:25:23Z","timestamp":1732515923000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2024.1482141\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,25]]},"references-count":37,"alternative-id":["10.3389\/frai.2024.1482141"],"URL":"https:\/\/doi.org\/10.3389\/frai.2024.1482141","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,25]]},"article-number":"1482141"}}