{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,6]],"date-time":"2026-11-06T03:09:36Z","timestamp":1793934576140,"version":"4.8.0"},"reference-count":42,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2025,5,31]],"date-time":"2025-05-31T00:00:00Z","timestamp":1748649600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Artificial intelligence (AI) and deep learning are increasingly applied in cardiovascular imaging. However, the \u201cblack box\u201d nature of these models raises challenges for clinical trust and integration. Explainable Artificial Intelligence (XAI) seeks to address these concerns by providing insights into model decision-making. This systematic review synthesizes current research on the use of XAI methods in radiological cardiovascular imaging. Methods: A systematic literature search was conducted in PubMed, Scopus, and Web of Science to identify peer-reviewed original research articles published between January 2015 and March 2025. Studies were included if they applied XAI techniques\u2014such as Gradient-Weighted Class Activation Mapping (Grad-CAM), Shapley Additive Explanations (SHAPs), Local Interpretable Model-Agnostic Explanations (LIMEs), or saliency maps\u2014to cardiovascular imaging modalities, including cardiac computed tomography (CT), magnetic resonance imaging (MRI), echocardiography and other ultrasound examinations, and chest X-ray (CXR). Studies focusing on nuclear medicine, structured\/tabular data without imaging, or lacking concrete explainability features were excluded. Screening and data extraction followed PRISMA guidelines. Results: A total of 28 studies met the inclusion criteria. Ultrasound examinations (n = 9) and CT (n = 9) were the most common imaging modalities, followed by MRI (n = 6) and chest X-rays (n = 4). Clinical applications included disease classification (e.g., coronary artery disease and valvular heart disease) and the detection of myocardial or congenital abnormalities. Grad-CAM was the most frequently employed XAI method, followed by SHAP. Most studies used saliency-based techniques to generate visual explanations of model predictions. Conclusions: XAI holds considerable promise for improving the transparency and clinical acceptance of deep learning models in cardiovascular imaging. However, the evaluation of XAI methods remains largely qualitative, and standardization is lacking. Future research should focus on the robust, quantitative assessment of explainability, prospective clinical validation, and the development of more advanced XAI techniques beyond saliency-based methods. Strengthening the interpretability of AI models will be crucial to ensuring their safe, ethical, and effective integration into cardiovascular care.<\/jats:p>","DOI":"10.3390\/diagnostics15111399","type":"journal-article","created":{"date-parts":[[2025,6,2]],"date-time":"2025-06-02T03:19:38Z","timestamp":1748834378000},"page":"1399","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Explainable Artificial Intelligence in Radiological Cardiovascular Imaging\u2014A Systematic Review"],"prefix":"10.3390","volume":"15","author":[{"given":"Matteo","family":"Haupt","sequence":"first","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Carl von Ossietzky Universit\u00e4t Oldenburg, 26129 Oldenburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin H.","family":"Maurer","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Carl von Ossietzky Universit\u00e4t Oldenburg, 26129 Oldenburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rohit Philip","family":"Thomas","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Carl von Ossietzky Universit\u00e4t Oldenburg, 26129 Oldenburg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Chong, B., Jayabaskaran, J., Jauhari, S.M., Chan, S.P., Goh, R., Kueh, M.T.W., Li, H., Chin, Y.H., Kong, G., and Anand, V.V. (2024). Global Burden of Cardiovascular Diseases: Projections from 2025 to 2050. Eur. J. Prev. Cardiol.","DOI":"10.1093\/eurjpc\/zwae281"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1007\/s10462-024-10852-w","article-title":"A Review of Evaluation Approaches for Explainable AI with Applications in Cardiology","volume":"57","author":"Salih","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"110786","DOI":"10.1016\/j.ejrad.2023.110786","article-title":"Explainable AI in Medical Imaging: An Overview for Clinical Practitioners\u2014Beyond Saliency-Based XAI Approaches","volume":"162","author":"Borys","year":"2023","journal-title":"Eur. J. Radiol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gala, D., Behl, H., Shah, M., and Makaryus, A.N. (2024). The Role of Artificial Intelligence in Improving Patient Outcomes and Future of Healthcare Delivery in Cardiology: A Narrative Review of the Literature. Healthcare, 12.","DOI":"10.3390\/healthcare12040481"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Linardatos, P., Papastefanopoulos, V., and Kotsiantis, S. (2020). Explainable AI: A Review of Machine Learning Interpretability Methods. Entropy, 23.","DOI":"10.3390\/e23010018"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Szabo, L., Raisi-Estabragh, Z., Salih, A., McCracken, C., Pujadas, E.R., Gkontra, P., Kiss, M., Maurovich-Horvath, P., Vago, H., and Merkely, B. (2022). Clinician\u2019s Guide to Trustworthy and Responsible Artificial Intelligence in Cardiovascular Imaging. Front. Cardiovasc. Med., 9.","DOI":"10.3389\/fcvm.2022.1016032"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1186\/s43055-024-01356-2","article-title":"Explainability, Transparency and Black Box Challenges of AI in Radiology: Impact on Patient Care in Cardiovascular Radiology","volume":"55","author":"Marey","year":"2024","journal-title":"Egypt. J. Radiol. Nucl. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s12559-023-10179-8","article-title":"Interpreting Black-Box Models: A Review on Explainable Artificial Intelligence","volume":"16","author":"Hassija","year":"2024","journal-title":"Cogn. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1016\/j.csbj.2024.08.005","article-title":"Unveiling the Black Box: A Systematic Review of Explainable Artificial Intelligence in Medical Image Analysis","volume":"24","author":"Muhammad","year":"2024","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Salahuddin, Z., Woodruff, H.C., Chatterjee, A., and Lambin, P. (2022). Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methods. Comput. Biol. Med., 140.","DOI":"10.1016\/j.compbiomed.2021.105111"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hassan, S.U., Abdulkadir, S.J., Zahid, M.S.M., and Al-Selwi, S.M. (2024). Local Interpretable Model-Agnostic Explanation Approach for Medical Imaging Analysis: A Systematic Literature Review. Comput. Biol. Med., 185.","DOI":"10.1016\/j.compbiomed.2024.109569"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gerbasi, A., Dagliati, A., Albi, G., Chiesa, M., Andreini, D., Baggiano, A., Mushtaq, S., Pontone, G., Bellazzi, R., and Colombo, G. (2023). CAD-RADS Scoring of Coronary CT Angiography with Multi-Axis Vision Transformer: A Clinically-Inspired Deep Learning Pipeline. Comput. Methods Programs Biomed., 244.","DOI":"10.1016\/j.cmpb.2023.107989"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Penso, M., Moccia, S., Caiani, E.G., Caredda, G., Lampus, M.L., Carerj, M.L., Babbaro, M., Pepi, M., Chiesa, M., and Pontone, G. (2022). A Token-Mixer Architecture for CAD-RADS Classification of Coronary Stenosis on Multiplanar Reconstruction CT Images. Comput. Biol. Med., 153.","DOI":"10.1016\/j.compbiomed.2022.106484"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"7136","DOI":"10.1007\/s00330-022-08901-5","article-title":"Machine Learning-Based Prediction of Insufficient Contrast Enhancement in Coronary Computed Tomography Angiography","volume":"32","author":"Lopes","year":"2022","journal-title":"Eur. Radiol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sakai, Y., Kim, J., Phi, H.Q., Hu, A.C., Balali, P., Guggenberger, K.V., Woo, J.H., Bos, D., Kasner, S.E., and Cucchiara, B.L. (2024). Explainable Machine-Learning Model to Classify Culprit Calcified Carotid Plaque in Embolic Stroke of Undetermined Source. MedRxiv.","DOI":"10.1101\/2024.10.25.24316081"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"101721","DOI":"10.1016\/j.compmedimag.2020.101721","article-title":"Automated Coronary Artery Atherosclerosis Detection and Weakly Supervised Localization on Coronary CT Angiography with a Deep 3-Dimensional Convolutional Neural Network","volume":"83","author":"Candemir","year":"2020","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"5792975","DOI":"10.1155\/2021\/5792975","article-title":"SOSPCNN: Structurally Optimized Stochastic Pooling Convolutional Neural Network for Tetralogy of Fallot Recognition","volume":"2021","author":"Wang","year":"2021","journal-title":"Wirel. Commun. Mob. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Iacono, F.L., Maragna, R., Pontone, G., and Corino, V.D.A. (2023). A Robust Radiomic-Based Machine Learning Approach to Detect Cardiac Amyloidosis Using Cardiac Computed Tomography. Front. Radiol., 3.","DOI":"10.3389\/fradi.2023.1193046"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Huo, Y., Terry, J.G., Wang, J., Nath, V., Bermudez, C., Bao, S., Parvathaneni, P., Carr, J.J., and Landman, B.A. (2019, January 16\u201321). Coronary Calcium Detection Using 3D Attention Identical Dual Deep Network Based on Weakly Supervised Learning. Proceedings of the SPIE 10949, Medical Imaging 2019: Image Processing, San Diego, CA, USA.","DOI":"10.1117\/12.2512541"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Fu, J., Fang, M., Lin, Z., Qiu, J., Yang, M., Tian, J., Dong, D., and Zou, Y. (2024). CT-Based Radiomics: Predicting Early Outcomes after Percutaneous Transluminal Renal Angioplasty in Patients with Severe Atherosclerotic Renal Artery Stenosis. Vis. Comput. Ind. Biomed. Art, 7.","DOI":"10.1186\/s42492-023-00152-5"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Paciorek, A.M., von Schacky, C.E., Foreman, S.C., Gassert, F.G., Gassert, F.T., Kirschke, J.S., Laugwitz, K.-L., Geith, T., Hadamitzky, M., and Nadjiri, J. (2024). Automated Assessment of Cardiac Pathologies on Cardiac MRI Using T1-Mapping and Late Gadolinium Phase Sensitive Inversion Recovery Sequences with Deep Learning. BMC Med. Imaging, 24.","DOI":"10.1186\/s12880-024-01217-4"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Dai, Y., Xue, P., Bao, X., Bai, X., Qiao, S., Gao, Y., Guo, X., Xue, Y., and Dai, Q. (2025). Prediction of Microvascular Obstruction from Angio-Based Microvascular Resistance and Available Clinical Data in Percutaneous Coronary Intervention: An Explainable Machine Learning Model. Sci. Rep., 15.","DOI":"10.1038\/s41598-025-87828-5"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1002\/hbm.25805","article-title":"Multimodal Biological Brain Age Prediction Using Magnetic Resonance Imaging and Angiography with the Identification of Predictive Regions","volume":"43","author":"Mouches","year":"2022","journal-title":"Hum. Brain Mapp."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Cai, L., Zhao, E., Niu, H., Liu, Y., Zhang, T., Liu, D., Zhang, Z., Li, J., Qiao, P., and Lv, H. (2023). A Machine Learning Approach to Predict Cerebral Perfusion Status Based on Internal Carotid Artery Blood Flow. Comput. Biol. Med., 164.","DOI":"10.1016\/j.compbiomed.2023.107264"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1471","DOI":"10.1038\/s41591-024-02971-2","article-title":"Screening and Diagnosis of Cardiovascular Disease Using Artificial Intelligence-Enabled Cardiac Magnetic Resonance Imaging","volume":"30","author":"Wang","year":"2024","journal-title":"Nat. Med."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sufian, A., Alsadder, L., Hamzi, W., Zaman, S., Sagar, A.S.M.S., and Hamzi, B. (2024). Mitigating Algorithmic Bias in AI-Driven Cardiovascular Imaging for Fairer Diagnostics. Diagnostics, 14.","DOI":"10.3390\/diagnostics14232675"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4592","DOI":"10.1093\/eurheartj\/ehad456","article-title":"Severe Aortic Stenosis Detection by Deep Learning Applied to Echocardiography","volume":"44","author":"Holste","year":"2023","journal-title":"Eur. Heart J."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1007\/s11548-021-02542-7","article-title":"Automatic Morphological Classification of Mitral Valve Diseases in Echocardiographic Images Based on Explainable Deep Learning Methods","volume":"17","author":"Vafaeezadeh","year":"2021","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1177\/01617346221099435","article-title":"Cardiac Segmentation Method Based on Domain Knowledge","volume":"44","author":"Wang","year":"2022","journal-title":"Ultrason. Imaging"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2567","DOI":"10.1007\/s11263-024-01996-x","article-title":"Deep Learning Based Prediction of Pulmonary Hypertension in Newborns Using Echocardiograms","volume":"132","author":"Ragnarsdottir","year":"2024","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nurmaini, S., Partan, R.U., Bernolian, N., Sapitri, A.I., Tutuko, B., Rachmatullah, M.N., Darmawahyuni, A., Firdaus, F., and Mose, J.C. (2022). Deep Learning for Improving the Effectiveness of Routine Prenatal Screening for Major Congenital Heart Diseases. J. Clin. Med., 11.","DOI":"10.3390\/jcm11216454"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Sakai, A., Komatsu, M., Komatsu, R., Matsuoka, R., Yasutomi, S., Dozen, A., Shozu, K., Arakaki, T., Machino, H., and Asada, K. (2022). Medical Professional Enhancement Using Explainable Artificial Intelligence in Fetal Cardiac Ultrasound Screening. Biomedicines, 10.","DOI":"10.3390\/biomedicines10030551"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1002\/uog.27577","article-title":"Interaction between Clinicians and Artificial Intelligence to Detect Fetal Atrioventricular Septal Defects on Ultrasound: How Can We Optimize Collaborative Performance?","volume":"64","author":"Day","year":"2024","journal-title":"Ultrasound Obstet. Gynecol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.jcmg.2023.09.011","article-title":"Echocardiography-Based Deep Learning Model to Differentiate Constrictive Pericarditis and Restrictive Cardiomyopathy","volume":"17","author":"Chao","year":"2023","journal-title":"JACC Cardiovasc. Imaging"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lee, H., Eun, Y., Hwang, J.Y., and Eun, L.Y. (2022). Explainable Deep Learning Algorithm for Distinguishing Incomplete Kawasaki Disease by Coronary Artery Lesions on Echocardiographic Imaging. Comput. Methods Programs Biomed., 223.","DOI":"10.1016\/j.cmpb.2022.106970"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2002","DOI":"10.1093\/eurheartj\/ehad782","article-title":"Deep Learning to Detect Left Ventricular Structural Abnormalities in Chest X-Rays","volume":"45","author":"Bhave","year":"2024","journal-title":"Eur. Heart J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"5890","DOI":"10.1007\/s00330-022-08752-0","article-title":"Artificial Intelligence-Based Detection of Atrial Fibrillation from Chest Radiographs","volume":"32","author":"Matsumoto","year":"2022","journal-title":"Eur. Radiol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"e525","DOI":"10.1016\/S2589-7500(23)00107-3","article-title":"Artificial Intelligence-Based Model to Classify Cardiac Functions from Chest Radiographs: A Multi-Institutional, Retrospective Model Development and Validation Study","volume":"5","author":"Ueda","year":"2023","journal-title":"Lancet Digit. Health"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kusunose, K., Hirata, Y., Yamaguchi, N., Kosaka, Y., Tsuji, T., Kotoku, J., and Sata, M. (2022). Deep Learning for Detection of Exercise-Induced Pulmonary Hypertension Using Chest X-Ray Images. Front. Cardiovasc. Med., 9.","DOI":"10.3389\/fcvm.2022.891703"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/s44163-021-00015-z","article-title":"A Novel Interaction-Based Methodology towards Explainable AI with Better Understanding of Pneumonia Chest X-Ray Images","volume":"1","author":"Lo","year":"2021","journal-title":"Discov. Artif. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1038\/s41746-024-01080-1","article-title":"The Clinician-AI Interface: Intended Use and Explainability in FDA-Cleared AI Devices for Medical Image Interpretation","volume":"7","author":"McNamara","year":"2024","journal-title":"NPJ Digit. Med."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Panigutti, C., Hamon, R., Hupont, I., Llorca, D.F., Yela, D.F., Junklewitz, H., Scalzo, S., Mazzini, G., Sanchez, I., and Garrido, J.S. (2023). The Role of Explainable AI in the Context of the AI Act. ACM International Conference Proceeding Series, Proceedings of the FAccT \u203223: 2023 ACM Conference on Fairness, Accountability, and Transparency, Chicago, IL, USA, 12\u201315 June 2023, Association for Computing Machinery.","DOI":"10.1145\/3593013.3594069"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/11\/1399\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:44:34Z","timestamp":1760031874000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/11\/1399"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,31]]},"references-count":42,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["diagnostics15111399"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15111399","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,31]]}}}