{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T13:01:38Z","timestamp":1781614898640,"version":"3.54.5"},"reference-count":62,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T00:00:00Z","timestamp":1738800000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>This study introduces the Pixel-Level Interpretability (PLI) model, a novel framework designed to address critical limitations in medical imaging diagnostics by enhancing model transparency and diagnostic accuracy. The primary objective is to evaluate PLI\u2019s performance against Gradient-Weighted Class Activation Mapping (Grad-CAM) and achieve fine-grained interpretability and improved localization precision. The methodology leverages the VGG19 convolutional neural network architecture and utilizes three publicly available COVID-19 chest radiograph datasets, consisting of over 1000 labeled images, which were preprocessed through resizing, normalization, and augmentation to ensure robustness and generalizability. The experiments focused on key performance metrics, including interpretability, structural similarity (SSIM), diagnostic precision, mean squared error (MSE), and computational efficiency. The results demonstrate that PLI significantly outperforms Grad-CAM in all measured dimensions. PLI produced detailed pixel-level heatmaps with higher SSIM scores, reduced MSE, and faster inference times, showcasing its ability to provide granular insights into localized diagnostic features while maintaining computational efficiency. In contrast, Grad-CAM\u2019s explanations often lack the granularity required for clinical reliability. By integrating fuzzy logic to enhance visual and numerical explanations, PLI can deliver interpretable outputs that align with clinical expectations, enabling practitioners to make informed decisions with higher confidence. This work establishes PLI as a robust tool for bridging gaps in AI model transparency and clinical usability. By addressing the challenges of interpretability and accuracy simultaneously, PLI contributes to advancing the integration of AI in healthcare and sets a foundation for broader applications in other high-stake domains.<\/jats:p>","DOI":"10.3390\/make7010012","type":"journal-article","created":{"date-parts":[[2025,2,6]],"date-time":"2025-02-06T08:57:46Z","timestamp":1738832266000},"page":"12","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":84,"title":["Advancing AI Interpretability in Medical Imaging: A Comparative Analysis of Pixel-Level Interpretability and Grad-CAM Models"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7117-6271","authenticated-orcid":false,"given":"Mohammad","family":"Ennab","sequence":"first","affiliation":[{"name":"Department of Computer Sciences and Mathematics, University of Qu\u00e9bec at Chicoutimi, 555 Bd de l\u2019Universit\u00e9, Chicoutimi, QC G7H 2B1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hamid","family":"Mcheick","sequence":"additional","affiliation":[{"name":"Department of Computer Sciences and Mathematics, University of Qu\u00e9bec at Chicoutimi, 555 Bd de l\u2019Universit\u00e9, Chicoutimi, QC G7H 2B1, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,2,6]]},"reference":[{"key":"ref_1","first-page":"14","article-title":"Survey of COVID-19 Prediction Models and Their Limitations","volume":"11","author":"Ennab","year":"2022","journal-title":"Inf. Syst."},{"key":"ref_2","unstructured":"Doshi-Velez, F., and Kim, B. (2017). Towards A Rigorous Science of Interpretable Machine Learning. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ennab, M., and Mcheick, H. (2022). Designing an Interpretability-Based Model to Explain the Artificial Intelligence Algorithms in Healthcare. Diagnostics, 12.","DOI":"10.3390\/diagnostics12071557"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ennab, M., and Mcheick, H. (2024). Enhancing Interpretability and Accuracy of AI Models in Healthcare: A Comprehensive Review on Challenges and Future Directions. Front. Robot. AI, 11.","DOI":"10.3389\/frobt.2024.1444763"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-cam: Visual explanations from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"100535","DOI":"10.1016\/j.smhl.2024.100535","article-title":"A novel convolutional interpretability model for pixel-level interpretation of medical image classification through fusion of machine learning and fuzzy logic","volume":"35","author":"Ennab","year":"2025","journal-title":"Smart Health"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ahmad, M.A., Eckert, C., and Teredesai, A. (September, January 29). Interpretable machine learning in healthcare. Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, Washington, DC, USA.","DOI":"10.1145\/3233547.3233667"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy Should I Trust You?\u201d: Explaining the Predictions of Any Classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_9","unstructured":"Lundberg, S.M., and Lee, S.-I. (2017). Consistent feature attribution for tree ensembles. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2018, January 2\u20137). Anchors: High-Precision Model-Agnostic Explanations. Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"ref_11","unstructured":"Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (2018). Model Agnostic Supervised Local Explanations. Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_12","unstructured":"Schutte, K., Moindrot, O., H\u00e9rent, P., Schiratti, J.-B., and J\u00e9gou, S. (2021). Using stylegan for visual interpretability of deep learning models on medical images. arXiv."},{"key":"ref_13","unstructured":"Dhore, V., Bhat, A., Nerlekar, V., Chavhan, K., and Umare, A. (2024). Enhancing Explainable AI: A Hybrid Approach Combining GradCAM and LRP for CNN Interpretability. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"102684","DOI":"10.1016\/j.media.2022.102684","article-title":"Guidelines and evaluation of clinical explainable AI in medical image analysis","volume":"84","author":"Jin","year":"2023","journal-title":"Med. Image Anal."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.ejmp.2021.03.009","article-title":"Artificial intelligence: Deep learning in oncological radiomics and challenges of interpretability and data harmonization","volume":"83","author":"Papadimitroulas","year":"2021","journal-title":"Phys. Medica"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Mahesh, T.R., Vinoth Kumar, V., and Guluwadi, S. (2024). Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50. BMC Med. Imaging, 24.","DOI":"10.1186\/s12880-024-01292-7"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Qiu, Z., Rivaz, H., and Xiao, Y. (2023, January 8). Is visual explanation with Grad-CAM more reliable for deeper neural networks? A case study with automatic pneumothorax diagnosis. Proceedings of the International Workshop on Machine Learning in Medical Imaging, Vancouver, BC, Canada.","DOI":"10.1007\/978-3-031-45676-3_23"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3114","DOI":"10.1016\/j.patcog.2012.02.009","article-title":"A saliency map based on sampling an image into random rectangular regions of interest","volume":"45","author":"Vikram","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mangla, P., Singh, V., and Balasubramanian, V.N. (2020, January 14\u201318). On saliency maps and adversarial robustness. Proceedings of the Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2020, Ghent, Belgium. Proceedings, Part II.","DOI":"10.1007\/978-3-030-67661-2_17"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Vinogradova, K., Dibrov, A., and Myers, G. (2020, January 7\u201312). Towards interpretable semantic segmentation via gradient-weighted class activation mapping (student abstract). Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i10.7244"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Devasia, J., Goswami, H., Lakshminarayanan, S., Rajaram, M., and Adithan, S. (2023). Deep learning classification of active tuberculosis lung zones wise manifestations using chest X-rays: A multi label approach. Sci. Rep., 13.","DOI":"10.1038\/s41598-023-28079-0"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"61417","DOI":"10.1109\/ACCESS.2021.3073465","article-title":"Exclusive feature constrained class activation mapping for better visual explanation","volume":"9","author":"Wang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: Visual explanations from deep networks via gradient-based localization","volume":"128","author":"Selvaraju","year":"2020","journal-title":"Int. J. Comput. Vis."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gerbasi, A., Clementi, G., Corsi, F., Albasini, S., Malovini, A., Quaglini, S., and Bellazzi, R. (2023). DeepMiCa: Automatic segmentation and classification of breast MIcroCAlcifications from mammograms. Comput. Methods Programs Biomed., 235.","DOI":"10.1016\/j.cmpb.2023.107483"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.-R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0130140"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Montavon, G., Binder, A., Lapuschkin, S., Samek, W., and M\u00fcller, K.-R. (2019). Layer-wise relevance propagation: An overview. Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, Springer.","DOI":"10.1007\/978-3-030-28954-6_10"},{"key":"ref_27","unstructured":"Singhi, S., and Kumari, A. (2024). Strengthening Interpretability: An Investigative Study of Integrated Gradient Methods. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Jha, A., Aicher, J.K., Gazzara, M.R., Singh, D., and Barash, Y. (2020). Enhanced integrated gradients: Improving interpretability of deep learning models using splicing codes as a case study. Genome Biol., 21.","DOI":"10.1186\/s13059-020-02055-7"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhou, L., Ma, C., Shi, X., Zhang, D., Li, W., and Wu, L. (2021, January 18\u201322). Salience-cam: Visual explanations from convolutional neural networks via salience score. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Virtual.","DOI":"10.1109\/IJCNN52387.2021.9534419"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sugeno, A., Ishikawa, Y., Ohshima, T., and Muramatsu, R. (2021). Simple methods for the lesion detection and severity grading of diabetic retinopathy by image processing and transfer learning. Comput. Biol. Med., 137.","DOI":"10.1016\/j.compbiomed.2021.104795"},{"key":"ref_31","unstructured":"Zhou, L., Shi, X., Ma, C., and Wang, Z. (2024). Axiomatization of Gradient Smoothing in Neural Networks. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Valois, P.H., Niinuma, K., and Fukui, K. (2024, January 3\u20138). Occlusion Sensitivity Analysis with Augmentation Subspace Perturbation in Deep Feature Space. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV57701.2024.00476"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"025015","DOI":"10.1088\/2632-2153\/ac7a03","article-title":"Explainability for deep learning in mammography image quality assessment","volume":"3","author":"Amanova","year":"2022","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhang, H., Manocha, D., Hudson, T., and Hoff III, K.E. (1997, January 3\u20138). Visibility culling using hierarchical occlusion maps. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques, Los Angeles, CA, USA.","DOI":"10.1145\/258734.258781"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.ejmp.2021.02.006","article-title":"AI applications to medical images: From machine learning to deep learning","volume":"83","author":"Castiglioni","year":"2021","journal-title":"Phys. Medica"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1016\/S0019-9958(65)90241-X","article-title":"Fuzzy sets","volume":"8","author":"Zadeh","year":"1965","journal-title":"Inf. Control"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"109370","DOI":"10.1016\/j.compeleceng.2024.109370","article-title":"A review of Explainable Artificial Intelligence in healthcare","volume":"118","author":"Sadeghi","year":"2024","journal-title":"Comput. Electr. Eng."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1016\/S0005-1098(99)00079-5","article-title":"Fault isolation filter design for linear stochastic systems","volume":"35","author":"Keller","year":"1999","journal-title":"Automatica"},{"key":"ref_39","unstructured":"Ou, L. (2024). Intuitive Human-Understandable AI Models based on Fuzzy Neural Network. [Ph.D. Thesis, University of Technology Sydney]."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"B\u00f6lat, K., and Kumbasar, T. (2020, January 19\u201324). Interpreting Variational Autoencoders with Fuzzy Logic: A step towards interpretable deep learning based fuzzy classifiers. Proceedings of the 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Glasgow, UK.","DOI":"10.1109\/FUZZ48607.2020.9177631"},{"key":"ref_41","unstructured":"Lan, X., Hu, Q., Chen, Q., Xue, J., and Cheng, J. (2021). Hih: Towards more accurate face alignment via heatmap in heatmap. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"66556","DOI":"10.1109\/ACCESS.2024.3398203","article-title":"Unraveling the Black Box: A Review of Explainable Deep Learning Healthcare Techniques","volume":"12","author":"Murad","year":"2024","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"108436","DOI":"10.1016\/j.compchemeng.2023.108436","article-title":"Data-driven distributionally robust optimization for long-term contract vs. spot allocation decisions: Application to electricity markets","volume":"179","author":"Papageorgiou","year":"2023","journal-title":"Comput. Chem. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1186\/s13049-021-00953-8","article-title":"Deep-learning model for screening sepsis using electrocardiography","volume":"29","author":"Kwon","year":"2021","journal-title":"Scand. J. Trauma Resusc. Emerg. Med."},{"key":"ref_45","unstructured":"Mostafavi, S.M. (2021). COVID19-CT-Dataset: An Open-Access Chest CT Image Repository of 1000+ Patients with Confirmed COVID-19 Diagnosis, Harvard Dataverse."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.patrec.2020.12.010","article-title":"Customized VGG19 architecture for pneumonia detection in chest X-rays","volume":"143","author":"Dey","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Huff, D.T., Weisman, A.J., and Jeraj, R. (2021). Interpretation and visualization techniques for deep learning models in medical imaging. Phys. Med. Biol., 66.","DOI":"10.1088\/1361-6560\/abcd17"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1038\/s42256-020-0180-7","article-title":"An interpretable mortality prediction model for COVID-19 patients","volume":"2","author":"Yan","year":"2020","journal-title":"Nat. Mach. Intell."},{"key":"ref_49","unstructured":"Thiagarajan, J.J., Sattigeri, P., Rajan, D., and Venkatesh, B. (2020). Calibrating healthcare ai: Towards reliable and interpretable deep predictive models. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Han, T., Tu, W.-W., and Li, Y.-F. (2021, January 2\u20139). Explanation consistency training: Facilitating consistency-based semi-supervised learning with interpretability. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual.","DOI":"10.1609\/aaai.v35i9.16934"},{"key":"ref_51","unstructured":"Hedstr\u00f6m, A. (2020). Explainable Artificial Intelligence: How to Evaluate Explanations of Deep Neural Network Predictions Using the Continuity Test. [Master\u2019s Thesis, KTH Royal Institute of Technology]. Available online: http:\/\/urn.kb.se\/resolve?urn=urn:nbn:se:kth:diva-281279."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1093\/jrsssa\/qnad032","article-title":"Interpretable sensitivity analysis for balancing weights","volume":"186","author":"Soriano","year":"2023","journal-title":"J. R. Stat. Soc. Ser. A Stat. Soc."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"101286","DOI":"10.1016\/j.imu.2023.101286","article-title":"Application of explainable artificial intelligence in medical health: A systematic review of interpretability methods","volume":"40","author":"Band","year":"2023","journal-title":"Inform. Med. Unlocked"},{"key":"ref_54","unstructured":"Cian, D., van Gemert, J., and Lengyel, A. (2020). Evaluating the performance of the LIME and Grad-CAM explanation methods on a LEGO multi-label image classification task. arXiv."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"127","DOI":"10.3905\/jfds.2020.1.047","article-title":"The best way to select features? comparing mda, lime, and shap","volume":"3","author":"Man","year":"2021","journal-title":"J. Financ. Data Sci. Winter"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"107829","DOI":"10.1016\/j.engappai.2023.107829","article-title":"Global and local interpretability techniques of supervised machine learning black box models for numerical medical data","volume":"131","author":"Hakkoum","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_57","unstructured":"Kumar, I.E., Venkatasubramanian, S., Scheidegger, C., and Friedler, S. (2020, January 13\u201318). Problems with Shapley-value-based explanations as feature importance measures. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"4752","DOI":"10.1109\/ACCESS.2023.3236315","article-title":"Verification of interpretability of phase-resolved partial discharge using a CNN with SHAP","volume":"11","author":"Kitani","year":"2023","journal-title":"IEEE Access"},{"key":"ref_59","unstructured":"Justin, T. (2024, May 12). [Interpretable Machine Learning (5\u20138)] Scope Rule (Anchor) [WWW Document]. Tistory. 2020. Available online: https:\/\/eair.tistory.com\/27."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"85","DOI":"10.3109\/10837450.2011.627869","article-title":"Real-time assessment of critical quality attributes of a continuous granulation process","volume":"18","author":"Fonteyne","year":"2013","journal-title":"Pharm. Dev. Technol."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"107861","DOI":"10.1016\/j.envint.2023.107861","article-title":"Contributions of various driving factors to air pollution events: Interpretability analysis from Machine learning perspective","volume":"173","author":"Li","year":"2023","journal-title":"Environ. Int."},{"key":"ref_62","unstructured":"Murindanyi, S., Nakatumba-Nabende, J., Sanya, R., Nakibuule, R., and Katumba, A. (2024). Enhanced Infield Agriculture with Interpretable Machine Learning Approaches for Crop Classification. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/1\/12\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:28:09Z","timestamp":1760027289000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/1\/12"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,6]]},"references-count":62,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["make7010012"],"URL":"https:\/\/doi.org\/10.3390\/make7010012","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,6]]}}}