{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:25:20Z","timestamp":1772252720159,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,1,17]],"date-time":"2022-01-17T00:00:00Z","timestamp":1642377600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["PD\/BDE\/150624\/2020"],"award-info":[{"award-number":["PD\/BDE\/150624\/2020"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["PD\/BDE\/150304\/2019"],"award-info":[{"award-number":["PD\/BDE\/150304\/2019"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BioMedInformatics"],"abstract":"<jats:p>In clinical practice, every decision should be reliable and explained to the stakeholders. The high accuracy of deep learning (DL) models pose a great advantage, but the fact that they function as black-boxes hinders their clinical applications. Hence, explainability methods became important as they provide explanation to DL models. In this study, two datasets with electrocardiogram (ECG) image representations of six heartbeats were built, one given the label of the last heartbeat and the other given the label of the first heartbeat. Each dataset was used to train one neural network. Finally, we applied well-known explainability methods to the resulting networks to explain their classifications. Explainability methods produced attribution maps where pixels intensities are proportional to their importance to the classification task. Then, we developed a metric to quantify the focus of the models in the heartbeat of interest. The classification models achieved testing accuracy scores of around 93.66% and 91.72%. The models focused around the heartbeat of interest, with values of the focus metric ranging between 8.8% and 32.4%. Future work will investigate the importance of regions outside the region of interest, besides the contribution of specific ECG waves to the classification.<\/jats:p>","DOI":"10.3390\/biomedinformatics2010008","type":"journal-article","created":{"date-parts":[[2022,1,17]],"date-time":"2022-01-17T08:20:42Z","timestamp":1642407642000},"page":"124-138","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Quantified Explainability: Convolutional Neural Network Focus Assessment in Arrhythmia Detection"],"prefix":"10.3390","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0237-3412","authenticated-orcid":false,"given":"Rui","family":"Varandas","sequence":"first","affiliation":[{"name":"LIBPhys (Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics), Faculdade de Ci\u00eancias e Tecnologia, Universidade Nova de Lisboa, 2825-149 Caparica, Portugal"},{"name":"PLUX Wireless Biosignals S.A., 1050-059 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8002-0391","authenticated-orcid":false,"given":"Bernardo","family":"Gon\u00e7alves","sequence":"additional","affiliation":[{"name":"Bee2Fire S.A., 2200-062 Abrantes, Portugal"},{"name":"Physics Department, NOVA School of Science and Technology, 2825-149 Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4022-7424","authenticated-orcid":false,"given":"Hugo","family":"Gamboa","sequence":"additional","affiliation":[{"name":"LIBPhys (Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics), Faculdade de Ci\u00eancias e Tecnologia, Universidade Nova de Lisboa, 2825-149 Caparica, Portugal"},{"name":"PLUX Wireless Biosignals S.A., 1050-059 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3823-1184","authenticated-orcid":false,"given":"Pedro","family":"Vieira","sequence":"additional","affiliation":[{"name":"Bee2Fire S.A., 2200-062 Abrantes, Portugal"},{"name":"Physics Department, NOVA School of Science and Technology, 2825-149 Caparica, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1159\/000492428","article-title":"Societal Issues Concerning the Application of Artificial Intelligence in Medicine","volume":"5","author":"Vellido","year":"2019","journal-title":"Kidney Dis."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1016\/j.neucom.2020.08.011","article-title":"Explaining the black-box model: A survey of local interpretation methods for deep neural networks","volume":"419","author":"Liang","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_3","unstructured":"Hamon, R., Junklewitz, H., and Sanchez, I. (2020). Robustness and Explainability of Artificial Intelligence, Publications Office of the European Union."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_5","unstructured":"Molnar, C. (2022, January 10). Interpretable Machine Learning. Available online: https:\/\/christophm.github.io\/interpretable-ml-book\/."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Linardatos, P., Papastefanopoulos, V., and Kotsiantis, S. (2021). Explainable ai: A review of machine learning interpretability methods. Entropy, 23.","DOI":"10.3390\/e23010018"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"101901","DOI":"10.1016\/j.artmed.2020.101901","article-title":"The four dimensions of contestable AI diagnostics\u2014A patient-centric approach to explainable AI","volume":"107","author":"Ploug","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_8","unstructured":"(2021, June 14). Chapter 3\u2014Rights of the Data Subject|General Data Protection Regulation (GDPR). General Data Protection Regulation (GDPR). Available online: https:\/\/gdpr-info.eu\/chapter-3\/."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.1007\/s11845-019-01980-2","article-title":"GDPR: An impediment to research?","volume":"188","author":"Clarke","year":"2019","journal-title":"Ir. J. Med Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.cmpb.2015.12.008","article-title":"ECG-based heartbeat classification for arrhythmia detection: A survey","volume":"127","author":"Luz","year":"2016","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3561","DOI":"10.1016\/j.eswa.2012.12.063","article-title":"ECG arrhythmia classification based on optimum-path forest","volume":"40","author":"Luz","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neucom.2017.02.056","article-title":"ECG beat classification via deterministic learning","volume":"240","author":"Dong","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"012004","DOI":"10.1088\/1742-6596\/913\/1\/012004","article-title":"Deep Learning for ECG Classification","volume":"913","author":"Pyakillya","year":"2017","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rim, B., Sung, N.J., Min, S., and Hong, M. (2020). Deep learning in physiological signal data: A survey. Sensors, 20.","DOI":"10.3390\/s20040969"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1179","DOI":"10.1093\/europace\/euaa377","article-title":"Deep learning and the electrocardiogram: Review of the current state-of-the-art","volume":"23","author":"Somani","year":"2021","journal-title":"Europace"},{"key":"ref_16","first-page":"100033","article-title":"A review on deep learning methods for ECG arrhythmia classification","volume":"7","author":"Ebrahimi","year":"2020","journal-title":"Expert Syst. Appl. X"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.ins.2016.01.082","article-title":"Deep learning approach for active classification of electrocardiogram signals","volume":"345","author":"Rahhal","year":"2016","journal-title":"Inf. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Degirmenci, M., Ozdemir, M.A., Izci, E., and Akan, A. (2021). Arrhythmic Heartbeat Classification Using 2D Convolutional Neural Networks. IRBM.","DOI":"10.21203\/rs.3.rs-44313\/v2"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e386","DOI":"10.7717\/peerj-cs.386","article-title":"From ECG signals to images: A transformation based approach for deep learning","volume":"7","author":"Naz","year":"2021","journal-title":"PeerJ Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"835","DOI":"10.32604\/iasc.2022.019877","article-title":"Arrhythmia and Disease Classification Based on Deep Learning Techniques","volume":"31","author":"Franklin","year":"2021","journal-title":"Intell. Autom. Soft Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"104057","DOI":"10.1016\/j.compbiomed.2020.104057","article-title":"HAN-ECG: An interpretable atrial fibrillation detection model using hierarchical attention networks","volume":"127","author":"Mousavi","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"102059","DOI":"10.1016\/j.artmed.2021.102059","article-title":"CEFEs: A CNN Explainable Framework for ECG Signals","volume":"115","author":"Maweu","year":"2021","journal-title":"Artif. Intell. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.jelectrocard.2021.06.006","article-title":"Detection and classification of arrhythmia using an explainable deep learning model","volume":"67","author":"Jo","year":"2021","journal-title":"J. Electrocardiol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"106281","DOI":"10.1016\/j.cmpb.2021.106281","article-title":"xECGNet: Fine-tuning attention map within convolutional neural network to improve detection and explainability of concurrent cardiac arrhythmias","volume":"208","author":"Yoo","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1109\/51.932724","article-title":"The impact of the MIT-BIH arrhythmia database","volume":"20","author":"Moody","year":"2001","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Singh, A., Sengupta, S., and Lakshminarayanan, V. (2020). Explainable deep learning models in medical image analysis. J. Imaging, 6.","DOI":"10.3390\/jimaging6060052"},{"key":"ref_29","unstructured":"Simonyan, K., Vedaldi, A., and Zisserman, A. (2014). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_31","unstructured":"Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2015, January 7\u20139). Striving for simplicity: The all convolutional net. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015\u2014Workshop Track Proceedings, San Diego, CA, USA."},{"key":"ref_32","first-page":"818","article-title":"Visualizing and Understanding Convolutional Networks","volume":"Volume 12","author":"Zeiler","year":"2014","journal-title":"Analytical Chemistry Research"},{"key":"ref_33","first-page":"331","article-title":"Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization","volume":"17","author":"Selvaraju","year":"2016","journal-title":"Rev. Hosp. Clin."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Varandas, R., and Gon\u00e7alves, B. (2022). Quantified Explainability: Convolutional Neural Network Focus Assessment in Arrhythmia Detection. Res. Sq.","DOI":"10.21203\/rs.3.rs-666509\/v1"}],"container-title":["BioMedInformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-7426\/2\/1\/8\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:02:25Z","timestamp":1760133745000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-7426\/2\/1\/8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,17]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["biomedinformatics2010008"],"URL":"https:\/\/doi.org\/10.3390\/biomedinformatics2010008","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-666509\/v1","asserted-by":"object"}]},"ISSN":["2673-7426"],"issn-type":[{"value":"2673-7426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,17]]}}}