{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T23:42:12Z","timestamp":1773704532489,"version":"3.50.1"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T00:00:00Z","timestamp":1730851200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Evaluate popular explanation methods using heatmap visualizations to explain the predictions of deep neural networks for electrocardiogram (ECG) analysis and provide recommendations for selection of explanations methods.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>A residual deep neural network was trained on ECGs to predict intervals and amplitudes. Nine commonly used explanation methods (Saliency, Deconvolution, Guided backpropagation, Gradient SHAP, SmoothGrad, Input \u00d7 gradient, DeepLIFT, Integrated gradients, GradCAM) were qualitatively evaluated by medical experts and objectively evaluated using a perturbation-based method.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>No single explanation method consistently outperformed the other methods, but some methods were clearly inferior. We found considerable disagreement between the human expert evaluation and the objective evaluation by perturbation.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The best explanation method depended on the ECG measure. To ensure that future explanations of deep neural networks for medical data analyses are useful to medical experts, data scientists developing new explanation methods should collaborate tightly with domain experts. Because there is no explanation method that performs best in all use cases, several methods should be applied.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>Several explanation methods should be used to determine the most suitable approach.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae280","type":"journal-article","created":{"date-parts":[[2024,11,6]],"date-time":"2024-11-06T20:10:53Z","timestamp":1730923853000},"page":"79-88","source":"Crossref","is-referenced-by-count":3,"title":["Evaluating gradient-based explanation methods for neural network ECG analysis using heatmaps"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1038-7080","authenticated-orcid":false,"given":"Andrea Marheim","family":"Stor\u00e5s","sequence":"first","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]},{"name":"Department of Computer Science, Oslo Metropolitan University , 0167 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steffen","family":"M\u00e6land","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences , 5063 Bergen,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonas L","family":"Isaksen","sequence":"additional","affiliation":[{"name":"Laboratory of Experimental Cardiology, University of Copenhagen , 2200 Copenhagen ,","place":["Denmark"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3332-1201","authenticated-orcid":false,"given":"Steven Alexander","family":"Hicks","sequence":"additional","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vajira","family":"Thambawita","sequence":"additional","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claus","family":"Graff","sequence":"additional","affiliation":[{"name":"Department of Health Science and Technology, Aalborg University , 9260 Aalborg,","place":["Denmark"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hugo Lewi","family":"Hammer","sequence":"additional","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]},{"name":"Department of Computer Science, Oslo Metropolitan University , 0167 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"P\u00e5l","family":"Halvorsen","sequence":"additional","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]},{"name":"Department of Computer Science, Oslo Metropolitan University , 0167 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3153-2064","authenticated-orcid":false,"given":"Michael Alexander","family":"Riegler","sequence":"additional","affiliation":[{"name":"Department of Holistic Systems, SimulaMet , 0170 Oslo,","place":["Norway"]},{"name":"Department of Computer Science, Oslo Metropolitan University , 0167 Oslo,","place":["Norway"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J\u00f8rgen K","family":"Kanters","sequence":"additional","affiliation":[{"name":"Laboratory of Experimental Cardiology, University of Copenhagen , 2200 Copenhagen ,","place":["Denmark"]},{"name":"Center for Biological Research, University of California, San Francisco , San Francisco, CA 94143,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"2024121616105274100_ocae280-B1","doi-asserted-by":"crossref","first-page":"4717","DOI":"10.1093\/eurheartj\/ehab649","article-title":"Application of artificial intelligence to the electrocardiogram","volume":"42","author":"Attia","year":"2021","journal-title":"Eur Heart J"},{"key":"2024121616105274100_ocae280-B2","doi-asserted-by":"crossref","first-page":"18069","DOI":"10.1007\/s00521-019-04051-w","article-title":"The importance of interpretability and visualization in machine learning for applications in medicine and health care","volume":"32","author":"Vellido","year":"2020","journal-title":"Neural Comput Appl"},{"key":"2024121616105274100_ocae280-B3","first-page":"9","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024121616105274100_ocae280-B4","doi-asserted-by":"crossref","first-page":"21896","DOI":"10.1038\/s41598-021-01295-2","article-title":"DeepFake electrocardiograms using generative adversarial networks are the beginning of the end for privacy issues in medicine","volume":"11","author":"Thambawita","year":"2021","journal-title":"Sci Rep"},{"key":"2024121616105274100_ocae280-B5","first-page":"A4693","article-title":"Study design, participation and characteristics of the Danish General Suburban Population Study","volume":"60","author":"Bergholdt","year":"2013","journal-title":"Dan Med J"},{"key":"2024121616105274100_ocae280-B6","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1111\/bjh.14099","article-title":"Increased iron stores prolong the QT interval\u2014a general population study including 20 261 individuals and meta-analysis of thalassaemia major","volume":"174","author":"Henriksen","year":"2016","journal-title":"Br J Haematol"},{"key":"2024121616105274100_ocae280-B7","doi-asserted-by":"crossref","first-page":"2523","DOI":"10.1093\/eurheartj\/ehv297","article-title":"Rare genetic variants previously associated with congenital forms of long QT syndrome have little or no effect on the QT interval","volume":"36","author":"Ghouse","year":"2015","journal-title":"Eur Heart J"},{"key":"2024121616105274100_ocae280-B8","author":"GE Healthcare","year":"2015"},{"key":"2024121616105274100_ocae280-B9","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","author":"He","year":"2016"},{"key":"2024121616105274100_ocae280-B10","first-page":"8026","volume":"721","author":"Paszke"},{"key":"2024121616105274100_ocae280-B11","doi-asserted-by":"crossref","first-page":"10949","DOI":"10.1038\/s41598-021-90285-5","article-title":"Explaining deep neural networks for knowledge discovery in electrocardiogram analysis","volume":"11","author":"Hicks","year":"2021","journal-title":"Sci Rep"},{"key":"2024121616105274100_ocae280-B12","doi-asserted-by":"publisher","author":"Kokhlikyan","year":"2020","DOI":"10.48550\/arXiv.2009.07896"},{"key":"2024121616105274100_ocae280-B13","author":"Simonyan","year":"2014"},{"key":"2024121616105274100_ocae280-B14","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1007\/978-3-319-10590-1_53","volume-title":"Computer Vision\u2014ECCV 2014","author":"Zeiler","year":"2014"},{"key":"2024121616105274100_ocae280-B15","doi-asserted-by":"publisher","author":"Springenberg","year":"2015","DOI":"10.48550\/arXiv.1412.6806"},{"key":"2024121616105274100_ocae280-B16","doi-asserted-by":"publisher","author":"Smilkov","year":"2017","DOI":"10.48550\/arXiv.1706.03825"},{"key":"2024121616105274100_ocae280-B17","first-page":"3145","author":"Shrikumar","year":"2017"},{"key":"2024121616105274100_ocae280-B18","first-page":"3319","author":"Sundararajan","year":"2017"},{"key":"2024121616105274100_ocae280-B19","first-page":"4768","author":"Lundberg","year":"2017"},{"key":"2024121616105274100_ocae280-B20","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: visual explanations from deep networks via gradient-based localization","volume":"128","author":"Selvaraju","year":"2020","journal-title":"Int J Comput Vis"},{"key":"2024121616105274100_ocae280-B21","doi-asserted-by":"crossref","first-page":"2660","DOI":"10.1109\/TNNLS.2016.2599820","article-title":"Evaluating the visualization of what a deep neural network has learned","volume":"28","author":"Samek","year":"2017","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"2024121616105274100_ocae280-B22","doi-asserted-by":"publisher","author":"Ancona","year":"2018","DOI":"10.48550\/arXiv.1711.06104"},{"key":"2024121616105274100_ocae280-B23","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ Psychol Meas"},{"key":"2024121616105274100_ocae280-B24","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"2024121616105274100_ocae280-B25","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1177\/0091270009344853","article-title":"Quantitative analysis of T-wave morphology increases confidence in drug-induced cardiac repolarization abnormalities: evidence from the investigational IKr inhibitor Lu 35-138","volume":"49","author":"Graff","year":"2009","journal-title":"J Clin Pharmacol"},{"key":"2024121616105274100_ocae280-B26","doi-asserted-by":"crossref","first-page":"102684","DOI":"10.1016\/j.media.2022.102684","article-title":"Guidelines and evaluation of clinical explainable AI in medical image analysis","volume":"84","author":"Jin","year":"2023","journal-title":"Med Image Anal"},{"key":"2024121616105274100_ocae280-B27","doi-asserted-by":"crossref","first-page":"e200267","DOI":"10.1148\/ryai.2021200267","article-title":"Assessing the trustworthiness of saliency maps for localizing abnormalities in medical imaging","volume":"3","author":"Arun","year":"2021","journal-title":"Radiol Artif Intell"},{"key":"2024121616105274100_ocae280-B28","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1167\/tvst.9.2.64","article-title":"Systematic comparison of heatmapping techniques in deep learning in the context of diabetic retinopathy lesion detection","volume":"9","author":"Van Craenendonck","year":"2020","journal-title":"Transl Vis Sci Technol"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/1\/79\/61201876\/ocae280.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/1\/79\/61201876\/ocae280.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,16]],"date-time":"2024-12-16T16:11:11Z","timestamp":1734365471000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/1\/79\/7879495"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,6]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,11,6]]},"published-print":{"date-parts":[[2025,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae280","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,11,6]]}}}