{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,17]],"date-time":"2025-11-17T21:32:14Z","timestamp":1763415134073,"version":"3.33.0"},"reference-count":54,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/page\/copyright"},{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100014440","name":"Ministerio de Ciencia, Innovaci\u00f3n y Universidades","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100014440","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100015528","name":"Universidad de La Laguna","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100015528","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Biomed. Phys. Eng. Express"],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>This paper systematically evaluates saliency methods as explainability tools for convolutional neural networks trained to diagnose glaucoma using simplified eye fundus images that contain only disc and cup outlines. These simplified images, a methodological novelty, were used to relate features highlighted in the saliency maps to the geometrical clues that experts consider in glaucoma diagnosis. Despite their simplicity, these images retained sufficient information for accurate classification, with balanced accuracies ranging from 0.8331 to 0.8890, compared to 0.8090 to 0.9203 for networks trained on the original images. The study used a dataset of 606 images, along with RIM-ONE DL and REFUGE datasets, and explored nine saliency methods. A discretization algorithm was applied to reduce noise and compute normalized attribution values for standard eye fundus sectors. Consistent with other medical imaging studies, significant variability was found in the attribution maps, influenced by the method, model, or architecture, and often deviating from typical sectors experts examine. However, globally, the results were relatively stable, with a strong correlation of 0.9289 (<jats:italic>p<\/jats:italic>\u00a0&lt;\u00a00.001) between relevant sectors in our dataset and RIM-ONE DL, and 0.7806 (<jats:italic>p<\/jats:italic>\u00a0&lt;\u00a00.001) for REFUGE. The findings suggest caution when using saliency methods in critical fields like medicine. These methods may be more suitable for broad image relevance interpretation rather than assessing individual cases, where results are highly sensitive to methodological choices. Moreover, the regions identified by the networks do not consistently align with established medical criteria for disease severity.<\/jats:p>","DOI":"10.1088\/2057-1976\/ada8ad","type":"journal-article","created":{"date-parts":[[2025,1,10]],"date-time":"2025-01-10T22:48:02Z","timestamp":1736549282000},"page":"025008","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Systematic application of saliency maps to explain the decisions of convolutional neural networks for glaucoma diagnosis based on disc and cup geometry"],"prefix":"10.1088","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9806-2477","authenticated-orcid":true,"given":"Francisco","family":"Fumero","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jose","family":"Sigut","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7452-2958","authenticated-orcid":true,"given":"Jos\u00e9","family":"Est\u00e9vez","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tinguaro","family":"D\u00edaz-Alem\u00e1n","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2025,1,21]]},"reference":[{"key":"bpexada8adbib1","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1080\/17579961.2024.2313795","article-title":"Unlocking the black box: analysing the EU artificial intelligence act\u2019s framework for explainability","volume":"16","author":"Pavlidis","year":"2024","journal-title":"Law, Innovation and Technology"},{"article-title":"Regulation (eu) 2024\/1689 (artificial intelligence act)","year":"2024","author":"European Parliament and Council","key":"bpexada8adbib2"},{"key":"bpexada8adbib3","doi-asserted-by":"publisher","first-page":"e339","DOI":"10.1016\/S2214-109X(13)70113-X","volume":"1","author":"Bourne","year":"2013","journal-title":"The Lancet Global Health"},{"key":"bpexada8adbib4","doi-asserted-by":"publisher","first-page":"2081","DOI":"10.1016\/j.ophtha.2014.05.013","volume":"121","author":"Tham","year":"2014","journal-title":"Ophthalmology"},{"key":"bpexada8adbib5","doi-asserted-by":"publisher","DOI":"10.1016\/j.xops.2022.100233","volume":"3","author":"Fan","year":"2022","journal-title":"Ophthalmology Science"},{"key":"bpexada8adbib6","article-title":"Medical Image Analysis","volume":"79","author":"van der Velden","year":"2022"},{"key":"bpexada8adbib7","doi-asserted-by":"publisher","first-page":"4793","DOI":"10.1109\/TNNLS.2020.3027314","volume":"32","author":"Tjoa","year":"2021","journal-title":"IEEE Trans Neural Netw. Learn. Syst."},{"key":"bpexada8adbib8","doi-asserted-by":"publisher","first-page":"2128","DOI":"10.1093\/jamia\/ocab127","volume":"28","author":"van der Veer","year":"2021","journal-title":"Journal of the American Medical Informatics Association"},{"key":"bpexada8adbib9","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1002\/ima.22309","volume":"29","author":"Ozturk","year":"2019","journal-title":"Int. J. Imaging Syst. Technol."},{"year":"2013","author":"Simonyan","key":"bpexada8adbib10"},{"key":"bpexada8adbib11","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-42946-w","article-title":"Ground truth based comparison of saliency maps algorithms","volume":"13","author":"Szczepankiewicz","year":"2023","journal-title":"Sci. Rep."},{"key":"bpexada8adbib12","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1001\/archopht.1969.00990020193008","volume":"82","author":"Armaly","year":"1969","journal-title":"Archives of Ophthalmology"},{"key":"bpexada8adbib13","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2021200267","article-title":"Assessing the Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging","volume":"3","author":"Arun","year":"2021","journal-title":"Radiology: Artificial Intelligence"},{"key":"bpexada8adbib14","first-page":"64","article-title":"Translational Vision Science & Technology","volume":"9","author":"Van Craenendonck","year":"2020"},{"key":"bpexada8adbib15","doi-asserted-by":"crossref","first-page":"2357","DOI":"10.1109\/BIBM.2018.8621168","article-title":"Computer-aided diagnosis and localization of glaucoma using deep learning","author":"Kim","year":"2018","journal-title":"2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)"},{"key":"bpexada8adbib16","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105341","volume":"192","author":"Martins","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"bpexada8adbib17","doi-asserted-by":"publisher","first-page":"1405","DOI":"10.1109\/JBHI.2019.2949075","volume":"24","author":"Liao","year":"2020","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"bpexada8adbib18","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.ajo.2021.04.021","volume":"231","author":"Mehta","year":"2021","journal-title":"American Journal of Ophthalmology"},{"key":"bpexada8adbib19","doi-asserted-by":"publisher","first-page":"510","DOI":"10.3390\/diagnostics11030510","volume":"11","author":"Oh","year":"2021","journal-title":"Diagnostics"},{"key":"bpexada8adbib20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3171613","volume":"71","author":"Kamal","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"bpexada8adbib21","doi-asserted-by":"publisher","first-page":"2573","DOI":"10.2147\/OPTH.S312236","volume":"15","author":"Singh","year":"2021","journal-title":"Clinical Ophthalmology"},{"key":"bpexada8adbib22","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1038\/s42256-022-00536-x","volume":"4","author":"Saporta","year":"2022","journal-title":"Nature Machine Intelligence"},{"key":"bpexada8adbib23","doi-asserted-by":"publisher","first-page":"103225","DOI":"10.1016\/j.ipm.2022.103225","volume":"60","author":"Amorim","year":"2023","journal-title":"Inf. Process. Manage."},{"key":"bpexada8adbib24","doi-asserted-by":"publisher","first-page":"161 167","DOI":"10.5566\/ias.2346","volume":"39","author":"Fumero","year":"2020","journal-title":"Image Analysis & Stereology"},{"key":"bpexada8adbib25","first-page":"1","article-title":"Rim-one: An open retinal image database for optic nerve evaluation","author":"Fumero","year":"2011","journal-title":"2011 24th International Symposium on Computer-Based Medical Systems (CBMS)"},{"key":"bpexada8adbib26","first-page":"91","article-title":"Interactive tool and database for optic disc and cup segmentation of stereo and monocular retinal fundus images","author":"Fumero","year":"2015","journal-title":"Short papers proceedings of the 23rd International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG 2015)"},{"key":"bpexada8adbib27","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.101570","volume":"59","author":"Orlando","year":"2020","journal-title":"Med. Image Anal."},{"article-title":"Very deep convolutional networks for large-scale image recognition","year":"2015","author":"Simonyan","key":"bpexada8adbib28"},{"key":"bpexada8adbib29","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"bpexada8adbib30","first-page":"2818","article-title":"Rethinking the inception architecture for computer vision","author":"Szegedy","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"bpexada8adbib31","first-page":"1800","article-title":"Xception: Deep learning with depthwise separable convolutions","author":"Chollet","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"article-title":"TensorFlow: Large-scale machine learning on heterogeneous systems software","year":"2015","author":"Abadi","key":"bpexada8adbib32"},{"key":"bpexada8adbib33","first-page":"169","article-title":"Gradient-based attribution methods","author":"Ancona","year":"2019"},{"key":"bpexada8adbib34","first-page":"1803","volume":"11","author":"Baherens","year":"2010","journal-title":"Journal of Machine Learning Research"},{"key":"bpexada8adbib35","first-page":"3145","article-title":"Learning important features through propagating activation differences","author":"Shrikumar","year":"2017","journal-title":"Proceedings of the 34th International Conference on Machine Learning (Proceedings of Machine Learning Research vol 70)"},{"key":"bpexada8adbib36","article-title":"Striving for simplicity: the all convolutional net","author":"Springenberg","year":"2015","journal-title":"International Conference on Learning Representations (workshop track) pp"},{"key":"bpexada8adbib37","article-title":"Sanity checks for saliency maps","volume":"vol 31","author":"Adebayo","year":"2018"},{"article-title":"Smoothgrad: removing noise by adding noise","year":"2017","author":"Smilkov","key":"bpexada8adbib38"},{"key":"bpexada8adbib39","first-page":"3319 3328","article-title":"Axiomatic attribution for deep networks","volume":"vol 70","author":"Sundararajan","year":"2017"},{"key":"bpexada8adbib40","first-page":"pp 818 833","article-title":"Visualizing and understanding convolutional networks","author":"Zeiler","year":"2014"},{"key":"bpexada8adbib41","doi-asserted-by":"publisher","first-page":"618","DOI":"10.1109\/ICCV.2017.74","article-title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","author":"Selvaraju","year":"2017","journal-title":"2017 IEEE International Conference on Computer Vision (ICCV) (Los Alamitos, CA, USA: IEEE Computer Society)"},{"key":"bpexada8adbib42","first-page":"24","article-title":"Score-CAM: Score-weighted visual explanations for convolutional neural networks","author":"Wang","year":"2020","journal-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition workshops"},{"key":"bpexada8adbib43","first-page":"261","article-title":"The Asia-Pacific","volume":"10","author":"Lee","year":"2021","journal-title":"Journal of Ophthalmology"},{"key":"bpexada8adbib44","first-page":"1","volume":"105","author":"(European Glaucoma Society)","year":"2021","journal-title":"British Journal of Ophthalmology"},{"key":"bpexada8adbib45","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1016\/S0039-6257(98)00049-6","volume":"43","author":"Jonas","year":"1999","journal-title":"Survey of Ophthalmology"},{"key":"bpexada8adbib46","doi-asserted-by":"publisher","first-page":"1809","DOI":"10.1016\/S0161-6420(00)00284-0","volume":"107","author":"Garway-Heath","year":"2000","journal-title":"Ophthalmology"},{"key":"bpexada8adbib47","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1080\/01621459.1972.10481251","volume":"67","author":"Zar","year":"1972","journal-title":"J. Am. Stat. Assoc."},{"key":"bpexada8adbib48","doi-asserted-by":"crossref","first-page":"3121","DOI":"10.1109\/ICPR.2010.764","article-title":"The balanced accuracy and its posterior distribution","author":"Brodersen","year":"2010","journal-title":"2010 20th International Conference on Pattern Recognition"},{"key":"bpexada8adbib49","doi-asserted-by":"publisher","first-page":"898","DOI":"10.1109\/TPAMI.2010.161","volume":"33","author":"Arbel\u00e1ez","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"bpexada8adbib50","article-title":"Imagenet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness","author":"Geirhos","year":"2019","journal-title":"International Conference on Learning Representations"},{"key":"bpexada8adbib51","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1097\/IJG.0b013e3181996f05","volume":"18","author":"Seider","year":"2009","journal-title":"Journal of Glaucoma"},{"key":"bpexada8adbib52","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1001\/archopthalmol.2011.1576","volume":"130","author":"(Cirrus OCT Normative Database Study Group f t)","year":"2012","journal-title":"Archives of Ophthalmology"},{"key":"bpexada8adbib53","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1109\/ICARC54489.2022.9754171","article-title":"Cnn based fundus images classification for glaucoma identification","author":"Shyamalee","year":"2022","journal-title":"2022 2nd International Conference on Advanced Research in Computing (ICARC)"},{"key":"bpexada8adbib54","doi-asserted-by":"publisher","first-page":"045024","DOI":"10.1088\/2632-2153\/ad0798","article-title":"Interpretable surrogate models to approximate the predictions of convolutional neural networks in glaucoma diagnosis","volume":"4","author":"Sigut","year":"2023","journal-title":"Machine Learning: Science and Technology"}],"container-title":["Biomedical Physics &amp; Engineering Express"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T14:54:31Z","timestamp":1737471271000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2057-1976\/ada8ad"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,21]]},"references-count":54,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,1,21]]},"published-print":{"date-parts":[[2025,3,31]]}},"URL":"https:\/\/doi.org\/10.1088\/2057-1976\/ada8ad","relation":{},"ISSN":["2057-1976"],"issn-type":[{"type":"electronic","value":"2057-1976"}],"subject":[],"published":{"date-parts":[[2025,1,21]]},"assertion":[{"value":"Systematic application of saliency maps to explain the decisions of convolutional neural networks for glaucoma diagnosis based on disc and cup geometry","name":"article_title","label":"Article Title"},{"value":"Biomedical Physics & Engineering Express","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2025 IOP Publishing Ltd. All rights, including for text and data mining, AI training, and similar technologies, are reserved.","name":"copyright_information","label":"Copyright Information"},{"value":"2024-09-13","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-01-10","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2025-01-21","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}