{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T15:03:30Z","timestamp":1775228610348,"version":"3.50.1"},"reference-count":73,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2020,6,20]],"date-time":"2020-06-20T00:00:00Z","timestamp":1592611200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Deep learning methods have been very effective for a variety of medical diagnostic tasks and have even outperformed human experts on some of those. However, the black-box nature of the algorithms has restricted their clinical use. Recent explainability studies aim to show the features that influence the decision of a model the most. The majority of literature reviews of this area have focused on taxonomy, ethics, and the need for explanations. A review of the current applications of explainable deep learning for different medical imaging tasks is presented here. The various approaches, challenges for clinical deployment, and the areas requiring further research are discussed here from a practical standpoint of a deep learning researcher designing a system for the clinical end-users.<\/jats:p>","DOI":"10.3390\/jimaging6060052","type":"journal-article","created":{"date-parts":[[2020,6,22]],"date-time":"2020-06-22T06:46:12Z","timestamp":1592808372000},"page":"52","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":574,"title":["Explainable Deep Learning Models in Medical Image Analysis"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3874-9570","authenticated-orcid":false,"given":"Amitojdeep","family":"Singh","sequence":"first","affiliation":[{"name":"Theoretical and Experimental Epistemology Laboratory, School of Optometry and Vision Science, University of Waterloo, Waterloo, ON N2L 3G1, Canada"},{"name":"Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada"}]},{"given":"Sourya","family":"Sengupta","sequence":"additional","affiliation":[{"name":"Theoretical and Experimental Epistemology Laboratory, School of Optometry and Vision Science, University of Waterloo, Waterloo, ON N2L 3G1, Canada"},{"name":"Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3473-1245","authenticated-orcid":false,"given":"Vasudevan","family":"Lakshminarayanan","sequence":"additional","affiliation":[{"name":"Theoretical and Experimental Epistemology Laboratory, School of Optometry and Vision Science, University of Waterloo, Waterloo, ON N2L 3G1, Canada"},{"name":"Department of Systems Design Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2020,6,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"220","DOI":"10.3389\/fnagi.2019.00220","article-title":"Deep learning in Alzheimer\u2019s disease: Diagnostic classification and prognostic prediction using neuroimaging data","volume":"11","author":"Jo","year":"2019","journal-title":"Front. Aging Neurosci."},{"key":"ref_2","first-page":"2015","article-title":"Computer-aided classification of lung nodules on computed tomography images via deep learning technique","volume":"8","author":"Hua","year":"2015","journal-title":"OncoTargets Ther."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"101758","DOI":"10.1016\/j.artmed.2019.101758","article-title":"Ophthalmic diagnosis using deep learning with fundus images\u2013A critical review","volume":"102","author":"Sengupta","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_4","unstructured":"El-Baz, A.S. (2020). Recent Advances in Deep Learning Applications for Retinal Diagnosis using OCT. State of the Art in Neural Networks, Elsevier. in press."},{"key":"ref_5","unstructured":"Holzinger, A., Biemann, C., Pattichis, C.S., and Kell, D.B. (2017). What do we need to build explainable AI systems for the medical domain?. arXiv."},{"key":"ref_6","unstructured":"Stano, M., Benesova, W., and Martak, L.S. (2019, January 16\u201318). Explainable 3D Convolutional Neural Network Using GMM Encoding. Proceedings of the Twelfth International Conference on Machine Vision, Amsterdam, The Netherlands."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2649","DOI":"10.1109\/TBME.2018.2813015","article-title":"Uncertainty-aware organ classification for surgical data science applications in laparoscopy","volume":"65","author":"Moccia","year":"2018","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1007\/s11548-019-01939-9","article-title":"Uncertainty-aware performance assessment of optical imaging modalities with invertible neural networks","volume":"14","author":"Adler","year":"2019","journal-title":"Int. J. Comput. Assist. Radiol. Surg."},{"key":"ref_9","unstructured":"Meyes, R., de Puiseau, C.W., Posada-Moreno, A., and Meisen, T. (2020). Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.inffus.2019.12.012","article-title":"Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI","volume":"58","author":"Arrieta","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Stiglic, G., Kocbek, P., Fijacko, N., Zitnik, M., Verbert, K., and Cilar, L. (2020). Interpretability of machine learning based prediction models in healthcare. arXiv.","DOI":"10.1002\/widm.1379"},{"key":"ref_12","unstructured":"Arya, V., Bellamy, R.K., Chen, P.Y., Dhurandhar, A., Hind, M., Hoffman, S.C., Houde, S., Liao, Q.V., Luss, R., and Mojsilovi\u0107, A. (2019). One explanation does not fit all: A toolkit and taxonomy of ai explainability techniques. arXiv."},{"key":"ref_13","unstructured":"Ying, Z., Bourgeois, D., You, J., Zitnik, M., and Leskovec, J. (2019, January 8\u201314). Gnnexplainer: Generating Explanations for Graph Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 32, Vancouver, BC, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/0169-7439(87)80084-9","article-title":"Principal component analysis","volume":"2","author":"Wold","year":"1987","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_15","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1109\/21.97458","article-title":"A survey of decision tree classifier methodology","volume":"21","author":"Safavian","year":"1991","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/j.patcog.2016.11.008","article-title":"Explaining nonlinear classification decisions with deep taylor decomposition","volume":"65","author":"Montavon","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_18","unstructured":"Ancona, M., Ceolini, E., \u00d6ztireli, C., and Gross, M. (2017). Towards better understanding of gradient-based attribution methods for deep neural networks. arXiv."},{"key":"ref_19","first-page":"1","article-title":"iNNvestigate neural networks","volume":"20","author":"Alber","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014, January 6\u201312). Visualizing and Understanding Convolutional Networks. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_21","unstructured":"Goodfellow, I.J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1002\/asmb.446","article-title":"Analysis of regression in game theory approach","volume":"17","author":"Lipovetsky","year":"2001","journal-title":"Appl. Stoch. Model. Bus. Ind."},{"key":"ref_23","unstructured":"Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv."},{"key":"ref_24","unstructured":"Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bach, S., Binder, A., Montavon, G., Klauschen, F., M\u00fcller, K.R., and Samek, W. (2015). On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0130140"},{"key":"ref_26","unstructured":"Shrikumar, A., Greenside, P., Shcherbina, A., and Kundaje, A. (2016). Not just a black box: Learning important features through propagating activation differences. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-Cam: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_28","unstructured":"Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6\u201311). Axiomatic Attribution for Deep Networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_29","unstructured":"Kindermans, P.J., Sch\u00fctt, K.T., Alber, M., M\u00fcller, K.R., Erhan, D., Kim, B., and D\u00e4hne, S. (2017). Learning how to explain neural networks: Patternnet and patternattribution. arXiv."},{"key":"ref_30","unstructured":"Shrikumar, A., Greenside, P., and Kundaje, A. (2017, January 6\u201311). Learning Important Features through Propagating Activation Differences. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_31","unstructured":"Smilkov, D., Thorat, N., Kim, B., Vi\u00e9gas, F., and Wattenberg, M. (2017). Smoothgrad: Removing noise by adding noise. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, H., Lundberg, S., and Lee, S.I. (2019). Explaining Models by Propagating Shapley Values of Local Components. arXiv.","DOI":"10.1007\/978-3-030-53352-6_24"},{"key":"ref_33","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014, January 8\u201313). How Transferable Are Features in Deep Neural Networks?. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, USA."},{"key":"ref_34","first-page":"111390U","article-title":"Glaucoma diagnosis using transfer learning methods","volume":"Volume 11139","author":"Singh","year":"2019","journal-title":"Proceedings of the Applications of Machine Learning"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_36","unstructured":"Suzuki, K. (2019). Testing the Robustness of Attribution Methods for Convolutional Neural Networks in MRI-Based Alzheimer\u2019s Disease Classification. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, ML-CDS 2019, IMIMIC 2019, Springer. Lecture Notes in Computer Science."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Pereira, S., Meier, R., Alves, V., Reyes, M., and Silva, C.A. (2018). Automatic brain tumor grading from MRI data using convolutional neural networks and quality assessment. Understanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer.","DOI":"10.1007\/978-3-030-02628-8_12"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1016\/j.ophtha.2018.11.016","article-title":"Using a deep learning algorithm and integrated gradients explanation to assist grading for diabetic retinopathy","volume":"126","author":"Sayres","year":"2019","journal-title":"Ophthalmology"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yang, H.L., Kim, J.J., Kim, J.H., Kang, Y.K., Park, D.H., Park, H.S., Kim, H.K., and Kim, M.S. (2019). Weakly supervised lesion localization for age-related macular degeneration detection using optical coherence tomography images. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0215076"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Singh, A., Sengupta, S., and Lakshminarayanan, V. (2020). Interpretation of deep learning using attributions: Application to ophthalmic diagnosis. Proceedings of the Applications of Machine Learning, International Society for Optics and Photonics (SPIE). in press.","DOI":"10.1117\/12.2568631"},{"key":"ref_41","first-page":"113140Z","article-title":"Explainable AI for medical imaging: Deep-learning CNN ensemble for classification of estrogen receptor status from breast MRI","volume":"Volume 11314","author":"Papanastasopoulos","year":"2020","journal-title":"Proceedings of the SPIE Medical Imaging 2020: Computer-Aided Diagnosis"},{"key":"ref_42","unstructured":"L\u00e9vy, D., and Jain, A. (2016). Breast mass classification from mammograms using deep convolutional neural networks. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_44","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_45","unstructured":"Mordvintsev, A., Olah, C., and Tyka, M. (2020, May 23). Inceptionism: Going Deeper into Neural Networks. Google AI Blog. Available online: https:\/\/ai.googleblog.com\/2015\/06\/inceptionism-going-deeper-into-neural.html."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Couteaux, V., Nempont, O., Pizaine, G., and Bloch, I. (2019). Towards Interpretability of Segmentation Networks by Analyzing DeepDreams. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_7"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Wang, L., and Wong, A. (2020). COVID-Net: A tailored deep convolutional neural network design for detection of COVID-19 cases from chest radiography images. arXiv.","DOI":"10.1038\/s41598-020-76550-z"},{"key":"ref_48","unstructured":"Lin, Z.Q., Shafiee, M.J., Bochkarev, S., Jules, M.S., Wang, X.Y., and Wong, A. (2019). Explaining with Impact: A Machine-centric Strategy to Quantify the Performance of Explainability Algorithms. arXiv."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Young, K., Booth, G., Simpson, B., Dutton, R., and Shrapnel, S. (2019). Deep neural network or dermatologist?. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_6"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Van Molle, P., De Strooper, M., Verbelen, T., Vankeirsbilck, B., Simoens, P., and Dhoedt, B. (2018). Visualizing convolutional neural networks to improve decision support for skin lesion classification. Understanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer.","DOI":"10.1007\/978-3-030-02628-8_13"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"101619","DOI":"10.1016\/j.media.2019.101619","article-title":"Uncertainty and interpretability in convolutional neural networks for semantic segmentation of colorectal polyps","volume":"60","author":"Kampffmeyer","year":"2020","journal-title":"Med. Image Anal."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"034502","DOI":"10.1117\/1.JMI.4.3.034502","article-title":"Confident texture-based laryngeal tissue classification for early stage diagnosis support","volume":"4","author":"Moccia","year":"2017","journal-title":"J. Med. Imaging"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat. Biomed. Eng."},{"key":"ref_54","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention Is All You Need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_55","first-page":"112320K","article-title":"Classification of brain lesions from MRI images using a novel neural network","volume":"Volume 11232","author":"Bamba","year":"2020","journal-title":"Multimodal Biomedical Imaging XV"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Xie, Y., Xing, F., McGough, M., and Yang, L. (2017, January 21\u201326). Mdnet: A Semantically and Visually Interpretable Medical Image Diagnosis Network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.378"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Sun, J., Darbeha, F., Zaidi, M., and Wang, B. (2020). SAUNet: Shape Attentive U-Net for Interpretable Medical Image Segmentation. arXiv.","DOI":"10.1007\/978-3-030-59719-1_77"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_59","unstructured":"Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., and Sayres, R. (2017). Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). arXiv."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Graziani, M., Andrearczyk, V., and M\u00fcller, H. (2018). Regression concept vectors for bidirectional explanations in histopathology. Understanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer.","DOI":"10.1007\/978-3-030-02628-8_14"},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Yeche, H., Harrison, J., and Berthier, T. (2019). UBS: A Dimension-Agnostic Metric for Concept Vector Interpretability Applied to Radiomics. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_2"},{"key":"ref_62","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Pisov, M., Goncharov, M., Kurochkina, N., Morozov, S., Gombolevsky, V., Chernina, V., Vladzymyrskyy, A., Zamyatina, K., Cheskova, A., and Pronin, I. (2019). Incorporating Task-Specific Structural Knowledge into CNNs for Brain Midline Shift Detection. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_4"},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Zhu, P., and Ogino, M. (2019). Guideline-Based Additive Explanation for Computer-Aided Diagnosis of Lung Nodules. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_5"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Codella, N.C., Lin, C.C., Halpern, A., Hind, M., Feris, R., and Smith, J.R. (2018). Collaborative Human-AI (CHAI): Evidence-based interpretable melanoma classification in dermoscopic images. Understanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer.","DOI":"10.1007\/978-3-030-02628-8_11"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Silva, W., Fernandes, K., Cardoso, M.J., and Cardoso, J.S. (2018). Towards complementary explanations using deep neural networks. Understanding and Interpreting Machine Learning in Medical Image Computing Applications, Springer.","DOI":"10.1007\/978-3-030-02628-8_15"},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Lee, H., Kim, S.T., and Ro, Y.M. (2019). Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosis. Interpretability of Machine Intelligence in Medical Image Computing and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1007\/978-3-030-33850-3_3"},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Biffi, C., Cerrolaza, J.J., Tarroni, G., Bai, W., De Marvao, A., Oktay, O., Ledig, C., Le Folgoc, L., Kamnitsas, K., and Doumou, G. (2020). Explainable Anatomical Shape Analysis through Deep Hierarchical Generative Models. IEEE Trans. Med. Imaging.","DOI":"10.1109\/TMI.2020.2964499"},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Eslami, T., Raiker, J.S., and Saeed, F. (2020). Explainable and Scalable Machine-Learning Algorithms for Detection of Autism Spectrum Disorder using fMRI Data. arXiv.","DOI":"10.1016\/B978-0-12-822822-7.00004-1"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Sha, Y., and Wang, M.D. (2017, January 20\u201323). Interpretable Predictions of Clinical Outcomes with an Attention-Based Recurrent Neural Network. Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, Boston, MA, USA.","DOI":"10.1145\/3107411.3107445"},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., and Wortman Vaughan, J. (2020, January 25\u201330). Interpreting Interpretability: Understanding Data Scientists\u2019 Use of Interpretability Tools for Machine Learning. Proceedings of the CHI Conference on Human Factors in Computing Systems, Honolulu, HI, USA.","DOI":"10.1145\/3313831.3376219"},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-017-0015-z","article-title":"Advanced machine learning in action: Identification of intracranial hemorrhage on computed tomography scans of the head with clinical workflow integration","volume":"1","author":"Arbabshirani","year":"2018","journal-title":"NPJ Digit. Med."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1007\/s10792-016-0329-x","article-title":"Agreement among ophthalmologists in marking the optic disc and optic cup in fundus images","volume":"37","author":"Almazroa","year":"2017","journal-title":"Int. Ophthalmol."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/6\/52\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:41:15Z","timestamp":1760175675000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/6\/6\/52"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,6,20]]},"references-count":73,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2020,6]]}},"alternative-id":["jimaging6060052"],"URL":"https:\/\/doi.org\/10.3390\/jimaging6060052","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,6,20]]}}}