{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T11:24:21Z","timestamp":1779189861571,"version":"3.51.4"},"reference-count":83,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,2,8]],"date-time":"2024-02-08T00:00:00Z","timestamp":1707350400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Structural and Investment Funds (ESF)","award":["ZS\/08\/80646"],"award-info":[{"award-number":["ZS\/08\/80646"]}]},{"name":"European Structural and Investment Funds (ESF)","award":["13GW0473A"],"award-info":[{"award-number":["13GW0473A"]}]},{"name":"European Structural and Investment Funds (ESF)","award":["13GW0473B"],"award-info":[{"award-number":["13GW0473B"]}]},{"name":"Federal Ministry of Education and Research","award":["ZS\/08\/80646"],"award-info":[{"award-number":["ZS\/08\/80646"]}]},{"name":"Federal Ministry of Education and Research","award":["13GW0473A"],"award-info":[{"award-number":["13GW0473A"]}]},{"name":"Federal Ministry of Education and Research","award":["13GW0473B"],"award-info":[{"award-number":["13GW0473B"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>The outbreak of COVID-19 has shocked the entire world with its fairly rapid spread, and has challenged different sectors. One of the most effective ways to limit its spread is the early and accurate diagnosing of infected patients. Medical imaging, such as X-ray and computed tomography (CT), combined with the potential of artificial intelligence (AI), plays an essential role in supporting medical personnel in the diagnosis process. Thus, in this article, five different deep learning models (ResNet18, ResNet34, InceptionV3, InceptionResNetV2, and DenseNet161) and their ensemble, using majority voting, have been used to classify COVID-19, pneumoni\u00e6 and healthy subjects using chest X-ray images. Multilabel classification was performed to predict multiple pathologies for each patient, if present. Firstly, the interpretability of each of the networks was thoroughly studied using local interpretability methods\u2014occlusion, saliency, input X gradient, guided backpropagation, integrated gradients, and DeepLIFT\u2014and using a global technique\u2014neuron activation profiles. The mean micro F1 score of the models for COVID-19 classifications ranged from 0.66 to 0.875, and was 0.89 for the ensemble of the network models. The qualitative results showed that the ResNets were the most interpretable models. This research demonstrates the importance of using interpretability methods to compare different models before making a decision regarding the best performing model.<\/jats:p>","DOI":"10.3390\/jimaging10020045","type":"journal-article","created":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T08:12:03Z","timestamp":1707466323000},"page":"45","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Exploration of Interpretability Techniques for Deep COVID-19 Classification Using Chest X-ray Images"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7594-1188","authenticated-orcid":false,"given":"Soumick","family":"Chatterjee","sequence":"first","affiliation":[{"name":"Data and Knowledge Engineering Group, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Faculty of Computer Science, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Genomics Research Centre, Human Technopole, 20157 Milan, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9732-4292","authenticated-orcid":false,"given":"Fatima","family":"Saad","sequence":"additional","affiliation":[{"name":"Institute for Medical Engineering, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Research Campus STIMULATE, Otto von Guericke University, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4760-2263","authenticated-orcid":false,"given":"Chompunuch","family":"Sarasaen","sequence":"additional","affiliation":[{"name":"Institute for Medical Engineering, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Research Campus STIMULATE, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Biomedical Magnetic Resonance, Otto von Guericke University, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suhita","family":"Ghosh","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Artificial Intelligence Lab, Otto von Guericke University, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Valerie","family":"Krug","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Artificial Intelligence Lab, Otto von Guericke University, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rupali","family":"Khatun","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Barcelona, 08028 Barcelona, Spain"},{"name":"Translational Radiobiology, Department of Radiation Oncology, Universit\u00e4tsklinikum Erlangen, 91054 Erlangen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rahul","family":"Mishra","sequence":"additional","affiliation":[{"name":"Apollo Hospitals, Bilaspur 495006, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nirja","family":"Desai","sequence":"additional","affiliation":[{"name":"HCG Cancer Centre, Vadodara 390012, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Petia","family":"Radeva","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Barcelona, 08028 Barcelona, Spain"},{"name":"Computer Vision Centre, 08193 Cerdanyola, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Georg","family":"Rose","sequence":"additional","affiliation":[{"name":"Institute for Medical Engineering, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Research Campus STIMULATE, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Centre for Behavioural Brain Sciences, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1717-4133","authenticated-orcid":false,"given":"Sebastian","family":"Stober","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Artificial Intelligence Lab, Otto von Guericke University, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6019-5597","authenticated-orcid":false,"given":"Oliver","family":"Speck","sequence":"additional","affiliation":[{"name":"Research Campus STIMULATE, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Biomedical Magnetic Resonance, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Centre for Behavioural Brain Sciences, 39106 Magdeburg, Germany"},{"name":"German Centre for Neurodegenerative Diseases, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4311-0624","authenticated-orcid":false,"given":"Andreas","family":"N\u00fcrnberger","sequence":"additional","affiliation":[{"name":"Data and Knowledge Engineering Group, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Faculty of Computer Science, Otto von Guericke University, 39106 Magdeburg, Germany"},{"name":"Centre for Behavioural Brain Sciences, 39106 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1056\/NEJMoa2001017","article-title":"A novel coronavirus from patients with pneumonia in China, 2019","volume":"382","author":"Zhu","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1056\/NEJMoa2001316","article-title":"Early transmission dynamics in Wuhan, China, of novel coronavirus\u2013infected pneumonia","volume":"382","author":"Li","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_3","unstructured":"(2024, January 24). Radiopaedia: COVID-19. Available online: https:\/\/radiopaedia.org\/articles\/covid-19-3."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"E32","DOI":"10.1148\/radiol.2020200642","article-title":"Correlation of chest CT and RT-PCR testing in coronavirus disease 2019 (COVID-19) in China: A report of 1014 cases","volume":"296","author":"Ai","year":"2020","journal-title":"Radiology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"E115","DOI":"10.1148\/radiol.2020200432","article-title":"Sensitivity of chest CT for COVID-19: Comparison to RT-PCR","volume":"296","author":"Fang","year":"2020","journal-title":"Radiology"},{"key":"ref_6","unstructured":"Zhang, J., Xie, Y., Li, Y., Shen, C., and Xia, Y. (2020). Covid-19 screening on chest x-ray images using deep learning based anomaly detection. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Narin, A., Kaya, C., and Pamuk, Z. (2020). Automatic detection of coronavirus disease (COVID-19) using x-ray images and deep convolutional neural networks. arXiv.","DOI":"10.1007\/s10044-021-00984-y"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1007\/s13246-020-00865-4","article-title":"COVID-19: Automatic detection from x-ray images utilizing transfer learning with convolutional neural networks","volume":"43","author":"Apostolopoulos","year":"2020","journal-title":"Phys. Eng. Sci. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1148\/radiol.2020200241","article-title":"Chest CT findings in 2019 novel coronavirus (2019-nCoV) infections from Wuhan, China: Key points for the radiologist","volume":"295","author":"Kanne","year":"2020","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1148\/radiol.2020200463","article-title":"Chest CT findings in coronavirus disease-19 (COVID-19): Relationship to duration of infection","volume":"295","author":"Bernheim","year":"2020","journal-title":"Radiology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"E41","DOI":"10.1148\/radiol.2020200343","article-title":"Chest CT for typical 2019-nCoV pneumonia: Relationship to negative RT-PCR testing","volume":"296","author":"Xie","year":"2020","journal-title":"Radiology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1148\/radiol.2020200330","article-title":"Use of chest CT in combination with negative RT-PCR assay for the 2019 novel coronavirus but high clinical suspicion","volume":"295","author":"Huang","year":"2020","journal-title":"Radiology"},{"key":"ref_13","first-page":"1767","article-title":"The COVID-19 pandemic in the US: A clinical update","volume":"323","author":"Omer","year":"2020","journal-title":"JAMA"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1148\/radiol.2020201365","article-title":"The role of chest imaging in patient management during the COVID-19 pandemic: A multinational consensus statement from the Fleischner Society","volume":"296","author":"Rubin","year":"2020","journal-title":"Radiology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1515\/dx-2020-0058","article-title":"The optimal diagnostic methods for COVID-19","volume":"7","author":"Harahwa","year":"2020","journal-title":"Diagnosis"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.clinimag.2020.04.001","article-title":"Portable chest X-ray in coronavirus disease-19 (COVID-19): A pictorial review","volume":"64","author":"Jacobi","year":"2020","journal-title":"Clin. Imaging"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1708","DOI":"10.1056\/NEJMoa2002032","article-title":"Clinical characteristics of coronavirus disease 2019 in China","volume":"382","author":"Guan","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"S22","DOI":"10.12669\/pjms.36.COVID19-S4.2778","article-title":"Chest X-rays findings in COVID 19 patients at a University Teaching Hospital\u2014A descriptive study","volume":"36","author":"Durrani","year":"2020","journal-title":"Pak. J. Med. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"E72","DOI":"10.1148\/radiol.2020201160","article-title":"Frequency and distribution of chest radiographic findings in COVID-19 positive patients","volume":"296","author":"Wong","year":"2020","journal-title":"Radiology"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e200034","DOI":"10.1148\/ryct.2020200034","article-title":"Imaging profile of the COVID-19 infection: Radiologic findings and literature review","volume":"2","author":"Ng","year":"2020","journal-title":"Radiol. Cardiothorac. Imaging"},{"key":"ref_21","unstructured":"Cohen, J.P., Morrison, P., and Dao, L. (2020). COVID-19 image data collection. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., and Acharya, U.R. (2020). Automated detection of COVID-19 cases using deep neural networks with X-ray images. Comput. Biol. Med., 121.","DOI":"10.1016\/j.compbiomed.2020.103792"},{"key":"ref_23","unstructured":"Liu, J., Cao, L., Akin, O., and Tian, Y. (2019). Accurate and Robust Pulmonary Nodule Detection by 3D Feature Pyramid Network with Self-Supervised Feature Learning. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"19518","DOI":"10.1038\/s41598-019-55972-4","article-title":"Prostate cancer Detection using Deep convolutional neural networks","volume":"9","author":"Yoo","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_25","unstructured":"T\u00f4, T.D., Lan, D.T., Nguyen, T.T.H., Nguyen, T.T.N., Nguyen, H.P., Phuong, L., and Nguyen, T.Z. (2024, January 24). Ensembled Skin Cancer Classification. ISIC 2019 Challenge Submission, 2019. Available online: https:\/\/hal.science\/hal-02335240v1\/file\/Combined_approach_to_skin_cancer_classification.pdf."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"803","DOI":"10.21037\/tcr.2018.05.02","article-title":"The role of deep learning and radiomic feature extraction in cancer-specific predictive modelling: A review","volume":"7","author":"Vial","year":"2018","journal-title":"Transl. Cancer Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"94","DOI":"10.7861\/futurehosp.6-2-94","article-title":"The potential for artificial intelligence in healthcare","volume":"6","author":"Davenport","year":"2019","journal-title":"Future Healthc. J."},{"key":"ref_28","unstructured":"Sloane, E.B., and Silva, R.J. (2020). Clinical Engineering Handbook, Elsevier."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e228","DOI":"10.1002\/mp.13562","article-title":"Artificial intelligence-based clinical decision support in modern medical physics: Selection, acceptance, commissioning, and quality assurance","volume":"47","author":"Mahadevaiah","year":"2020","journal-title":"Med. Phys."},{"key":"ref_30","unstructured":"Agrebi, S., and Larbi, A. (2020). Artificial Intelligence in Precision Health, Elsevier."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1504\/IJBIC.2019.098405","article-title":"Computer aided diagnosis of drug sensitive pulmonary tuberculosis with cavities, consolidations and nodular manifestations on lung CT images","volume":"13","author":"Sweetlin","year":"2019","journal-title":"Int. J. Bio Inspired Comput."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/j.acra.2010.11.013","article-title":"Computer-aided diagnosis of pulmonary infections using texture analysis and support vector machine classification","volume":"18","author":"Yao","year":"2011","journal-title":"Acad. Radiol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"E65","DOI":"10.1148\/radiol.2020200905","article-title":"Using artificial intelligence to detect COVID-19 and community-acquired pneumonia based on pulmonary CT: Evaluation of the diagnostic accuracy","volume":"296","author":"Li","year":"2020","journal-title":"Radiology"},{"key":"ref_34","first-page":"1","article-title":"Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography","volume":"10","author":"Chen","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, L., Wong, A., and Qui Lin, Z. (2020). COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-ray Images. arXiv.","DOI":"10.1038\/s41598-020-76550-z"},{"key":"ref_36","unstructured":"Ghoshal, B., and Tucker, A. (2020). Estimating uncertainty and interpretability in deep learning for coronavirus (COVID-19) detection. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"85198","DOI":"10.1109\/ACCESS.2021.3087583","article-title":"An interpretable deep learning model for COVID-19 detection with chest X-ray images","volume":"9","author":"Singh","year":"2021","journal-title":"IEEE Access"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Singh, G., and Yow, K.C. (2021). Object or background: An interpretable deep learning model for COVID-19 detection from CT-scan images. Diagnostics, 11.","DOI":"10.3390\/diagnostics11091732"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-020-00392-9","article-title":"Deep Learning applications for COVID-19","volume":"8","author":"Shorten","year":"2021","journal-title":"J. Big Data"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"16061","DOI":"10.1007\/s00521-021-06806-w","article-title":"Classification of Covid-19 chest X-ray images by means of an interpretable evolutionary rule-based approach","volume":"35","author":"Sannino","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014, January 6\u201312). Visualizing and understanding convolutional networks. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_42","unstructured":"Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Mahendran, A., and Vedaldi, A. (2016, January 11\u201314). Salient deconvolutional networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46466-4_8"},{"key":"ref_44","unstructured":"Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6\u201311). Axiomatic attribution for deep networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia."},{"key":"ref_45","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_46","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zagoruyko, S., and Komodakis, N. (2016). Wide residual networks. arXiv.","DOI":"10.5244\/C.30.87"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_52","unstructured":"Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv."},{"key":"ref_53","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_55","unstructured":"(2024, January 24). Interpretable Machine Learning: A Guide for Making Black-Box Models Explainable. 2022. Available online: https:\/\/christophm.github.io\/interpretable-ml-book."},{"key":"ref_56","unstructured":"Kopitar, L., Cilar, L., Kocbek, P., and Stiglic, G. (2019). Artificial Intelligence in Medicine: Knowledge Representation and Transparent and Explainable Systems, Springer."},{"key":"ref_57","unstructured":"Kindermans, P.J., Sch\u00fctt, K., M\u00fcller, K.R., and D\u00e4hne, S. (2016). Investigating the influence of noise and distractors on the interpretation of neural networks. arXiv."},{"key":"ref_58","unstructured":"Springenberg, J.T., Dosovitskiy, A., Brox, T., and Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. arXiv."},{"key":"ref_59","unstructured":"Shrikumar, A., Greenside, P., and Kundaje, A. (2017, January 6\u201311). Learning important features through propagating activation differences. Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia."},{"key":"ref_60","unstructured":"Krug, A., Knaebel, R., and Stober, S. (2018, January 8). Neuron Activation Profiles for Interpreting Convolutional Speech Recognition Models. Proceedings of the NeurIPS Workshop IRASL: Interpretability and Robustness for Audio, Speech and Language, Montreal, QC, Canada."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Krug, A., Ebrahimzadeh, M., Alemann, J., Johannsmeier, J., and Stober, S. (2021). Analyzing and visualizing deep neural networks for speech recognition with saliency-adjusted neuron activation profiles. Electronics, 10.","DOI":"10.3390\/electronics10111350"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Krug, A., Ratul, R.K., and Stober, S. (2022). Visualizing Deep Neural Networks with Topographic Activation Maps. arXiv.","DOI":"10.3233\/FAIA230080"},{"key":"ref_63","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_64","unstructured":"Kokhlikyan, N., Miglani, V., Martin, M., Wang, E., Alsallakh, B., Reynolds, J., Melnikov, A., Kliushkina, N., Araya, C., and Yan, S. (2020). Captum: A unified and generic model interpretability library for PyTorch. arXiv."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Chatterjee, S., Das, A., Mandal, C., Mukhopadhyay, B., Vipinraj, M., Shukla, A., Nagaraja Rao, R., Sarasaen, C., Speck, O., and N\u00fcrnberger, A. (2022). TorchEsegeta: Framework for Interpretability and Explainability of Image-based Deep Learning Models. Appl. Sci., 12.","DOI":"10.20944\/preprints202201.0072.v1"},{"key":"ref_66","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_67","unstructured":"(2024, January 24). PyTorch Reproducibility. Available online: https:\/\/pytorch.org\/docs\/stable\/notes\/randomness.html."},{"key":"ref_68","unstructured":"(2024, January 24). Nvidia Apex. Available online: https:\/\/github.com\/NVIDIA\/apex."},{"key":"ref_69","unstructured":"(2024, January 24). COVID-19 Image Data Collection. Available online: https:\/\/github.com\/ieee8023\/covid-chestxray-dataset."},{"key":"ref_70","first-page":"651","article-title":"Labeled optical coherence tomography (oct) and chest X-ray images for classification","volume":"2","author":"Kermany","year":"2018","journal-title":"Mendeley Data"},{"key":"ref_71","unstructured":"(2024, January 24). Chest X-ray Images (Pneumonia). Available online: https:\/\/www.kaggle.com\/paultimothymooney\/chest-xray-pneumonia."},{"key":"ref_72","unstructured":"(2024, January 24). Radiopaedia: Chest Radiograph. Available online: https:\/\/radiopaedia.org\/articles\/chest-radiograph?lang=us."},{"key":"ref_73","unstructured":"Diamond, M., Peniston, H.L., Sanghavi, D., Mahapatra, S., and Doerr, C. (2021). Acute Respiratory Distress Syndrome (Nursing), StatPearls Publishing."},{"key":"ref_74","first-page":"1","article-title":"Acute respiratory distress syndrome","volume":"5","author":"Matthay","year":"2019","journal-title":"Nat. Rev. Dis. Prim."},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1016\/S2213-2600(20)30304-0","article-title":"COVID-19-associated acute respiratory distress syndrome: Is a different approach to management warranted?","volume":"8","author":"Fan","year":"2020","journal-title":"Lancet Respir. Med."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1186\/s13054-020-02880-z","article-title":"COVID-19 pneumonia: ARDS or not?","volume":"24","author":"Gattinoni","year":"2020","journal-title":"Crit. Care"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1513\/AnnalsATS.202008-1026OC","article-title":"COVID-19 versus non\u2013COVID-19 acute respiratory distress syndrome: Comparison of demographics, physiologic parameters, inflammatory biomarkers, and clinical outcomes","volume":"18","author":"Bain","year":"2021","journal-title":"Ann. Am. Thorac. Soc."},{"key":"ref_78","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/jdwm.2007070101","article-title":"Multi-label classification: An overview","volume":"3","author":"Tsoumakas","year":"2007","journal-title":"Int. J. Data Warehous. Min. IJDWM"},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.neucom.2014.08.091","article-title":"Addressing imbalance in multilabel classification: Measures and random resampling algorithms","volume":"163","author":"Charte","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Denise, E., Morris, D.W.C., and Clarke, S.C. (2017). Secondary Bacterial Infections Associated with Influenza Pandemics. Front. Microbiol., 8.","DOI":"10.3389\/fmicb.2017.01041"},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"2640","DOI":"10.3389\/fimmu.2018.02640","article-title":"Respiratory Viral Infection-Induced Microbiome Alterations and Secondary Bacterial Pneumonia","volume":"9","author":"Hanada","year":"2018","journal-title":"Front. Immunol."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy should I trust you?\u201d Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"},{"key":"ref_83","unstructured":"Lundberg, S.M., and Lee, S.I. (2017). Advances in Neural Information Processing Systems, Springer."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/2\/45\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:57:36Z","timestamp":1760104656000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/10\/2\/45"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,8]]},"references-count":83,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["jimaging10020045"],"URL":"https:\/\/doi.org\/10.3390\/jimaging10020045","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,8]]}}}