{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T14:17:24Z","timestamp":1781273844897,"version":"3.54.1"},"reference-count":66,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,11]],"date-time":"2021-01-11T00:00:00Z","timestamp":1610323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"by King Abdulaziz City for Science &amp; Technology (KACST) through the Fast Track Funding Path for Coronavirus (COVID-19)","award":["5-20-01-004-0022"],"award-info":[{"award-number":["5-20-01-004-0022"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The recent Coronavirus Disease 2019 (COVID-19) pandemic has put a tremendous burden on global health systems. Medical practitioners are under great pressure for reliable screening of suspected cases employing adjunct diagnostic tools to standard point-of-care testing methodology. Chest X-rays (CXRs) are appearing as a prospective diagnostic tool with easy-to-acquire, low-cost and less cross-contamination risk features. Artificial intelligence (AI)-attributed CXR evaluation has shown great potential for distinguishing COVID-19-induced pneumonia from other associated clinical instances. However, one of the associated challenges with diagnostic imaging-based modeling is incorrect feature attribution, which leads the model to learn misguiding disease patterns, causing wrong predictions. Here, we demonstrate an effective deep learning-based methodology to mitigate the problem, thereby allowing the classification algorithm to learn from relevant features. The proposed deep-learning framework consists of an ensemble of convolutional neural network (CNN) models focusing on both global and local pathological features from CXR lung images, while the latter is extracted using a multi-instance learning scheme and a local attention mechanism. An inspection of a series of backbone CNN models using global and local features, and an ensemble of both features, trained from high-quality CXR images of 1311 patients, further augmented for achieving the symmetry in class distribution, to localize lung pathological features followed by the classification of COVID-19 and other related pneumonia, shows that a DenseNet161 architecture outperforms all other models, as evaluated on an independent test set of 159 patients with confirmed cases. Specifically, an ensemble of DenseNet161 models with global and local attention-based features achieve an average balanced accuracy of 91.2%, average precision of 92.4%, and F1-score of 91.9% in a multi-label classification framework comprising COVID-19, pneumonia, and control classes. The DenseNet161 ensembles were also found to be statistically significant from all other models in a comprehensive statistical analysis. The current study demonstrated that the proposed deep learning-based algorithm can accurately identify the COVID-19-related pneumonia in CXR images, along with differentiating non-COVID-19-associated pneumonia with high specificity, by effectively alleviating the incorrect feature attribution problem, and exploiting an enhanced feature descriptor.<\/jats:p>","DOI":"10.3390\/sym13010113","type":"journal-article","created":{"date-parts":[[2021,1,11]],"date-time":"2021-01-11T20:32:47Z","timestamp":1610397167000},"page":"113","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":39,"title":["An Ensemble of Global and Local-Attention Based Convolutional Neural Networks for COVID-19 Diagnosis on Chest X-ray Images"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8434-8578","authenticated-orcid":false,"given":"Ahmed","family":"Afifi","sequence":"first","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia"},{"name":"Faculty of Computers and Information, Menoufia University, Menoufia 32511, Egypt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8288-1782","authenticated-orcid":false,"given":"Noor E","family":"Hafsa","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2192-3504","authenticated-orcid":false,"given":"Mona A. S.","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdulaziz","family":"Alhumam","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Safa","family":"Alsalman","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information Technology, King Faisal University, P.O. Box 400, Al-Ahsa 31982, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,11]]},"reference":[{"key":"ref_1","unstructured":"(2020, June 19). WHO Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19). Available online: https:\/\/www.who.int\/publications-detail\/report-of-the-who-china-joint-mission-on-coronavirus-disease-2019-(covid-19)."},{"key":"ref_2","unstructured":"(2020, June 19). Coronavirus Update (Live), Worldometer. Available online: https:\/\/www.worldometers.info\/coronavirus\/."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1038\/s41591-020-0869-5","article-title":"Temporal dynamics in viral shedding and transmissibility of COVID-19","volume":"26","author":"He","year":"2020","journal-title":"Nat. Med."},{"key":"ref_4","unstructured":"(2020, June 20). COVID-19 Radiology Reference Article, Radiopaedia. Available online: https:\/\/radiopaedia.org\/articles\/covid-19-3."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"E32","DOI":"10.1148\/radiol.2020200642","article-title":"Correlation of Chest CT and RT-PCR Testing for Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases","volume":"296","author":"Ai","year":"2020","journal-title":"Radiology"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"E115","DOI":"10.1148\/radiol.2020200432","article-title":"Sensitivity of chest CT for COVID-19: Comparison to RT-PCR","volume":"296","author":"Fang","year":"2020","journal-title":"Radiology"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107","DOI":"10.5607\/en20009","article-title":"Development of a laboratory-safe and low-cost detection protocol for SARS-CoV-2 of the Coronavirus Disease 2019 (COVID-19)","volume":"29","author":"Won","year":"2020","journal-title":"Exp. Neurobiol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1148\/radiol.2020200241","article-title":"Chest CT findings in 2019 novel coronavirus (2019-NCoV) infections from Wuhan, China: Key points for the radiologist","volume":"295","author":"Kanne","year":"2020","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1148\/radiol.2020200463","article-title":"Chest CT findings in coronavirus disease 2019 (COVID-19): Relationship to duration of infection","volume":"295","author":"Bernheim","year":"2020","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"E41","DOI":"10.1148\/radiol.2020200343","article-title":"Chest CT for Typical Coronavirus Disease 2019 (COVID-19) Pneumonia: Relationship to Negative RT-PCR Testing","volume":"296","author":"Xie","year":"2020","journal-title":"Radiology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1148\/radiol.2020201365","article-title":"The Role of Chest Imaging in Patient Management during the Covid-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society","volume":"296","author":"Rubin","year":"2020","journal-title":"Radiology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1016\/j.crad.2020.03.003","article-title":"An update on COVID-19 for the radiologist\u2014A British society of Thoracic Imaging statement","volume":"75","author":"Rodrigues","year":"2020","journal-title":"Clin. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1056\/NEJMe2001126","article-title":"Another Decade, another Coronavirus","volume":"382","author":"Perlman","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"E113","DOI":"10.1148\/radiol.2020200527","article-title":"Essentials for Radiologists on COVID-19: An Update-Radiology Scientific Expert Panel","volume":"296","author":"Kanne","year":"2020","journal-title":"Radiology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"839","DOI":"10.2214\/AJR.20.23202","article-title":"Chest CT and Coronavirus Disease (COVID-19): A Critical Review of the Literature to Date","volume":"215","author":"Raptis","year":"2020","journal-title":"Am. J. Roentgenol. Roentgenol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.clinimag.2020.04.001","article-title":"Portable Chest X-Ray in Coronavirus Disease-19 (COVID-19): A Pictorial Review","volume":"64","author":"Jacobi","year":"2020","journal-title":"Clin. Imaging"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L., Li, K., and Li, F. (2009, January 20\u201325). ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, X., Yang, X., Zhang, S., Zhao, J., Zhang, Y., Xing, E., and Xie, P. (2020). Sample-Efficient Deep Learning for COVID-19 Diagnosis Based on CT Scans. medRxiv.","DOI":"10.1101\/2020.04.13.20063941"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Pham, H.H., Le, T.T., Ngo, D.T., Tran, D.Q., and Nguyen, H.Q. (2020). Interpreting Chest X-Rays Via CNNS That Exploit Hierarchical Disease Dependencies and Uncertainty Labels. arXiv.","DOI":"10.1101\/19013342"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., and Shpanskaya, K. (February, January 27). CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison. Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","DOI":"10.1609\/aaai.v33i01.3301590"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gabruseva, T., Poplavskiy, D., and Kalinin, A. (2020, January 14\u201319). Deep Learning for Automatic Pneumonia Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00183"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Bansal, N. (2020). Classification of X-ray Images for Detecting Covid-19 Using Deep Transfer Learning. Res. Square.","DOI":"10.21203\/rs.3.rs-32247\/v1"},{"key":"ref_23","unstructured":"Zech, J. (2020, June 19). Reproduce-Chexnet. Available online: https:\/\/github.com\/jrzech\/reproduce-chexnet."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bassi, P.R., and Attux, R. (2020). A Deep Convolutional Neural Network for COVID-19 Detection Using Chest X-Rays. arXiv.","DOI":"10.1007\/s42600-021-00132-9"},{"key":"ref_25","first-page":"117","article-title":"Deep Transfer Learning with Apache Spark to Detect COVID-19 in Chest X-ray Images","volume":"23","author":"Benbrahim","year":"2020","journal-title":"Rom. J. Inf. Sci. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/2934664","article-title":"Apache Spark: A Unified Engine for Big Data Processing","volume":"59","author":"Zaharia","year":"2016","journal-title":"Commun. ACM"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"132665","DOI":"10.1109\/ACCESS.2020.3010287","article-title":"Can AI Help in Screening Viral and COVID-19 Pneumonia?","volume":"8","author":"Chowdhury","year":"2020","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"De Moura, J., Novo, J., and Ortega, M. (2020). Fully Automatic Deep Convolutional Approaches for The Analysis of Covid-19 Using Chest X-Ray Images. medRxiv.","DOI":"10.1101\/2020.05.01.20087254"},{"key":"ref_29","unstructured":"Ghoshal, B., and Tucker, A. (2020). Estimating Uncertainty and Interpretability in Deep Learning for Coronavirus (COVID-19) Detection. arXiv."},{"key":"ref_30","unstructured":"Chatterjee, S., Saad, F., Sarasaen, C., Ghosh, S., Khatun, R., Radeva, P., Rose, G., Stober, S., Speck, O., and N\u00fcrnberger, A. (2020). Exploration of Interpretability Techniques for Deep Covid-19 Classification Using Chest X-Ray Images. arXiv."},{"key":"ref_31","unstructured":"Hussain, S., Khan, A., and Zafar, M.M. (2020). Coronavirus Disease Analysis using Chest X-ray Images and a Novel Deep Convolutional Neural Network. Res. Gate."},{"key":"ref_32","unstructured":"Lv, D., Qi, W., Li, Y., Sun, L., and Wang, Y. (2020). A Cascade Network for Detecting Covid-19 Using Chest X-rays. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Narin, A., Kaya, C., and Pamuk, Z. (2020). Automatic Detection of Coronavirus Disease (COVID-19) Using X-ray Images and Deep Convolutional Neural Networks. arXiv.","DOI":"10.1007\/s10044-021-00984-y"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2688","DOI":"10.1109\/TMI.2020.2993291","article-title":"Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets","volume":"39","author":"Oh","year":"2020","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"115041","DOI":"10.1109\/ACCESS.2020.3003810","article-title":"Iteratively Pruned Deep Learning Ensembles for COVID-19 Detection in Chest X-Rays","volume":"8","author":"Rajaraman","year":"2020","journal-title":"IEEE Access"},{"key":"ref_36","unstructured":"Ramadhan, M.M., Faza, A., Lubis, L.E., Yunus, R.E., Salamah, T., Handayani, D., Lestariningsih, I., Resa, A., Alam, C.R., and Prajitno, P. (2020). Fast and Accurate Detection of Covid-19-Related Pneumonia from Chest X-Ray Images with Novel Deep Learning Model. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Duchesne, S., Gourdeau, D., Archambault, P., Chartrand-Lefebvre, C., Dieumegarde, L., Forghani, R., Gagn\u00e9, C., Hains, A., Hornstein, D., and Le, H. (2020). Tracking and Predicting Covid-19 Radiological Trajectory Using Deep Learning on Chest X-rays: Initial Accuracy Testing. medRxiv.","DOI":"10.1101\/2020.05.01.20086207"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"109761","DOI":"10.1016\/j.mehy.2020.109761","article-title":"COVIDiagnosis-Net: Deep Bayes-Squeezenet Based Diagnosis of the Coronavirus Disease 2019 (COVID-19) from X-ray Images","volume":"140","author":"Ucar","year":"2020","journal-title":"Med. Hypotheses"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ezzat, D., ell Hassanien, A., and Ella, H.A. (2020). GSA-DenseNet121-COVID-19: A Hybrid Deep Learning Architecture for the Diagnosis of COVID-19 Disease based on Gravitational Search Optimization Algorithm. arXiv.","DOI":"10.1016\/j.asoc.2020.106742"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rajaraman, S., and Antani, S. (2020). Weakly Labeled Data Augmentation for Deep Learning: A Study on COVID-19 Detection in Chest X-rays. Diagnostics, 10.","DOI":"10.3390\/diagnostics10060358"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"19549","DOI":"10.1038\/s41598-020-76550-z","article-title":"COVID-Net: A Tailored Deep Convolutional Neural Network Design for Detection of COVID-19 Cases from Chest X-ray Images","volume":"10","author":"Wang","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"15364","DOI":"10.1038\/s41598-020-71294-2","article-title":"COVID-19 Image Classification Using Deep Features and Fractional-Order Marine Predators Algorithm","volume":"10","author":"Sahlol","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_43","unstructured":"De la Vay\u00e1, M.I., Saborit, J.M., Montell, J.A., Pertusa, A., Bustos, A., Cazorla, M., Galant, J., Barber, X., Orozco-Beltr\u00e1n, D., and Garc\u00eda-Garc\u00eda, F. (2020). BIMCV COVID-19+: A Large Annotated Dataset of RX and CT Images from COVID-19 Patients. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bustos, A., Pertusa, A., Salinas, J.M., and de la Iglesia-Vay\u00e1, M. (2019). Padchest: A Large Chest X-ray Image Dataset with Multi-Label Annotated Reports. arXiv.","DOI":"10.1016\/j.media.2020.101797"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"De Grave, A.J., Janizek, J.D., and Lee, S. (2020). AI for radiographic COVID-19 detection selects shortcuts over signal. medRxiv.","DOI":"10.1101\/2020.09.13.20193565"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Goyal, P., Girshick, R., and He, K. (2017, January 22\u201329). Dollar, Focal Loss for Dense Object Detection. Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.324"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature Pyramid Networks for Object Detection. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Comelli, A., Coronnello, C., Dahiya, N., Benfante, V., Palmucci, S., Basile, A., Vancheri, C., Russo, G., Yezzi, A., and Stefano, A. (2020). Lung Segmentation on High-Resolution Computerized Tomography Images Using Deep Learning: A Preliminary Step for Radiomics Studies. J. Imaging, 6.","DOI":"10.3390\/jimaging6110125"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Comelli, A., Dahiya, N., Stefano, A., Benfante, V., Gentile, G., Agnese, V., Raffa, G.M., Pilato, M., Yezzi, A., and Petrucci, G. (2020). Deep Learning Approach for the Segmentation of Aneurysmal Ascending Aorta. Biomed. Eng. Lett., 1\u201310.","DOI":"10.1007\/s13534-020-00179-0"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated Residual Transformations for Deep Neural Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_52","unstructured":"(2020, November 01). COVID-19 Xray Dataset [Dataset]. Available online: https:\/\/github.com\/v7labs\/covid-19-xray-dataset."},{"key":"ref_53","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s43055-020-00296-x","article-title":"Chest X-Ray Findings Monitoring COVID-19 Disease Course and Severity","volume":"51","author":"Yasin","year":"2020","journal-title":"Egypt. J. Radiol. Nucl. Med."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A.A. (2017, January 4\u20139). Inception-v4, Inception-Resnet and the Impact of Residual Connections on Learning. Proceedings of the 31st AAAI Conference on Artificial Intelligence.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Li, Z., Wang, C., Han, M., Xue, Y., Wei, W., Li, L.J., and Fei, L. (2018, January 18\u201323). Thoracic Disease Identification and Localization with Limited Supervision. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00865"},{"key":"ref_58","unstructured":"Ilse, M., Tomczak, J.M., and Welling, M. (2018, January 10\u201315). Attention-Based Deep Multiple Instance Learning. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_59","first-page":"1","article-title":"Statistical Comparisons of Classifiers over Multiple Data Sets","volume":"7","author":"Janez","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_60","unstructured":"Nemenyi, P. (1963). Distribution-Free Multiple Comparisons. [Ph.D. Thesis, Princeton University]."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"2173","DOI":"10.21105\/joss.02173","article-title":"Autorank: A Python package for automated ranking of classifiers","volume":"5","author":"Herbold","year":"2020","journal-title":"J. Open Source Softw."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., and Fergus, R. (2014). Visualizing and Understanding Convolutional Networks. Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_63","unstructured":"(2020, November 01). Captum-Model Interpretability for PyTorch [Software Framework]. Available online: https:\/\/captum.ai\/."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Ahsan, M.M., Alam, T.E., and Trafalis, T. (2020). Huebner, Deep MLP-CNN Model Using Mixed-Data to Distinguish between COVID-19 and Non-COVID-19 Patients. Symmetry, 12.","DOI":"10.3390\/sym12091526"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"105581","DOI":"10.1016\/j.cmpb.2020.105581","article-title":"CoroNet: A Deep Neural Network for Detection and Diagnosis of COVID-19 from Chest X-ray Images","volume":"196","author":"Khan","year":"2020","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Lovric, M. (2011). Wilcoxon-Signed-Rank Test. International Encyclopedia of Statistical Science, Springer.","DOI":"10.1007\/978-3-642-04898-2"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/1\/113\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:09:44Z","timestamp":1760159384000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/1\/113"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,11]]},"references-count":66,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["sym13010113"],"URL":"https:\/\/doi.org\/10.3390\/sym13010113","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,11]]}}}