{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T09:48:24Z","timestamp":1771840104048,"version":"3.50.1"},"reference-count":24,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Scientific Grant Agency of the Ministry of Education, Science, Research, and Sport of the Slovak Republic","award":["1\/0685\/21"],"award-info":[{"award-number":["1\/0685\/21"]}]},{"name":"Scientific Grant Agency of the Ministry of Education, Science, Research, and Sport of the Slovak Republic","award":["APVV-20-0232"],"award-info":[{"award-number":["APVV-20-0232"]}]},{"name":"Scientific Grant Agency of the Ministry of Education, Science, Research, and Sport of the Slovak Republic","award":["APVV-16-0213"],"award-info":[{"award-number":["APVV-16-0213"]}]},{"name":"Slovak Research and Development Agency","award":["1\/0685\/21"],"award-info":[{"award-number":["1\/0685\/21"]}]},{"name":"Slovak Research and Development Agency","award":["APVV-20-0232"],"award-info":[{"award-number":["APVV-20-0232"]}]},{"name":"Slovak Research and Development Agency","award":["APVV-16-0213"],"award-info":[{"award-number":["APVV-16-0213"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Machine learning (ML) has been used in different ways in the fight against COVID-19 disease. ML models have been developed, e.g., for diagnostic or prognostic purposes and using various modalities of data (e.g., textual, visual, or structured). Due to the many specific aspects of this disease and its evolution over time, there is still not enough understanding of all relevant factors influencing the course of COVID-19 in particular patients. In all aspects of our work, there was a strong involvement of a medical expert following the human-in-the-loop principle. This is a very important but usually neglected part of the ML and knowledge extraction (KE) process. Our research shows that explainable artificial intelligence (XAI) may significantly support this part of ML and KE. Our research focused on using ML for knowledge extraction in two specific scenarios. In the first scenario, we aimed to discover whether adding information about the predominant COVID-19 variant impacts the performance of the ML models. In the second scenario, we focused on prognostic classification models concerning the need for an intensive care unit for a given patient in connection with different explainability AI (XAI) methods. We have used nine ML algorithms, namely XGBoost, CatBoost, LightGBM, logistic regression, Naive Bayes, random forest, SGD, SVM-linear, and SVM-RBF. We measured the performance of the resulting models using precision, accuracy, and AUC metrics. Subsequently, we focused on knowledge extraction from the best-performing models using two different approaches as follows: (a) features extracted automatically by forward stepwise selection (FSS); (b) attributes and their interactions discovered by model explainability methods. Both were compared with the attributes selected by the medical experts in advance based on the domain expertise. Our experiments showed that adding information about the COVID-19 variant did not influence the performance of the resulting ML models. It also turned out that medical experts were much more precise in the identification of significant attributes than FSS. Explainability methods identified almost the same attributes as a medical expert and interesting interactions among them, which the expert discussed from a medical point of view. The results of our research and their consequences are discussed.<\/jats:p>","DOI":"10.3390\/make5040064","type":"journal-article","created":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T08:58:17Z","timestamp":1695718697000},"page":"1266-1281","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Unraveling COVID-19 Dynamics via Machine Learning and XAI: Investigating Variant Influence and Prognostic Classification"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0060-6972","authenticated-orcid":false,"given":"Oliver","family":"Lohaj","sequence":"first","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 040 01 Ko\u0161ice, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4603-0411","authenticated-orcid":false,"given":"J\u00e1n","family":"Parali\u010d","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 040 01 Ko\u0161ice, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Bedn\u00e1r","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 040 01 Ko\u0161ice, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zuzana","family":"Parali\u010dov\u00e1","sequence":"additional","affiliation":[{"name":"Department of Infectology and Travel Medicine, Faculty of Medicine, L. Pasteur University Hospital, Pavol Jozef \u0160af\u00e1rik University, 041 90 Ko\u0161ice, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mat\u00fa\u0161","family":"Huba","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Artificial Intelligence, Faculty of Electrical Engineering and Informatics, Technical University of Kosice, Letna 9, 040 01 Ko\u0161ice, Slovakia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,25]]},"reference":[{"key":"ref_1","unstructured":"Cascella, M., Rajnik, M., Aleem, A., Dulebohn, S.C., and Di Napoli, R. (2023, January 21). Features, Evaluation, and Treatment of Coronavirus (COVID-19), Available online: https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK554776\/."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"18716","DOI":"10.1038\/s41598-020-75767-2","article-title":"Machine learning prediction for mortality of patients diagnosed with COVID-19: A nationwide Korean cohort study","volume":"10","author":"An","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5097","DOI":"10.1038\/s41467-020-18926-3","article-title":"A population-based cohort study of socio-demographic risk factors for COVID-19 deaths in Sweden","volume":"11","author":"Drefahl","year":"2020","journal-title":"Nat. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1080\/07853890.2020.1868564","article-title":"Clinical and inflammatory features based machine learning model for fatal risk prediction of hospitalized COVID-19 patients: Results from a retrospective cohort study","volume":"53","author":"Guan","year":"2021","journal-title":"Ann. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"e29544","DOI":"10.2196\/29544","article-title":"Uncovering Clinical Risk Factors and Predicting Severe COVID-19 Cases Using UK Biobank Data: Machine Learning Approach","volume":"7","author":"Wong","year":"2021","journal-title":"JMIR Public Health Surveill."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Krajah, A., Almadani, Y.F., Saadeh, H., and Sleit, A. (2021, January 16\u201318). Analyzing COVID-19 Data Using Various Algorithms. Proceedings of the 2021 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT), Amman, Jordan.","DOI":"10.1109\/JEEIT53412.2021.9634124"},{"key":"ref_7","unstructured":"Mukherjee, T. (2023, March 01). COVID-19 Patient Pre-Condition Dataset. Available online: https:\/\/Kaggle.com."},{"key":"ref_8","unstructured":"Fransiska, A., Holy, C., and Prima Rosa, P.H. (2021, January 18\u201320). Classification of COVID-19 Patients Requiring Intensive Care Unit. Proceedings of the 25th International Computer Science and Engineering Conference, Chiang Rai, Thailand."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1451","DOI":"10.1038\/s41418-020-0530-3","article-title":"COVID-19 infection: The perspectives on immune responses","volume":"27","author":"Shi","year":"2020","journal-title":"Cell Death Differ."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1016\/S0140-6736(20)30566-3","article-title":"Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study","volume":"395","author":"Zhou","year":"2020","journal-title":"Lancet"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Majnari\u0107, L.T., Babi\u010d, F., O\u2019Sullivan, S., and Holzinger, A. (2021). AI and Big Data in Healthcare: Towards a More Comprehensive Research Framework for Multimorbidity. J. Clin. Med., 10.","DOI":"10.3390\/jcm10040766"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1962","DOI":"10.1093\/cid\/ciaa674","article-title":"Predictors for Severe COVID-19 Infection","volume":"71","author":"Bhargava","year":"2020","journal-title":"Clin. Infect. Dis."},{"key":"ref_13","first-page":"615","article-title":"Gastrointestinal predictors of severe COVID-19: Systematic review and meta-analysis","volume":"33","author":"Aziz","year":"2020","journal-title":"Ann. Gastroenterol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"104204","DOI":"10.1016\/j.archger.2020.104204","article-title":"Clinical course and prognostic factors of COVID-19 infection in an elderly hospitalized population","volume":"91","author":"Mostaza","year":"2020","journal-title":"Arch. Gerontol. Geriatr."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"108293","DOI":"10.1016\/j.diabres.2020.108293","article-title":"Risk factors for mortality among COVID-19 patients","volume":"166","author":"Albitar","year":"2020","journal-title":"Diabetes Res. Clin. Pr."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2406","DOI":"10.1038\/s41591-022-02001-z","article-title":"Long-term neurologic outcomes of COVID-19","volume":"28","author":"Xu","year":"2022","journal-title":"Nat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1016\/j.procs.2021.01.199","article-title":"A Systematic Literature Review on Applying CRISP-DM Process Model","volume":"181","author":"Kruse","year":"2021","journal-title":"Procedia Comput. Sci."},{"key":"ref_18","first-page":"9226","article-title":"Artificial intelligence technology for diagnosing COVID-19 cases: A review of substantial issues","volume":"24","author":"Alsharif","year":"2020","journal-title":"Eur. Rev. Med. Pharmacol. Sci."},{"key":"ref_19","first-page":"11455","article-title":"Deep learning applications to combat the dissemination of COVID-19 disease: A review","volume":"24","author":"Alsharif","year":"2020","journal-title":"Eur. Rev. Med. Pharmacol. Sci."},{"key":"ref_20","unstructured":"Gobierno de Mexico (2023, March 01). Datos Abiertos. Available online: https:\/\/www.gob.mx\/salud\/documentos\/datos-abiertos-152127."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Swana, E.F., Doorsamy, W., and Bokoro, P. (2022). Tomek Link and SMOTE Approaches for Machine Fault Classification with an Imbalanced Dataset. Sensors, 22.","DOI":"10.3390\/s22093246"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"560685","DOI":"10.3389\/fmed.2020.560685","article-title":"COVID-19-Related Fatalities and Intensive-Care-Unit Admissions by Age Groups in Europe: A Meta-Analysis","volume":"7","author":"Cohen","year":"2021","journal-title":"Front. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1186\/s13613-023-01127-8","article-title":"Risk factors for severe COVID-19 in the young\u2014Before and after ICU admission","volume":"13","author":"Bohlin","year":"2023","journal-title":"Ann. Intensiv. Care"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1016\/j.clinthera.2022.01.007","article-title":"Early Use of Remdesivir and Risk of Disease Progression in Hospitalized Patients with Mild to Moderate COVID-19","volume":"44","author":"Falcone","year":"2022","journal-title":"Clin. Ther."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/5\/4\/64\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:57:57Z","timestamp":1760129877000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/5\/4\/64"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":24,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["make5040064"],"URL":"https:\/\/doi.org\/10.3390\/make5040064","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,25]]}}}