{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T12:28:52Z","timestamp":1775219332994,"version":"3.50.1"},"reference-count":67,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Linz Institute of Technology","award":["LIT-2018-6-YOU-212"],"award-info":[{"award-number":["LIT-2018-6-YOU-212"]}]},{"name":"State of Upper Austria","award":["Wi-2018-439843"],"award-info":[{"award-number":["Wi-2018-439843"]}]},{"name":"Johannes Kepler University Linz"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Med Syst"],"published-print":{"date-parts":[[2022,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Many previous studies claim to have developed machine learning models that diagnose COVID-19 from blood tests. However, we hypothesize that changes in the underlying distribution of the data, so called domain shifts, affect the predictive performance and reliability and are a reason for the failure of such machine learning models in clinical application. Domain shifts can be caused, e.g., by changes in the disease prevalence (spreading or tested population), by refined RT-PCR testing procedures (way of taking samples, laboratory procedures), or by virus mutations. Therefore, machine learning models for diagnosing COVID-19 or other diseases may not be reliable and degrade in performance over time. We investigate whether domain shifts are present in COVID-19 datasets and how they affect machine learning methods. We further set out to estimate the mortality risk based on routinely acquired blood tests in a hospital setting throughout pandemics and under domain shifts. We reveal domain shifts by evaluating the models on a large-scale dataset with different assessment strategies, such as temporal validation. We present the novel finding that domain shifts strongly affect machine learning models for COVID-19 diagnosis and deteriorate their predictive performance and credibility. Therefore, frequent re-training and re-assessment are indispensable for robust models enabling clinical utility.<\/jats:p>","DOI":"10.1007\/s10916-022-01807-1","type":"journal-article","created":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T02:14:26Z","timestamp":1648520066000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Domain Shifts in Machine Learning Based Covid-19 Diagnosis From Blood Tests"],"prefix":"10.1007","volume":"46","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7893-6146","authenticated-orcid":false,"given":"Theresa","family":"Roland","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2207-2411","authenticated-orcid":false,"given":"Carl","family":"B\u00f6ck","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3350-7532","authenticated-orcid":false,"given":"Thomas","family":"Tschoellitsch","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4378-7854","authenticated-orcid":false,"given":"Alexander","family":"Maletzky","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7449-2528","authenticated-orcid":false,"given":"Sepp","family":"Hochreiter","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8727-2257","authenticated-orcid":false,"given":"Jens","family":"Meier","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2861-5552","authenticated-orcid":false,"given":"G\u00fcnter","family":"Klambauer","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"1807_CR1","doi-asserted-by":"publisher","first-page":"2000045","DOI":"10.2807\/1560-7917.ES.2020.25.3.2000045","volume":"25","author":"VM Corman","year":"2020","unstructured":"V. M. Corman, O. Landt, M. Kaiser, R. Molenkamp, A. Meijer, D. K. Chu, T. Bleicker, S. Br\u00fcnink, J. Schneider, M. L. Schmidt, D. G. Mulders, B. L. Haagmans, B. Veer, S. Brink, L. Wijsman, G. Goderski, J. L. Romette, J. Ellis, M. Zambon, M. Peiris, H. Goossens, C. Reusken, M. P. G. Koopmans and C. Drosten, \u201eDetection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR\u201c, Euro Surveill., 25, p. 2000045, 2020. https:\/\/doi.org\/10.2807\/1560-7917.ES.2020.25.3.2000045","journal-title":"Euro Surveill."},{"key":"1807_CR2","doi-asserted-by":"publisher","first-page":"727","DOI":"10.1056\/NEJMoa2001017","volume":"382","author":"N Zhu","year":"2020","unstructured":"N. Zhu, D. Zhang, W. Wang, X. Li, B. Yang, J. Song, X. Zhao, B. Huang, W. Shi, R. Lu, P. Niu, F. Zhan, X. Ma, D. Wang, W. Xu, G. Wu, G. F. Gao and W. Tan, \u201eA Novel Coronavirus from Patients with Pneumonia in China, 2019\u201c, N. Engl. J. Med., 382, pp. 727-733, 2020. https:\/\/doi.org\/10.1056\/NEJMoa2001017","journal-title":"N. Engl. J. Med."},{"key":"1807_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcv.2020.104500","volume":"129","author":"GC Mak","year":"2020","unstructured":"G. C. Mak, P. K. Cheng, S. S. Lau, K. K. Wong, C. S. Lau, E. T. Lam, R. C. Chan and D. N. Tsang, \u201eEvaluation of rapid antigen test for detection of SARS-CoV-2 virus\u201c, J. Clin. Virol., 129, p. 104500, 2020. https:\/\/doi.org\/10.1016\/j.jcv.2020.104500","journal-title":"J. Clin. Virol."},{"key":"1807_CR4","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMp2025631","volume":"383","author":"MJ Mina","year":"2020","unstructured":"M. J. Mina, R. Parker and D. B. Larremore, \u201eRethinking Covid-19 Test Sensitivity \u2014 A Strategy for Containment\u201c, N. Engl. J. Med., 383, p. e120, 2020. https:\/\/doi.org\/10.1056\/NEJMp2025631","journal-title":"N. Engl. J. Med."},{"key":"1807_CR5","doi-asserted-by":"publisher","unstructured":"E. T. Chin, B. Q. Huynh, L. A. C. Chapman, M. Murrill, S. Basu and N. C. Lo, \u201eFrequency of Routine Testing for Coronavirus Disease 2019 (COVID-19) in High-risk Healthcare Environments to Reduce Outbreaks\u201c, Clin. Infect. Dis., p. ciaa1383, 2020. https:\/\/doi.org\/10.1093\/cid\/ciaa1383","DOI":"10.1093\/cid\/ciaa1383"},{"key":"1807_CR6","doi-asserted-by":"publisher","unstructured":"D. B. Larremore, B. Wilder, E. Lester, S. Shehata, J. M. Burke, J. A. Hay, M. Tambe, M. J. Mina and R. Parker, \u201eTest sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance\u201c, Sci. Adv. 7, 2020. https:\/\/doi.org\/10.1126\/sciadv.abd5393","DOI":"10.1126\/sciadv.abd5393"},{"key":"1807_CR7","doi-asserted-by":"publisher","unstructured":"M. van der Schaar, A. M. Alaa, A. Floto, A. Gimson, S. Scholtes, A. Wood, E. McKinney, D. Jarrett, P. Lio A. Ercole, \u201cHow artificial intelligence and machine learning can help healthcare systems respond to COVID-19\u201d, Mach. Learn., 110, 1\u201314, 2021. https:\/\/doi.org\/10.1007\/s10994-020-05928-x","DOI":"10.1007\/s10994-020-05928-x"},{"key":"1807_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/ijerph18031117","volume":"18","author":"T Alafif","year":"2021","unstructured":"T. Alafif, A. M. Tehame, S. Bajaba, A. Barnawi and S. Zia, \u201eMachine and Deep Learning towards COVID-19 Diagnosis and Treatment: Survey, Challenges, and Future Directions\u201c, Int. J. Environ. Res. Public Health, 18, pp. 1-24, 2021. https:\/\/doi.org\/10.3390\/ijerph18031117","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"1807_CR9","doi-asserted-by":"publisher","unstructured":"A. S. Adly, A. S. Adly and M. S. Adly, \u201eApproaches Based on Artificial Intelligence and the Internet of Intelligent Things to Prevent the Spread of COVID-19: Scoping Review\u201c, J. Med. Internet Res., 22, 8, p. e19104, 8 2020. https:\/\/doi.org\/10.2196\/19104","DOI":"10.2196\/19104"},{"key":"1807_CR10","doi-asserted-by":"publisher","first-page":"40","DOI":"10.3390\/asi4030040","volume":"4","author":"A Majeed","year":"2021","unstructured":"A. Majeed, and S. Lee, \u201cApplications of Machine Learning and High- Performance Computing in the Era of COVID-19\u201d, Appl. Syst. Innov., 4, 40, 2021, https:\/\/doi.org\/10.3390\/asi4030040","journal-title":"Appl. Syst. Innov."},{"key":"1807_CR11","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1007\/s10916-020-01617-3","volume":"44","author":"A Bansal","year":"2020","unstructured":"A. Bansal, R.P. Padappayil, C. Garg, A. Singal, M. Gupta and A. Klein, \u201cUtility of Artificial Intelligence Amidst the COVID 19 Pandemic: A Review\u201d, J. Med. Syst., 44, 156, 2020. https:\/\/doi.org\/10.1007\/s10916-020-01617-3","journal-title":"J. Med. Syst."},{"key":"1807_CR12","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1007\/s10916-021-01705-y","volume":"45","author":"CM Williams","year":"2021","unstructured":"C.M. Williams, R. Chaturvedi, R.D. Urman, R.S. Waterman and R.A. Gabriel, \u201cArtificial Intelligence and a Pandemic: an Analysis of the Potential Uses and Drawbacks\u201d, J. Med. Syst., 45, 26, 2021. https:\/\/doi.org\/10.1007\/s10916-021-01705-y","journal-title":"J. Med. Syst."},{"key":"1807_CR13","doi-asserted-by":"publisher","unstructured":"A.S. Albahri, R.A. Hamid, J.k. Alwan, Z.T. Al-qays, A.A. Zaidan, B.B. Zaidan, A.O.S. Albahri, A.H. AlAmoodi, J.M. Khlaf, E.M. Almahdi, E. Thabet, S.M. Hadi, K.I. Mohammed, M.A. Alsalem, J.R. Al-Obaidi and H.T. Madhloom, \u201cRole of biological Data Mining and Machine Learning Techniques in Detecting and Diagnosing the Novel Coronavirus (COVID-19): A Systematic Review\u201d, J. Med. Syst., 44, 122, 2020. https:\/\/doi.org\/10.1007\/s10916-020-01582-x","DOI":"10.1007\/s10916-020-01582-x"},{"key":"1807_CR14","doi-asserted-by":"publisher","first-page":"65","DOI":"10.3389\/frai.2020.00065","volume":"3","author":"AK Arshadi","year":"2020","unstructured":"A. K. Arshadi, J. Webb, M. Salem, E. Cruz, S. Calad-Thomson, N. Ghadirian, J. Collins, E. Diez-Cecilia, B. Kelly, H. Goodarzi and J. S. Yuan, \u201eArtificial Intelligence for COVID-19 Drug Discovery and Vaccine Development\u201c, Front. Artif. Intell. Appl., 3, p. 65, 2020. https:\/\/doi.org\/10.3389\/frai.2020.00065","journal-title":"Front. Artif. Intell. Appl."},{"key":"1807_CR15","doi-asserted-by":"publisher","first-page":"1581","DOI":"10.3389\/fimmu.2020.01581","volume":"11","author":"E Ong","year":"2020","unstructured":"E. Ong, M. U. Wong, A. Huffman and Y. He, \u201eCOVID-19 Coronavirus Vaccine Design Using Reverse Vaccinology and Machine Learning\u201c, Front. Immunol., 11, p. 1581, 2020. https:\/\/doi.org\/10.3389\/fimmu.2020.01581","journal-title":"Front. Immunol."},{"key":"1807_CR16","doi-asserted-by":"crossref","unstructured":"M. Hofmarcher, A. Mayr, E. Rumetshofer, P. Ruch, P. Renz, J. Schimunek, P. Seidl, A. Vall, M. Widrich, S. Hochreiter and G. Klambauer, \u201eLarge-scale ligand-based virtual screening for SARS-CoV-2 inhibitors using deep neural networks\u201c, arXiv, pp. 2010.06498v2, preprint: not peer reviewed, 2021. arXiv:\u00a02004.00979","DOI":"10.2139\/ssrn.3561442"},{"key":"1807_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2020\/9756518","volume":"2020","author":"I Ozsahin","year":"2020","unstructured":"I. Ozsahin, B. Sekeroglu, M. S. Musa, M. T. Mustapha and D. U. Ozsahin, \u201eReview on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence\u201c, Comput. Math. Method. M., 2020, p. 1-10, 2020. https:\/\/doi.org\/10.1155\/2020\/9756518","journal-title":"Comput. Math. Method. M."},{"key":"1807_CR18","doi-asserted-by":"publisher","first-page":"398","DOI":"10.12788\/fp.0045","volume":"37","author":"AA Borkowski","year":"2020","unstructured":"A. A. Borkowski, N. A. Viswanadhan, L. B. Thomas, R. D. Guzman, L. A. Deland and S. M. Mastorides, \u201eUsing Artificial Intelligence for COVID-19 Chest X-ray Diagnosis\u201c, Fed. Pract., 37, p. 398\u2013404, 2020. https:\/\/doi.org\/10.12788\/fp.0045","journal-title":"Fed. Pract."},{"key":"1807_CR19","doi-asserted-by":"publisher","DOI":"10.1016\/j.imu.2020.100505","volume":"22","author":"P Saha","year":"2021","unstructured":"P. Saha, M. S. Sadi and M. M. Islam, \u201eEMCNet: Automated COVID-19 diagnosis from X-ray images using convolutional neural network and ensemble of machine learning classifiers\u201c, Inform. Med. Unlocked, 22, p. 100505, 2021. https:\/\/doi.org\/10.1016\/j.imu.2020.100505","journal-title":"Inform. Med. Unlocked"},{"key":"1807_CR20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s13755-020-00135-3","volume":"9","author":"TD Pham","year":"2021","unstructured":"T. D. Pham, \u201eClassification of COVID-19 chest X-rays with deep learning: new models or fine tuning?\u201c, Health inf. sci. syst., 9, p. 1-11, 2021. https:\/\/doi.org\/10.1007\/s13755-020-00135-3","journal-title":"Health inf. sci. syst."},{"key":"1807_CR21","doi-asserted-by":"publisher","first-page":"3595","DOI":"10.1109\/JBHI.2020.3037127","volume":"24","author":"S Tabik","year":"2020","unstructured":"S. Tabik, A. G\u00f3mez-R\u00edos, J. L. Mart\u00edn-Rodr\u00edguez, I. Sevillano-Garc\u00eda, M. Rey-Area, D. Charte, E. Guirado, J. L. Su\u00e1rez, J. Luengo, M. A. Valero-Gonz\u00e1lez, P. Garc\u00eda-Villanova, E. Olmedo-S\u00e1nchez and F. Herrera, \u201eCOVIDGR Dataset and COVID-SDNet Methodology for Predicting COVID-19 Based on Chest X-Ray Images\u201c, IEEE J. Biomed. and Health Inform., 24, p. 3595-3605, 2020. https:\/\/doi.org\/10.1109\/JBHI.2020.3037127","journal-title":"IEEE J. Biomed. and Health Inform."},{"key":"1807_CR22","doi-asserted-by":"publisher","unstructured":"G. Wang, X. Liu, J. Shen, C. Wang, Z. Li, L. Ye, X. Wu, T. Chen, K. Wang, X. Zhang, Z. Zhou, J. Yang, Y. Sang, R. Deng, W. Liang, T. Yu, M. Gao, J. Wang, Z. Yang, H. Cai, G. Lu, L. Zhang, L. Yang, W. Xu, W. Wang, A. Olevera, I. Ziyar, C. Zhang, O. Li, W. Liao, J. Liu, W. Chen, W. Chen, J. Shi, L. Zheng, L. Zhang, Z. Yan, X. Zou, G. Lin, G. Cao, L. L. Lau, L. Mo, Y. Liang, M. Roberts, E. Sala, C.-B. Sch\u00f6nlieb, M. Fok, J. Yiu-Nam Lau, T. Xu, J. He, K. Zhang, W. Li, T. Lin, \u201cA deep-learning pipeline for the diagnosis and discrimination of viral, non-viral and COVID-19 pneumonia from chest X-ray images\u201d, Nat. Biomed. Eng., 5, p. 509\u2013521, 2021. https:\/\/doi.org\/10.1038\/s41551-021-00704-1","DOI":"10.1038\/s41551-021-00704-1"},{"key":"1807_CR23","doi-asserted-by":"publisher","unstructured":"M. Roberts, D. Driggs, M. Thorpe, J. Gilbey, M. Yeung, S. Ursprung, A. I. Aviles-Rivero, C. Etmann, C. McCague, L. Beer, J. R. Weir-McCall, Z. Teng, E. Gkrania-Klotsas, AIX-COVNET, J. H. F. Rudd, E. Sala & C.-B. Sch\u00f6nlieb, \u201cCommon pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans\u201d, Nat. Mach. Intell., 2021, https:\/\/doi.org\/10.1038\/s42256-021-00307-0","DOI":"10.1038\/s42256-021-00307-0"},{"key":"1807_CR24","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1007\/s10916-021-01745-4","volume":"45","author":"K Hammoudi","year":"2021","unstructured":"K. Hammoudi, H. Benhabiles, M. Melkemi, F. Dornaika, I. Arganda-Carreras, D. Collard and A. Scherpereel, \u201cDeep Learning on Chest X-ray Images to Detect and Evaluate Pneumonia Cases at the Era of COVID-19\u201d, J. Med. Syst., 45, 75, 2021. https:\/\/doi.org\/10.1007\/s10916-021-01745-4","journal-title":"J. Med. Syst."},{"key":"1807_CR25","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1007\/s10916-021-01747-2","volume":"45","author":"K Santosh","year":"2021","unstructured":"K. Santosh, S. Ghosh, \u201cCovid-19 Imaging Tools: How Big Data is Big?\u201d, J. Med. Syst., 45, 71, 2021. https:\/\/doi.org\/10.1007\/s10916-021-01747-2","journal-title":"J. Med. Syst."},{"key":"1807_CR26","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1007\/s42979-020-00422-6","volume":"2","author":"P Mouawad","year":"2021","unstructured":"P. Mouawad, T. Dubnov and S. Dubnov, \u201eRobust Detection of COVID-19 in Cough Sounds\u201c, SN Computer Science, 2, p. 34, 2021. https:\/\/doi.org\/10.1007\/s42979-020-00422-6","journal-title":"SN Computer Science"},{"key":"1807_CR27","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2021.564906","author":"BW Schuller","year":"2020","unstructured":"B. W. Schuller, D. M. Schuller, K. Qian, J. Liu, H. Zheng and X. Li, \u201eCOVID-19 and Computer Audition: An Overview on WhatSpeech & Sound Analysis Could Contribute in the SARS-CoV-2 Corona Crisis\u201c, Front. Digit. Health, 2020. https:\/\/doi.org\/10.3389\/fdgth.2021.564906","journal-title":"Front. Digit. Health"},{"key":"1807_CR28","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1109\/OJEMB.2020.3026928","volume":"1","author":"J Laguarta","year":"2020","unstructured":"J. Laguarta, F. Hueto and B. Subirana, \u201eCOVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings\u201c, IEEE open j. eng. med. biol., 1, p. 275-281, 2020. https:\/\/doi.org\/10.1109\/OJEMB.2020.3026928","journal-title":"IEEE open j. eng. med. biol."},{"key":"1807_CR29","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1093\/labmed\/lmaa111","volume":"52","author":"T Tschoellitsch","year":"2020","unstructured":"T. Tschoellitsch, M. D\u00fcnser, C. B\u00f6ck, K. Schwarzbauer and J. Meier, \u201eMachine Learning Prediction of SARS-CoV-2 Polymerase Chain Reaction Results with Routine Blood Tests\u201c, Lab. Med., 52, p. 146-149, 2020. https:\/\/doi.org\/10.1093\/labmed\/lmaa111","journal-title":"Lab. Med."},{"key":"1807_CR30","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1007\/s10916-020-01597-4","volume":"44","author":"D Brinati","year":"2020","unstructured":"D. Brinati, A. Campagner, D. Ferrari, M. Locatelli, G. Banfi and F. Cabitza, \u201eDetection of COVID-19 Infection from Routine Blood Exams with Machine Learning: A Feasibility Study\u201c, J. Med. Syst., 44, p. 135, 2020. https:\/\/doi.org\/10.1007\/s10916-020-01597-4","journal-title":"J. Med. Syst."},{"key":"1807_CR31","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0239474","volume":"15","author":"D Goodman-Meza","year":"2020","unstructured":"D. Goodman-Meza, A. Rudas, J. N.Chiang, P. C. Adamson, J. Ebinger, N. Sun, P. Botting, J. A. Fulcher, F. G. Saab, R. Brook, E. Eskin, U. An, M. Kordi, B. Jew, B. Balliu, Z. Chen, B. L. Hill, E. Rahman, E. Halperin and V. Manue, \u201eA machine learning algorithm to increase COVID-19 inpatient diagnostic capacity\u201c, Plos One, 15, p. e0239474, 2020. https:\/\/doi.org\/10.1371\/journal.pone.0239474","journal-title":"Plos One"},{"key":"1807_CR32","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1515\/cclm-2020-1294","volume":"59","author":"F Cabitza","year":"2021","unstructured":"F. Cabitza, A. Campagner, D. Ferrari, C. D. Resta, D. Ceriotti, E. Sabetta, A. Colombini, E. D. Vecchi, G. Banfi, M. Locatelli and A. Carobene, \u201eDevelopment, evaluation, and validation of machine learning models for COVID-19 detection based on routine blood tests\u201c, Clin. Chem. Lab. Med., 59, p. 421-431, 2021. https:\/\/doi.org\/10.1515\/cclm-2020-1294","journal-title":"Clin. Chem. Lab. Med."},{"key":"1807_CR33","doi-asserted-by":"publisher","unstructured":"T. Langer, M. Favarato, R. Giudici, G. Bassi, R. Garberi, F. Villa, H. Gay, A. Zeduri, S. Bragagnolo, A. Molteni, M. C. Andrea Beretta, M. Moreno, C. Vismara, C. F. Perno, M. Buscema, E. Grossi and R. Fumagalli, \u201eDevelopment of machine learning models to predict RT-PCR results for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) in patients with influenza-like symptoms using only basic clinical data\u201c, Scand. j. trauma resusc., 28, p. 1\u201314, 2020. https:\/\/doi.org\/10.1186\/s13049-020-00808-8","DOI":"10.1186\/s13049-020-00808-8"},{"key":"1807_CR34","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/s2589-7500(20)30274-0","volume":"3","author":"AAS Soltan","year":"2021","unstructured":"A. A. S. Soltan, S. Kouchaki, T. Zhu, D. Kiyasseh, T. Taylor, Z. B. Hussain, T. Peto, A. J. Brent, D. W. Eyre and D. A. Clifton, \u201eRapid triage for COVID-19 using routine clinical data for patients attending hospital: development and prospective validation of an artificial intelligence screening test\u201c, Lancet Digit. Health, 3, p. 78-87, 2021. https:\/\/doi.org\/10.1016\/s2589-7500(20)30274-0","journal-title":"Lancet Digit. Health"},{"key":"1807_CR35","doi-asserted-by":"publisher","first-page":"1396","DOI":"10.1093\/clinchem\/hvaa200","volume":"66","author":"HS Yang","year":"2020","unstructured":"H. S. Yang, Y. Hou, L. V. Vasovic, P. A. D. Steel, A. Chadburn, S. E. Racine-Brzostek, P. Velu, M. M. Cushing, M. Loda, R. Kaushal, Z. Zhao and F. Wang, \u201eRoutine Laboratory Blood Tests Predict SARS-CoV-2 Infection Using Machine Learning\u201c, Clin. Chem., 66, p. 1396-1404, 2020. https:\/\/doi.org\/10.1093\/clinchem\/hvaa200","journal-title":"Clin. Chem."},{"key":"1807_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2196\/24048","volume":"22","author":"TB Plante","year":"2020","unstructured":"T. B. Plante, A. M. Blau, A. N. Berg, A. S. Weinberg, I. C. Jun, V. F. Tapson, T. S. Kanigan and A. B. Adib, \u201eDevelopment and External Validation of a Machine Learning Tool to Rule Out COVID-19 Among Adults in the Emergency Department Using Routine Blood Tests: A Large, Multicenter, Real-World Study\u201c, J. Med. Internet Res., 22, p. 1-19, 2020. https:\/\/doi.org\/10.2196\/24048","journal-title":"J. Med. Internet Res."},{"key":"1807_CR37","doi-asserted-by":"publisher","unstructured":"M. Almansoor and N. M. Hewahi, \u201eExploring the Relation between Blood Tests and Covid-19 Using Machine Learning\u201c, ICDABI, p. 1\u20136, 2020. https:\/\/doi.org\/10.1109\/ICDABI51230.2020.9325673","DOI":"10.1109\/ICDABI51230.2020.9325673"},{"key":"1807_CR38","doi-asserted-by":"publisher","DOI":"10.1016\/j.imu.2020.100449","volume":"21","author":"M AlJame","year":"2020","unstructured":"M. AlJame, I. Ahmad, A. Imtiaz and A. Mohammed, \u201eEnsemble learning model for diagnosing COVID-19 from routine blood tests\u201c, Inform. Med. Unlocked, 21, p. 100449, 2020. https:\/\/doi.org\/10.1016\/j.imu.2020.100449","journal-title":"Inform. Med. Unlocked"},{"key":"1807_CR39","doi-asserted-by":"publisher","first-page":"e114","DOI":"10.7861\/clinmed.2020-0373","volume":"20","author":"V Formica","year":"2020","unstructured":"V. Formica, M. Minieri, S. Bernardini, M. Ciotti, C. D\u2019Agostini, M. Roselli, M. Andreoni, C. Morelli, G. Parisi, M. Federici, C. Paganelli and J. M. Legramante, \u201eComplete blood count might help to identify subjects with high probability of testing positive to SARS-CoV-2\u201c, Clin. Med., 20, p. e114--e119, 2020. https:\/\/doi.org\/10.7861\/clinmed.2020-0373","journal-title":"Clin. Med."},{"key":"1807_CR40","doi-asserted-by":"publisher","unstructured":"V. A. Freitas Barbosa, J. C. Gomes, M. A. Santana, J. E. A. Albuquerque, R. G. Souza, R. E. Souza and W. P. Santos, \u201eHeg.IA: an intelligent system to support diagnosis of Covid-19 based on blood tests\u201c, Res. Biomed. Eng., 2021. https:\/\/doi.org\/10.1007\/s42600-020-00112-5","DOI":"10.1007\/s42600-020-00112-5"},{"key":"1807_CR41","doi-asserted-by":"publisher","unstructured":"A. Banerjee, S. Ray, B. Vorselaars, J. Kitson, M. Mamalakis, S. Weeks, M. Baker and L. S. Mackenzie, \u201eUse of Machine Learning and Artificial Intelligence to predict SARS-CoV-2 infection from Full Blood Counts in a population\u201c, Int. Immunopharmacol., 86, 2020. https:\/\/doi.org\/10.1016\/j.intimp.2020.106705","DOI":"10.1016\/j.intimp.2020.106705"},{"key":"1807_CR42","doi-asserted-by":"publisher","first-page":"39","DOI":"10.30699\/fhi.v9i1.234","volume":"9","author":"EC Silveira","year":"2020","unstructured":"E. C. Silveira, \u201ePrediction of COVID-19 From Hemogram Results and Age Using Machine Learning\u201c, Front. health inform., 9, p. 39, 2020. https:\/\/doi.org\/10.30699\/fhi.v9i1.234","journal-title":"Front. health inform."},{"key":"1807_CR43","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.9482","volume":"8","author":"E Avila","year":"2020","unstructured":"E. Avila, A. Kahmann, C. Alho and M. Dorn, \u201eHemogram data as a tool for decision-making in COVID-19 management: applications to resource scarcity scenarios\u201c, PeerJ, 8, p. e9482, 2020. https:\/\/doi.org\/10.7717\/peerj.9482","journal-title":"PeerJ"},{"key":"1807_CR44","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1093\/infdis\/jiaa663","volume":"223","author":"H Sun","year":"2020","unstructured":"H. Sun, A. Jain, M. J. Leone, H. S. Alabsi, L. N. Brenner, E. Ye, W. Ge, Y.-P. Shao, C. L. Boutros, R. Wang, R. A. Tesh, C. Magdamo, S. I. Collens, W. Ganglberger, I. V. Bassett, J. B. Meigs, J. Kalpathy-Cramer, M. D. Li, J. T. Chu, M. L. Dougan, L. W. Stratton, J. Rosand, B. Fischl, S. Das, S. S. Mukerji, G. K. Robbins and M. B. Westover, \u201eCoVA: An Acuity Score for Outpatient Screening that Predicts Coronavirus Disease 2019 Prognosis\u201c, J. Infect. Dis., 223, p. 38-46, 2020. https:\/\/doi.org\/10.1093\/infdis\/jiaa663","journal-title":"J. Infect. Dis."},{"key":"1807_CR45","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-83784-y","volume":"2020","author":"Y Zhou","year":"2020","unstructured":"Y. Zhou, B. Li, J. Liu and D. Chen, \u201eThe Predictive Effectiveness of Blood Biochemical Indexes for the Severity of COVID-19\u201c, Can. J. Infect. Dis. Med. Microbiol., 2020, p. 732081, 2020. https:\/\/doi.org\/10.1038\/s41598-021-83784-y","journal-title":"Can. J. Infect. Dis. Med. Microbiol."},{"key":"1807_CR46","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1038\/s41379-020-00700-x","volume":"34","author":"AL Booth","year":"2020","unstructured":"A. L. Booth, E. Abels and P. McCaffrey, \u201eDevelopment of a prognostic model for mortality in COVID-19 infection using machine learning\u201c, Mod. Pathol., 34, p. 522-531, 2020. https:\/\/doi.org\/10.1038\/s41379-020-00700-x","journal-title":"Mod. Pathol."},{"key":"1807_CR47","doi-asserted-by":"publisher","DOI":"10.7717\/peerj.10337","volume":"8","author":"X Li","year":"2020","unstructured":"X. Li, P. Ge, J. Zhu, H. Li, J. Graham, A. Singer, P. S. Richman and T. Q. Duong, \u201eDeep learning prediction of likelihood of ICU admission and mortality in COVID-19 patients using clinical variables\u201c, PeerJ, 8, p. e10337, 2020. https:\/\/doi.org\/10.7717\/peerj.10337","journal-title":"PeerJ"},{"key":"1807_CR48","doi-asserted-by":"publisher","unstructured":"F. S. Heldt, M. P. Vizcaychipi, S. Peacock, M. Cinelli, L. McLachlan, F. Andreotti, S. Jovanovi\u0107, N. L. Robert D\u00fcrichen, R. A. Fletcher, A. Hancock, A. McCarthy, R. A. Pointon, A. Brown, J. Eaton, R. Liddi, L. Mackillop, L. Tarassenko and R. T. Khan, \u201eEarly risk assessment for COVID-19 patients from emergency department data using machine learning\u201c, Sci. Rep., 11, p. 4200, 2021. https:\/\/doi.org\/10.1038\/s41598-021-83784-y","DOI":"10.1038\/s41598-021-83784-y"},{"key":"1807_CR49","doi-asserted-by":"publisher","unstructured":"S. Heber, D. Pereyra, W. C. Schrottmaier, K. Kammerer, J. Santol, E. Pawelka, M. Hana, A. Scholz, M. Liu, A. Hell, K. Heiplik, B. Lickefett, S. Havervall, M. T. Traugott, M. Neub\u00f6ck, C. Sch\u00f6rgenhofer, T. Seitz, C. Firbas, M. Karolyi, G. Weiss, B. Jilma, C. Thralin, R. Bellmann-Weiler, H. J. F. Salzer, M. J. M. Fischer, A. Zoufaly and A. Assinger, \u201dDevelopment and external validation of a logistic regression derived formula based on repeated routine hematological measurements predicting survival of hospitalized Covid-19 patients\u201c, medRxiv, 2020. https:\/\/doi.org\/10.1101\/2020.12.20.20248563","DOI":"10.1101\/2020.12.20.20248563"},{"key":"1807_CR50","doi-asserted-by":"publisher","first-page":"5033","DOI":"10.1038\/s41467-020-18684-2","volume":"11","author":"Y Gao","year":"2020","unstructured":"Y. Gao, G.-Y. Cai, W. Fang, H.-Y. Li, S.-Y. Wang, L. Chen, Y. Yu, D. Liu, S. Xu, P.-F. Cui, S.-Q. Zeng, X.-X. Feng, R.-D. Yu, Y. Wang, Y. Yuan, X.-F. Jiao, J.-H. Chi, J.-H. Liu, R.-Y. Li, X. Zheng, C.-Y. Song, N. Jin, W.-J. Gong, X.-Y. Liu, L. Huang, X. Tian, L. Li, H. Xing, D. Ma, C.-R. Li, F. Ye and Q.-L. Gao, \u201dMachine learning based early warning system enables accurate mortality risk prediction for COVID-19\u201c, Nat. Commun., 11, p. 5033, 2020. https:\/\/doi.org\/10.1038\/s41467-020-18684-2","journal-title":"Nat. Commun."},{"key":"1807_CR51","doi-asserted-by":"publisher","unstructured":"A. Vaid, S. Somani, A. J. Russak, J. K. De Freitas, F. F. Chaudhry, I. Paranjpe, K. W. Johnson, S. J. Lee, R. Miotto, F. Richter, S. Zhao, N. D. Beckmann, N. Naik, A. Kia, P. Timsina, A. Lala, M. Paranjpe, E. Golden, M. Danieletto, M. Singh, D. Meyer, P. F. O\\'Reilly, L. Huckins, P. Kovatch, J. Finkelstein, R. M. Freeman, E. Argulian, A. Kasarskis, B. Percha, J. A. Aberg, E. Bagiella, C. R. Horowitz, B. Murphy, E. J. Nestler, E. E. Schadt, J. H. Cho, C. Cordon-Cardo, V. Fuster, D. S. Charney, D. L. Reich, E. P. Bottinger, M. A. Levin, J. Narula, Z. A. Fayad, A. C. Just, A. W. Charney, G. N. Nadkarni and B. S. Glicksberg, \u201eMachine Learning to Predict Mortality and Critical Events in a Cohort of Patients With COVID-19 in New York City: Model Development and Validation\u201c, J. Med. Internet Res., 22, p. 1\u201319, 2020. https:\/\/doi.org\/10.2196\/24018","DOI":"10.2196\/24018"},{"key":"1807_CR52","doi-asserted-by":"publisher","DOI":"10.2196\/25442","volume":"22","author":"H Ko","year":"2020","unstructured":"H. Ko, H. Chung, W. S. Kang, C. Park, D. W. Kim, S. E. Kim, C. R. Chung, R. E. Ko, H. Lee, J. H. Seo, T.-Y. Choi, R. Jaimes, K. W. Kim and J. Lee, \u201eAn Artificial Intelligence Model to Predict the Mortality of COVID-19 Patients at Hospital Admission Time Using Routine Blood Samples: Development and Validation of an Ensemble Model\u201c, J. Med. Internet Res., 22, p. e25442, 2020. https:\/\/doi.org\/10.2196\/25442","journal-title":"J. Med. Internet Res."},{"issue":"2","key":"1807_CR53","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1109\/JBHI.2020.3032060","volume":"25","author":"K Stacke","year":"2021","unstructured":"K. Stacke, G. Eilertsen, J. Unger, C. Lundstrom, \u201cMeasuring Domain Shift for Deep Learning in Histopathology\u201d. IEEE J. Biomed. Health Inform., 25(2), p.325-336, 2021 https:\/\/doi.org\/10.1109\/JBHI.2020.3032060","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"1807_CR54","unstructured":"P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang, A. Balsubramani, W. Hu, M. Yasunaga, R. L. Phillips, I. Gao, T. Lee, E. David, I. Stavness, W. Guo, B. A. Earnshaw, I. S. Haque, S. Beery, J. Leskovec, A. Kundaje, E. Pierson, S. Levine, C. Finn and P. Liang, \u201eWILDS: A Benchmark of in-the-Wild Distribution Shifts\u201c, Proceedings of Machine Learning Research, 139:5637\u20135664,\u00a02021. arXiv:\u00a02012.07421"},{"key":"1807_CR55","unstructured":"J. J. Thiagarajan, R. Deepta, P. Sattigeri, \u201cUnderstanding Behavior of Clinical Models under Domain Shifts\u201d, arXiv, p. 1809.07806v2, 2019, preprint: not peer reviewed., arXiv: 1809.07806"},{"key":"1807_CR56","doi-asserted-by":"crossref","unstructured":"M. Schneider, L. Wang, C. Marr, \u201dEvaluation of Domain Adaptation Approaches for Robust Classification of Heterogeneous Biological Data Sets\u201d, Artificial Neural Networks and Machine Learning -- ICANN 2019: Deep Learning, pp. 673\u2014686, 2019, ISBN: 978\u20133\u2013030\u201330484\u20133","DOI":"10.1007\/978-3-030-30484-3_53"},{"key":"1807_CR57","doi-asserted-by":"publisher","unstructured":"G.S. Collins, P. Dhiman, C. L. Andaur Navarro, J. Ma , L. Hooft, J. B. Reitsma, P. Logullo, A. L. Beam , L. Peng, B. Van Calster , M. van Smeden, R. D. Riley, K. G. M. Moons, \u201cProtocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence\u201d, BMJ Open 11:e048008. 2021 https:\/\/doi.org\/10.1136\/bmjopen-2020-048008","DOI":"10.1136\/bmjopen-2020-048008"},{"key":"1807_CR58","unstructured":"Federal Ministry of Social Affairs, Health, Care and Consumer Protection (BMSGPK), Austrian COVID-19 Open Data Information portal. https:\/\/www.data.gv.at\/covid-19\/"},{"key":"1807_CR59","doi-asserted-by":"publisher","unstructured":"M. Wulfmeier, A. Bewley and I. Posner, \u201eIncremental Adversarial Domain Adaptation for Continually Changing Environments\u201c, ICRA, p. 1\u20139, 2018. https:\/\/doi.org\/10.1109\/ICRA.2018.8460982","DOI":"10.1109\/ICRA.2018.8460982"},{"key":"1807_CR60","unstructured":"T. Adler, J. Brandstetter, M. Widrich, A. Mayr, D. Kreil, M. Kopp, G. Klambauer and S. Hochreiter, \u201eCross-Domain Few-Shot Learning by Representation Fusion,\u201c arXiv, p. 2010.06498v2, preprint: not peer reviewed., 2021. arXiv:\u00a02010.06498"},{"key":"1807_CR61","doi-asserted-by":"publisher","unstructured":"V. L. S. Crawford, O. Sweeney, P. V. Coyle, I. M. Halliday and R. W. Stout, \u201eThe relationship between elevated fibrinogen and markers of infection: a comparison of seasonal cycles\u201c, QJM: An International Journal of Medicine, 93, p. 745\u2013750, 2000. https:\/\/doi.org\/10.1093\/qjmed\/93.11.745","DOI":"10.1093\/qjmed\/93.11.745"},{"key":"1807_CR62","doi-asserted-by":"publisher","first-page":"2163","DOI":"10.18653\/v1\/D19-1222","volume":"9","author":"H Elsahar","year":"2019","unstructured":"H. Elsahar and M. Gall\u00e9, \u201eTo Annotate or Not? Predicting Performance Drop under Domain Shift\u201c, EMNLP-IJCNLP, 9, p. 2163-2173, 2019. https:\/\/doi.org\/10.18653\/v1\/D19-1222","journal-title":"EMNLP-IJCNLP"},{"key":"1807_CR63","unstructured":"G. Klambauer, T. Unterthiner, A. Mayr and S. Hochreiter, \u201eSelf-normalizing neural networks\u201c, NIPS, p. 971\u2013980, 2017. arXiv:1706.02515"},{"key":"1807_CR64","doi-asserted-by":"publisher","unstructured":"F. E. Harrell Jr., K. L. Lee and D. B. Mark, \u201eMultivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors\u201c, Stat. Med., 15, p. 361-387, 1996.\u00a0https:\/\/doi.org\/10.1002\/0470023678.CH2B(I)","DOI":"10.1002\/0470023678.CH2B(I)"},{"key":"1807_CR65","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1145\/1143844.1143874","volume":"23","author":"J Davis","year":"2006","unstructured":"J. Davis and M. Goadrich, \u201eThe relationship between Precision-Recall and ROC Curves\u201c, ICML, 23, p. 233\u2013240, 2006. https:\/\/doi.org\/10.1145\/1143844.1143874","journal-title":"ICML"},{"key":"1807_CR66","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2907070","volume":"49","author":"P Branco","year":"2016","unstructured":"P. Branco, L. Torgo and R. P. Ribeiro, \u201eA Survey of Predictive Modeling on Imbalanced Domains\u201c, ACM Comput. Surv., 49, p. 1-50, 2016. https:\/\/doi.org\/10.1145\/2907070","journal-title":"ACM Comput. Surv."},{"key":"1807_CR67","unstructured":"J. H. Zar, \u201cBiostatistical Analysis\u201d, 5th edition, Prentice Hall, Upper Saddle River, New Jersey USA, 2010, ISBN: 9780321656865"}],"container-title":["Journal of Medical Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-022-01807-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10916-022-01807-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10916-022-01807-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,4,21]],"date-time":"2022-04-21T09:53:23Z","timestamp":1650534803000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10916-022-01807-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,29]]},"references-count":67,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2022,5]]}},"alternative-id":["1807"],"URL":"https:\/\/doi.org\/10.1007\/s10916-022-01807-1","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.04.06.21254997","asserted-by":"object"}]},"ISSN":["1573-689X"],"issn-type":[{"value":"1573-689X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,29]]},"assertion":[{"value":"1 October 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Ethics approval for this study was obtained from the ethics committee of the Johannes Kepler University, Linz (approval number: 1104\/2020). In our study, we analyze anonymized data only.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval"}},{"value":"The authors declare no conflicts of interest.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflicts of Interest"}}],"article-number":"23"}}