{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:45:11Z","timestamp":1760150711521,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T00:00:00Z","timestamp":1701648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute of Information &amp; Communications Technology Planning &amp; Evaluation grant funded by the Korean government","doi-asserted-by":"publisher","award":["2021-0-00312"],"award-info":[{"award-number":["2021-0-00312"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Coronavirus has caused many casualties and is still spreading. Some people experience rapid deterioration that is mild at first. The aim of this study is to develop a deterioration prediction model for mild COVID-19 patients during the isolation period. We collected vital signs from wearable devices and clinical questionnaires. The derivation cohort consisted of people diagnosed with COVID-19 between September and December 2021, and the external validation cohort collected between March and June 2022. To develop the model, a total of 50 participants wore the device for an average of 77 h. To evaluate the model, a total of 181 infected participants wore the device for an average of 65 h. We designed machine learning-based models that predict deterioration in patients with mild COVID-19. The prediction model, 10 min in advance, showed an area under the receiver characteristic curve (AUC) of 0.99, and the prediction model, 8 h in advance, showed an AUC of 0.84. We found that certain variables that are important to model vary depending on the point in time to predict. Efficient deterioration monitoring in many patients is possible by utilizing data collected from wearable sensors and symptom self-reports.<\/jats:p>","DOI":"10.3390\/s23239597","type":"journal-article","created":{"date-parts":[[2023,12,4]],"date-time":"2023-12-04T05:28:21Z","timestamp":1701667701000},"page":"9597","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Predicting Deterioration from Wearable Sensor Data in People with Mild COVID-19"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3659-3149","authenticated-orcid":false,"given":"Jin-Yeong","family":"Kang","sequence":"first","affiliation":[{"name":"Department of Medical Informatics, Keimyung University, Daegu 42601, Republic of Korea"},{"name":"Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0763-5458","authenticated-orcid":false,"given":"Ye Seul","family":"Bae","sequence":"additional","affiliation":[{"name":"Department of Family Medicine, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul 03181, Republic of Korea"},{"name":"Department of Future Healthcare Planning, Kangbuk Samsung Hospital, Sungkyunkwan University School of Medicine, Seoul 03181, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2027-7472","authenticated-orcid":false,"given":"Eui Kyu","family":"Chie","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Seoul National University College of Medicine, Seoul 03080, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6817-5793","authenticated-orcid":false,"given":"Seung-Bo","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Medical Informatics, Keimyung University, Daegu 42601, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,4]]},"reference":[{"unstructured":"World Health Organization (2020). Coronavirus 2019 (COVID-19), World Health Organization.","key":"ref_1"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e516","DOI":"10.1016\/S2589-7500(20)30217-X","article-title":"Clinical features of COVID-19 mortality: Development and validation of a clinical prediction model","volume":"2","author":"Yadaw","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1016\/j.ajem.2020.04.048","article-title":"Cardiovascular complications in COVID-19","volume":"38","author":"Long","year":"2020","journal-title":"Am. J. Emerg. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1528","DOI":"10.15585\/mmwr.mm6942e3","article-title":"Risk for in-hospital complications associated with COVID-19 and influenza\u2014Veterans Health Administration, United States, October 1, 2018\u2013May 31, 2020","volume":"69","author":"Cates","year":"2020","journal-title":"Morb. Mortal. Wkly. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1708","DOI":"10.1056\/NEJMoa2002032","article-title":"Clinical characteristics of coronavirus disease 2019 in China","volume":"382","author":"Guan","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1007\/s15010-022-01783-1","article-title":"A simple risk score for mortality including the PCR Ct value upon admission in patients hospitalized due to COVID-19","volume":"50","author":"Kurzeder","year":"2022","journal-title":"Infection"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1016\/j.jinf.2020.05.064","article-title":"A clinical risk score to identify patients with COVID-19 at high risk of critical care admission or death: An observational cohort study","volume":"81","author":"Galloway","year":"2020","journal-title":"J. Infect."},{"doi-asserted-by":"crossref","unstructured":"Bertsimas, D., Lukin, G., Mingardi, L., Nohadani, O., Orfanoudaki, A., Stellato, B., Wiberg, H., Gonzalez-Garcia, S., Parra-Calder\u00f3n, C.L., and Robinson, K. (2020). COVID-19 mortality risk assessment: An international multi-center study. PLoS ONE, 15.","key":"ref_8","DOI":"10.1101\/2020.07.07.20148304"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"948.e1","DOI":"10.1016\/j.cmi.2020.04.040","article-title":"Clinical characteristics of asymptomatic and symptomatic patients with mild COVID-19","volume":"26","author":"Kim","year":"2020","journal-title":"Clin. Microbiol. Infect."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1186\/s12967-020-02505-7","article-title":"ANDC: An early warning score to predict mortality risk for patients with Coronavirus Disease 2019","volume":"18","author":"Weng","year":"2020","journal-title":"J. Transl. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1111\/resp.14026","article-title":"Epidemiology, clinical spectrum, viral kinetics and impact of COVID-19 in the Asia-Pacific region","volume":"26","author":"Kwok","year":"2021","journal-title":"Respirology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1007\/s10654-023-00973-x","article-title":"Prognostic models in COVID-19 infection that predict severity: A systematic review","volume":"38","author":"Buttia","year":"2023","journal-title":"Eur. J. Epidemiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1007\/s10462-021-09982-2","article-title":"Use of learning approaches to predict clinical deterioration in patients based on various variables: A review of the literature","volume":"55","author":"Moghbel","year":"2022","journal-title":"Artif. Intell. Rev."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2630","DOI":"10.1038\/s41598-022-05822-7","article-title":"A machine learning model for predicting deterioration of COVID-19 inpatients","volume":"12","author":"Noy","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"7097","DOI":"10.1038\/s41598-022-09771-z","article-title":"Machine learning-based model for prediction of clinical deterioration in hospitalized patients by COVID 19","volume":"12","author":"Lafuente","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1016\/j.jaip.2020.06.013","article-title":"Prompt predicting of early clinical deterioration of moderate-to-severe COVID-19 patients: Usefulness of a combined score using IL-6 in a preliminary study","volume":"8","author":"Vultaggio","year":"2020","journal-title":"J. Allergy Clin. Immunol. Pract."},{"unstructured":"Zhou, Z., Li, W., Qian, J., Lin, B., Nan, Y., Lu, F., Wan, L., Zhao, X., Luo, A., and Liao, X. (2023, November 27). Predicting the Risk of Clinical Deterioration in Patients with Severe COVID-19 Infection Using Machine Learning. Available online: https:\/\/ses.library.usyd.edu.au\/handle\/2123\/23370.","key":"ref_17"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"115","DOI":"10.4046\/trd.2020.0139","article-title":"Factors Associated with Worsening Oxygenation in Patients with Non-severe COVID-19 Pneumonia","volume":"84","author":"Hahm","year":"2021","journal-title":"Tuberc. Respir. Dis."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1080\/03007995.2021.1876005","article-title":"Predictors of clinical deterioration in non-severe patients with COVID-19: A retrospective cohort study","volume":"37","author":"Yitao","year":"2021","journal-title":"Curr. Med. Res. Opin."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1038\/s41746-021-00533-1","article-title":"Passive detection of COVID-19 with wearable sensors and explainable machine learning algorithms","volume":"4","author":"Gadaleta","year":"2021","journal-title":"Npj Digit. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e85","DOI":"10.1016\/S2589-7500(19)30222-5","article-title":"Harnessing wearable device data to improve state-level real-time surveillance of influenza-like illness in the USA: A population-based study","volume":"2","author":"Radin","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"107170","DOI":"10.1016\/j.ypmed.2022.107170","article-title":"Wearable technology for early detection of COVID-19: A systematic scoping review","volume":"162","author":"Cheong","year":"2022","journal-title":"Prev. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1038\/s41591-021-01339-0","article-title":"Wearable sensors enable personalized predictions of clinical laboratory measurements","volume":"27","author":"Dunn","year":"2021","journal-title":"Nat. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1016\/j.jinf.2020.03.004","article-title":"Clinical progression of patients with COVID-19 in Shanghai, China","volume":"80","author":"Chen","year":"2020","journal-title":"J. Infect."},{"doi-asserted-by":"crossref","unstructured":"Joosten, S.A., Smeets, M.J., Arbous, M.S., Manni\u00ebn, J., Laverman, S., Driessen, M.M., Cannegieter, S.C., Roukens, A.H., and Leiden University Medical Center BEAT-COVID Group (2023). Daily disease severity in patients with COVID-19 admitted to the hospital: The SCODA (severity of coronavirus disease assessment) score. PLoS ONE, 18.","key":"ref_25","DOI":"10.1371\/journal.pone.0291212"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"104911","DOI":"10.1016\/j.ijmedinf.2022.104911","article-title":"Prediction of low pulse oxygen saturation in COVID-19 using remote monitoring post hospital discharge","volume":"169","author":"Doheny","year":"2023","journal-title":"Int. J. Med. Inform."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 17). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","key":"ref_28","DOI":"10.1145\/2939672.2939785"},{"unstructured":"Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., and Liu, T.Y. (2017). Lightgbm: A highly efficient gradient boosting decision tree. Adv. Neural Inf. Process. Syst., 30.","key":"ref_29"},{"unstructured":"Dorogush, A.V., Ershov, V., and Gulin, A. (2018). CatBoost: Gradient boosting with categorical features support. arXiv.","key":"ref_30"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"S29","DOI":"10.1097\/QMH.0000000000000397","article-title":"Order of Occurrence of COVID-19 Symptoms","volume":"32","author":"Wojtusiak","year":"2023","journal-title":"Qual. Manag. Health Care"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e162","DOI":"10.1017\/S0950268822001297","article-title":"Comparative symptomatology of infection with SARS-CoV-2 variants Omicron (B. 1.1. 529) and Delta (B. 1.617. 2) from routine contact tracing data in England","volume":"150","author":"Ekroth","year":"2022","journal-title":"Epidemiol. Infect."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e20710","DOI":"10.2196\/20710","article-title":"The Accuracy of Wrist Skin Temperature in Detecting Ovulation Compared to Basal Body Temperature: Prospective Comparative Diagnostic Accuracy Study","volume":"23","author":"Zhu","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1023\/A:1012487302797","article-title":"Gene Selection for Cancer Classification using Support Vector Machines","volume":"46","author":"Guyon","year":"2002","journal-title":"Mach. Learn."},{"doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 17). \u201cWhy should i trust you?\u201d Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","key":"ref_35","DOI":"10.1145\/2939672.2939778"},{"key":"ref_36","first-page":"458","article-title":"Estimation of the Youden Index and its associated cutoff point","volume":"47","author":"Fluss","year":"2005","journal-title":"Biom. J. J. Math. Methods Biosci."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1038\/s41591-020-1123-x","article-title":"Wearable sensor data and self-reported symptoms for COVID-19 detection","volume":"27","author":"Quer","year":"2020","journal-title":"Nat. Med."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1038\/s41746-020-00363-7","article-title":"Assessment of physiological signs associated with COVID-19 measured using wearable devices","volume":"3","author":"Natarajan","year":"2020","journal-title":"NPJ Digit. Med."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9597\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:37:08Z","timestamp":1760132228000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/23\/9597"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,4]]},"references-count":39,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["s23239597"],"URL":"https:\/\/doi.org\/10.3390\/s23239597","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,12,4]]}}}