{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T22:08:36Z","timestamp":1793398116125,"version":"4.5.0"},"reference-count":38,"publisher":"Public Library of Science (PLoS)","issue":"4","license":[{"start":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T00:00:00Z","timestamp":1712880000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004853","name":"Chinese University of Hong Kong","doi-asserted-by":"publisher","award":["IRF2022"],"award-info":[{"award-number":["IRF2022"]}],"id":[{"id":"10.13039\/501100004853","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosmedicine.org"],"crossmark-restriction":false},"short-container-title":["PLoS Med"],"abstract":"<jats:sec id=\"sec001\">\n<jats:title>Background<\/jats:title>\n<jats:p>Older adults with diabetes are at high risk of severe hypoglycemia (SH). Many machine-learning (ML) models predict short-term hypoglycemia are not specific for older adults and show poor precision-recall. We aimed to develop a multidimensional, electronic health record (EHR)-based ML model to predict one-year risk of SH requiring hospitalization in older adults with diabetes.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec002\">\n<jats:title>Methods and findings<\/jats:title>\n<jats:p>We adopted a case-control design for a retrospective territory-wide cohort of 1,456,618 records from 364,863 unique older adults (age \u226565 years) with diabetes and at least 1 Hong Kong Hospital Authority attendance from 2013 to 2018. We used 258 predictors including demographics, admissions, diagnoses, medications, and routine laboratory tests in a one-year period to predict SH events requiring hospitalization in the following 12 months. The cohort was randomly split into training, testing, and internal validation sets in a 7:2:1 ratio. Six ML algorithms were evaluated including logistic-regression, random forest, gradient boost machine, deep neural network (DNN), XGBoost, and Rulefit. We tested our model in a temporal validation cohort in the Hong Kong Diabetes Register with predictors defined in 2018 and outcome events defined in 2019. Predictive performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC) statistics, and positive predictive value (PPV). We identified 11,128 SH events requiring hospitalization during the observation periods. The XGBoost model yielded the best performance (AUROC = 0.978 [95% CI 0.972 to 0.984]; AUPRC = 0.670 [95% CI 0.652 to 0.688]; PPV = 0.721 [95% CI 0.703 to 0.739]). This was superior to an 11-variable conventional logistic-regression model comprised of age, sex, history of SH, hypertension, blood glucose, kidney function measurements, and use of oral glucose-lowering drugs (GLDs) (AUROC = 0.906; AUPRC = 0.085; PPV = 0.468). Top impactful predictors included non-use of lipid-regulating drugs, in-patient admission, urgent emergency triage, insulin use, and history of SH. External validation in the HKDR cohort yielded AUROC of 0.856 [95% CI 0.838 to 0.873]. Main limitations of this study included limited transportability of the model and lack of geographically independent validation.<\/jats:p>\n<\/jats:sec>\n<jats:sec id=\"sec003\">\n<jats:title>Conclusions<\/jats:title>\n<jats:p>Our novel-ML model demonstrated good discrimination and high precision in predicting one-year risk of SH requiring hospitalization. This may be integrated into EHR decision support systems for preemptive intervention in older adults at highest risk.<\/jats:p>\n<\/jats:sec>","DOI":"10.1371\/journal.pmed.1004369","type":"journal-article","created":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T17:20:52Z","timestamp":1712942452000},"page":"e1004369","update-policy":"https:\/\/doi.org\/10.1371\/journal.pmed.corrections_policy","source":"Crossref","is-referenced-by-count":36,"title":["A novel electronic health record-based, machine-learning model to predict severe hypoglycemia leading to hospitalizations in older adults with diabetes: A territory-wide cohort and modeling study"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2798-3268","authenticated-orcid":true,"given":"Mai","family":"Shi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2968-1762","authenticated-orcid":true,"given":"Aimin","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1581-5643","authenticated-orcid":true,"given":"Eric S. H.","family":"Lau","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5244-6069","authenticated-orcid":true,"given":"Andrea O. Y.","family":"Luk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronald C. W.","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8927-6764","authenticated-orcid":true,"given":"Alice P. S.","family":"Kong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Raymond S. M.","family":"Wong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jones C. M.","family":"Chan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1325-1194","authenticated-orcid":true,"given":"Juliana C. N.","family":"Chan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4147-3387","authenticated-orcid":true,"given":"Elaine","family":"Chow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,4,12]]},"reference":[{"issue":"3","key":"pmed.1004369.ref001","doi-asserted-by":"crossref","first-page":"e40","DOI":"10.2337\/dc20-2798","article-title":"Severe Hypoglycemia and Risk of Atherosclerotic Cardiovascular Disease in Patients With Diabetes","volume":"44","author":"JS Rana","year":"2021","journal-title":"Diabetes Care"},{"issue":"9","key":"pmed.1004369.ref002","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1111\/dom.12689","article-title":"Rates and predictors of hypoglycaemia in 27 585 people from 24 countries with insulin-treated type 1 and type 2 diabetes: the global HAT study","volume":"18","author":"K Khunti","year":"2016","journal-title":"Diabetes Obes Metab"},{"issue":"8","key":"pmed.1004369.ref003","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s11892-018-1018-0","article-title":"Hypoglycemia Among Patients with Type 2 Diabetes: Epidemiology, Risk Factors, and Prevention Strategies","volume":"18","author":"R Silbert","year":"2018","journal-title":"Curr Diab Rep"},{"issue":"12","key":"pmed.1004369.ref004","doi-asserted-by":"crossref","first-page":"2967","DOI":"10.2337\/dc20-0260","article-title":"Trends in Glucose-Lowering Drug Use, Glycemic Control, and Severe Hypoglycemia in Adults With Diabetes in Hong Kong, 2002\u20132016","volume":"43","author":"A Yang","year":"2020","journal-title":"Diabetes Care"},{"issue":"3","key":"pmed.1004369.ref005","doi-asserted-by":"crossref","first-page":"512","DOI":"10.2337\/dci19-0069","article-title":"How Significant Is Severe Hypoglycemia in Older Adults With Diabetes?","volume":"43","author":"L Chow","year":"2020","journal-title":"Diabetes Care"},{"issue":"6","key":"pmed.1004369.ref006","doi-asserted-by":"crossref","first-page":"1164","DOI":"10.2337\/dc22-2146","article-title":"A National Physician Survey of Deintensifying Diabetes Medications for Older Adults With Type 2","volume":"46","author":"SJ Pilla","year":"2023","journal-title":"Diabetes. Diabetes Care"},{"issue":"Suppl 1","key":"pmed.1004369.ref007","doi-asserted-by":"crossref","first-page":"S168","DOI":"10.2337\/dc21-S012","article-title":"12 Older adults: Standards of Medical Care in Diabetes-2021","volume":"44","author":"American Diabetes Association","year":"2021","journal-title":"Diabetes Care"},{"issue":"4","key":"pmed.1004369.ref008","doi-asserted-by":"crossref","first-page":"371","DOI":"10.2337\/cd20-0053","article-title":"Addressing Therapeutic Inertia in 2020 and Beyond: A 3-Year Initiative of the American Diabetes Association","volume":"38","author":"RA Gabbay","year":"2020","journal-title":"Clin Diabetes"},{"issue":"10","key":"pmed.1004369.ref009","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1001\/jamainternmed.2017.3844","article-title":"Development and Validation of a Tool to Identify Patients With Type 2 Diabetes at High Risk of Hypoglycemia-Related Emergency Department or Hospital Use","volume":"177","author":"AJ Karter","year":"2017","journal-title":"JAMA Intern Med"},{"issue":"7","key":"pmed.1004369.ref010","doi-asserted-by":"crossref","first-page":"1158","DOI":"10.1016\/j.jdiacomp.2017.04.004","article-title":"Predicting the 6-month risk of severe hypoglycemia among adults with diabetes: Development and external validation of a prediction model","volume":"31","author":"EB Schroeder","year":"2017","journal-title":"J Diabetes Complications"},{"key":"pmed.1004369.ref011","doi-asserted-by":"crossref","first-page":"1545","DOI":"10.2147\/CLEP.S169835","article-title":"Development and validation of a risk prediction model for severe hypoglycemia in adult patients with type 2 diabetes: a nationwide population-based cohort study","volume":"10","author":"K Han","year":"2018","journal-title":"Clin Epidemiol"},{"issue":"1","key":"pmed.1004369.ref012","doi-asserted-by":"crossref","first-page":"e000527","DOI":"10.1136\/bmjdrc-2018-000527","article-title":"Development of a model to predict 5-year risk of severe hypoglycemia in patients with type 2 diabetes","volume":"6","author":"LS Chow","year":"2018","journal-title":"BMJ Open Diabetes Res Care"},{"issue":"1","key":"pmed.1004369.ref013","doi-asserted-by":"crossref","first-page":"107490","DOI":"10.1016\/j.jdiacomp.2019.107490","article-title":"Risk prediction for severe hypoglycemia in a type 2 diabetes population with previous non-severe hypoglycemia","volume":"34","author":"AD Misra-Hebert","year":"2020","journal-title":"J Diabetes Complications"},{"issue":"3","key":"pmed.1004369.ref014","doi-asserted-by":"crossref","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","article-title":"The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets","volume":"10","author":"T Saito","year":"2015","journal-title":"PLoS ONE"},{"issue":"7","key":"pmed.1004369.ref015","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.2337\/dc19-1743","article-title":"Predicting the Risk of Inpatient Hypoglycemia With Machine Learning Using Electronic Health Records","volume":"43","author":"Y Ruan","year":"2020","journal-title":"Diabetes Care"},{"issue":"2","key":"pmed.1004369.ref016","doi-asserted-by":"crossref","DOI":"10.3390\/s21020546","article-title":"Machine Learning Techniques for Hypoglycemia Prediction: Trends and Challenges","volume":"21","author":"O Mujahid","year":"2021","journal-title":"Sensors (Basel)"},{"issue":"Pt 1","key":"pmed.1004369.ref017","first-page":"307","article-title":"Principles-based medical informatics for success\u2014how Hong Kong built one of the world\u2019s largest integrated longitudinal electronic patient records","volume":"129","author":"NT Cheung","year":"2007","journal-title":"Stud Health Technol Inform"},{"issue":"11","key":"pmed.1004369.ref018","doi-asserted-by":"crossref","first-page":"2022","DOI":"10.2337\/dci19-0003","article-title":"From Hong Kong Diabetes Register to JADE Program to RAMP-DM for data-driven actions","volume":"42","author":"JC Chan","year":"2019","journal-title":"Diabetes Care"},{"issue":"6","key":"pmed.1004369.ref019","doi-asserted-by":"crossref","first-page":"423","DOI":"10.4093\/dmj.2016.40.6.423","article-title":"Risk Factors and Adverse Outcomes of Severe Hypoglycemia in Type 2 Diabetes Mellitus","volume":"40","author":"JS Yun","year":"2016","journal-title":"Diabetes Metab J"},{"key":"pmed.1004369.ref020","doi-asserted-by":"crossref","first-page":"1044059","DOI":"10.3389\/fpubh.2023.1044059","article-title":"Data-based modeling for hypoglycemia prediction: Importance, trends, and implications for clinical practice","volume":"11","author":"L Zhang","year":"2023","journal-title":"Front Public Health"},{"issue":"7","key":"pmed.1004369.ref021","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1007\/s00125-021-05444-0","article-title":"Derivation and validation of a machine learning risk score using biomarker and electronic patient data to predict progression of diabetic kidney disease","volume":"64","author":"L Chan","year":"2021","journal-title":"Diabetologia"},{"issue":"1","key":"pmed.1004369.ref022","doi-asserted-by":"crossref","first-page":"e22458","DOI":"10.2196\/22458","article-title":"Ability of Current Machine Learning Algorithms to Predict and Detect Hypoglycemia in Patients With Diabetes Mellitus: Meta-analysis","volume":"6","author":"S Kodama","year":"2021","journal-title":"JMIR Diabetes"},{"key":"pmed.1004369.ref023","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1002\/9781119654834.ch15","article-title":"Hands-on H2O machine learning tool","author":"UN Dulhare","year":"2020","journal-title":"Machine Learning and Big Data: Concepts, Algorithms, Tools and Applications"},{"issue":"1","key":"pmed.1004369.ref024","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1093\/ckj\/sfaa188","article-title":"External validation of prognostic models: what, why, how, when and where?","volume":"14","author":"CL Ramspek","year":"2021","journal-title":"Clin Kidney J"},{"key":"pmed.1004369.ref025","first-page":"101","article-title":"In defense of one-vs-all classification","volume":"5","author":"R Rifkin","year":"2004","journal-title":"J Mach Learn Res"},{"key":"pmed.1004369.ref026","author":"S Lundberg","year":"2017","journal-title":"A Unified Approach to Interpreting Model Predictions"},{"key":"pmed.1004369.ref027","doi-asserted-by":"crossref","first-page":"108388","DOI":"10.1016\/j.diabres.2020.108388","article-title":"Artificial Intelligence (AI) based machine learning models predict glucose variability and hypoglycaemia risk in patients with type 2 diabetes on a multiple drug regimen who fast during ramadan (The PROFAST\u2014IT Ramadan study)","volume":"169","author":"T Elhadd","year":"2020","journal-title":"Diabetes Res Clin Pract"},{"issue":"1","key":"pmed.1004369.ref028","doi-asserted-by":"crossref","first-page":"e2030913","DOI":"10.1001\/jamanetworkopen.2020.30913","article-title":"Development and Validation of a Machine Learning Model to Predict Near-Term Risk of Iatrogenic Hypoglycemia in Hospitalized Patients","volume":"4","author":"NN Mathioudakis","year":"2021","journal-title":"JAMA Netw Open"},{"issue":"12","key":"pmed.1004369.ref029","doi-asserted-by":"crossref","first-page":"2248","DOI":"10.1111\/dom.14208","article-title":"Development of a hypoglycaemia risk score to identify high-risk individuals with advanced type 2 diabetes in DEVOTE","volume":"22","author":"S Heller","year":"2020","journal-title":"Diabetes Obes Metab"},{"key":"pmed.1004369.ref030","article-title":"Feature-Based Machine Learning Model for Real-Time Hypoglycemia Prediction","author":"D Dave","year":"2020","journal-title":"J Diabetes Sci Technol"},{"key":"pmed.1004369.ref031","first-page":"e3426","article-title":"Identifying patients at increased risk of hypoglycaemia in primary care: Development of a machine learning-based screening tool","author":"S Crutzen","year":"2020","journal-title":"Diabetes Metab Res Rev"},{"issue":"10072","key":"pmed.1004369.ref032","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1016\/S0140-6736(17)30402-6","article-title":"Non-communicable disease syndemics: poverty, depression, and diabetes among low-income populations","volume":"389","author":"E Mendenhall","year":"2017","journal-title":"Lancet"},{"issue":"2","key":"pmed.1004369.ref033","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1353\/hpu.2014.0106","article-title":"Low socioeconomic status is associated with increased risk for hypoglycemia in diabetes patients: the Diabetes Study of Northern California (DISTANCE).","volume":"25","author":"SA Berkowitz","year":"2014","journal-title":"J Health Care Poor Underserved"},{"issue":"1","key":"pmed.1004369.ref034","doi-asserted-by":"crossref","first-page":"e2143597","DOI":"10.1001\/jamanetworkopen.2021.43597","article-title":"Association of Area-Level Socioeconomic Deprivation With Hypoglycemic and Hyperglycemic Crises in US Adults With Diabetes","volume":"5","author":"SS Kurani","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"12","key":"pmed.1004369.ref035","doi-asserted-by":"crossref","first-page":"1209","DOI":"10.1016\/j.jacc.2009.10.053","article-title":"Atorvastatin Causes Insulin Resistance and Increases Ambient Glycemia in Hypercholesterolemic Patients","volume":"55","author":"KK Koh","year":"2010","journal-title":"J Am Coll Cardiol"},{"issue":"3","key":"pmed.1004369.ref036","doi-asserted-by":"crossref","first-page":"e236315","DOI":"10.1001\/jamanetworkopen.2023.6315","article-title":"Validation of a Hypoglycemia Risk Stratification Tool Using Data From Continuous Glucose Monitors","volume":"6","author":"AJ Karter","year":"2023","journal-title":"JAMA Netw Open"},{"issue":"8","key":"pmed.1004369.ref037","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.2337\/dci23-0021","article-title":"The Growing Role of Technology in the Care of Older Adults With Diabetes","volume":"46","author":"ES Huang","year":"2023","journal-title":"Diabetes Care"},{"issue":"23","key":"pmed.1004369.ref038","doi-asserted-by":"crossref","first-page":"2397","DOI":"10.1001\/jama.2020.6928","article-title":"Effect of Continuous Glucose Monitoring on Hypoglycemia in Older Adults With Type 1 Diabetes: A Randomized Clinical Trial","volume":"323","author":"RE Pratley","year":"2020","journal-title":"JAMA"}],"container-title":["PLOS Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmed.1004369","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,12]],"date-time":"2024-04-12T17:21:13Z","timestamp":1712942473000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmed.1004369"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,12]]},"references-count":38,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,4,12]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pmed.1004369","relation":{},"ISSN":["1549-1676"],"issn-type":[{"value":"1549-1676","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,12]]}}}