{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T15:49:56Z","timestamp":1791560996738,"version":"4.3.4"},"reference-count":27,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Machine learning models are increasingly used to predict patients at risk of high health care usage for targeted interventions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to evaluate the predictive performance of multiclass ensemble models across different levels of health care usage and assess their potential application through real-world economic impact analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A total of 4 previously developed binary classification models (base learners)\u2014boosted trees, multivariate adaptive regression splines, multilayer perceptron, and logistic regression\u2014were extended using a stacking ensemble approach. These base learner models generated individual-level predicted probabilities, which were used as inputs to build multiclass prediction models forecasting usage across defined strata: length of stay (LOS) of &lt;7, 7-13, 14-29, and \u226530 days, and emergency department (ED) visits of &lt;3, 3-4, 5-9, and \u226510 visits. In total, 3 ensemble algorithms were evaluated: random forest, boosted trees, and linear support vector machines. Ensemble models were trained on registry data from 2020-2021 and temporally validated on 2021-2022 data. Performance was assessed using multiclass area under the receiver operating curve, accuracy, and confusion matrix\u2013derived metrics. Economic impact was estimated via Monte Carlo simulations using inpatient billing data, assuming a 20% cost reduction in the following year.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The models were trained on 108,886 patients and validated on 111,004 patients. Among all ensemble configurations, boosted tree models regardless of base learner achieved the highest performance, with multiclass area under the receiver operating curve scores of 0.6877 (95% CI 0.6927-0.7255) for LOS and 0.7601 (95% CI 0.7301-0.7654) for ED visits, and corresponding accuracies of 0.6522 (95% CI 0.6465-0.6579) and 0.7457 (95% CI 0.7405-0.7508), respectively. In the validation set, these models correctly assigned 30.3% of inpatient LOS and 39.8% of ED visits to the correct class, identifying 77% of future inpatient users and 73.9% of future ED users. Economic impact analysis for LOS identified the boosted tree model with logistic regression base learner as dominant, achieving a simulated average cost reduction of SGD $152 million (US $111 million), SGD $2.4 million (US $1.75 million; 1.5%) more than the next best model using a multilayer perceptron base learner.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Ensemble models can effectively predict multilevel health care usage and potentially generate meaningful cost savings when applied to real-world settings. These models may support targeted interventions and guide planning and budgeting in diabetes-related population health programs.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/77202","type":"journal-article","created":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T20:15:09Z","timestamp":1771618509000},"page":"e77202","source":"Crossref","is-referenced-by-count":3,"title":["Ensemble Machine Learning Models for Predicting Patients With High Usage: Model Validation and Economic Impact Analysis"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4518-0477","authenticated-orcid":false,"given":"Joshua Kuan","family":"Tan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-3306-3000","authenticated-orcid":false,"given":"Le","family":"Quan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8904-8936","authenticated-orcid":false,"given":"Hao Yi","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3758-9982","authenticated-orcid":false,"given":"Su-Yen","family":"Goh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6712-5535","authenticated-orcid":false,"given":"Julian","family":"Thumboo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3653-0962","authenticated-orcid":false,"given":"Marianne","family":"Au","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5482-2646","authenticated-orcid":false,"given":"Yong Mong","family":"Bee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7785-2987","authenticated-orcid":false,"given":"Jen Hong","family":"Tan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,2,20]]},"reference":[{"key":"ref1","unstructured":"OECD Health Statistics 2023 - OECDOECD2024-01-19https:\/\/www.oecd.org\/health\/health-data.htm"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.2337\/dci23-0085"},{"key":"ref3","unstructured":"MEPs Mobilising for Diabetes Blueprint for Action on Diabetes in the European Union by 2030MEPs Mobilising for Diabetes2024-09-25https:\/\/www.mepinterestgroupdiabetes.eu\/wp-content\/uploads\/2021\/03\/MMD-BLUEPRINT-FOR-AC TION-ON-DIABETES.pdf"},{"key":"ref4","unstructured":"Percentage of spending on mental health, diabetes and ageing versus total healthcare spending in 2018 against amount budgeted for 2019Ministry of Health Singapore20192024-09-25https:\/\/www.moh.gov.sg\/news-highlights\/details\/percentage-of -spending-on-mental-health-diabetes-and-ageing-versus-total-healthcare-spending-in-2018-against-amount-budgeted-for-2019\/"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2018-023113"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-021-07333-z"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1097\/EDE.0b013e3181c30fb2"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1093\/eurheartj\/ehu207"},{"key":"ref9","unstructured":"MadvigPPearlRManaging the Most Expensive PatientsHarv Bus Rev 2020;5(February)2020-02-05https:\/\/hbr.org\/2020\/01\/managing-the-most-expensive-patients"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-023-09957-9"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.2196\/58463"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.2147\/CLEP.S300663"},{"key":"ref13","unstructured":"MaxKJedWSteveWAndreWChrisKcaret: Classification and Regression Training2026-01-17https:\/\/cran.r-project.org\/web\/packages\/caret\/caret.pdf"},{"key":"ref14","author":"Zhi-Hua, Z","year":"2012","journal-title":"Ensemble Methods: Foundations and Algorithms 1st Ed"},{"key":"ref15","unstructured":"MaxKHadleyWTidymodels: a collection of packages for modeling and machine learning using tidyverse principles2026-01-17https:\/\/www.tidymodels.org\/"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h1258"},{"key":"ref17","year":"2015","journal-title":"Collaborative Data Science"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2023-078378"},{"key":"ref19","unstructured":"Monte-Carlo-Simulations-using-Ensemble-Model-for-Population-HealthGithub2026-01-26https:\/\/joshuatankuan.github.io\/Monte-Carlo-Simulations-using-Ensemble-Model-for-Population-Health\/"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2022-062786"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-019-4239-2"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1377\/hlthaff.2014.1186"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1016\/j.health.2025.100411"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0000000000001807"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.108651"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.7326\/M21-4562"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00970-0"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e77202","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:09:18Z","timestamp":1782842958000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e77202"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,20]]},"references-count":27,"URL":"https:\/\/doi.org\/10.2196\/77202","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,20]]}}}