{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T06:49:44Z","timestamp":1791355784820,"version":"4.3.1"},"reference-count":26,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T00:00:00Z","timestamp":1791331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Risk stratification at emergency department triage may help identify patients who require closer assessment or monitoring. However, prediction models that rely only on information available at the triage stage remain less well evaluated. This study aimed to develop and internally validate machine learning models based on triage-stage information for predicting short-term critical outcomes in adult emergency department patients and to interpret a representative explainable model using SHAP analysis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This retrospective observational study included 1,842 consecutive adult patients who completed standardized electronic triage and subsequently entered an emergency observation or hospital-based care pathway at a tertiary general hospital between January 2022 and December 2025. The prediction time point and landmark were defined as completion of electronic emergency triage. Only patients who were free of all outcome components at the landmark were included, and candidate predictors were restricted to routinely available information recorded no later than triage completion. The primary outcome was the first critical event occurring during the subsequent 72\u2005h: intensive care unit admission, endotracheal intubation or invasive mechanical ventilation, cardiopulmonary resuscitation, objectively documented escalation involving an implemented resuscitation-level intervention, or in-hospital death. Additional sensitivity analyses restricted the outcome to intensive care unit admission, intubation or invasive ventilation, cardiopulmonary resuscitation, or death and refitted the model without emergency triage level. The cohort was randomly divided into a development cohort and a 30% hold-out cohort. Because the hold-out cohort had already been examined during the original analysis, it was not considered an independent validation set. The primary internal-validation analysis therefore consisted of 10 repetitions of nested cross-validation within the development cohort, using five stratified outer folds for performance estimation and four stratified inner folds for preprocessing and hyperparameter tuning. Logistic regression, random forest, support vector machine, XGBoost, and LightGBM were evaluated using the same resampling framework. The previously examined exploratory hold-out cohort was retained only for secondary exploratory analysis. Model performance was assessed using discrimination, calibration, classification metrics, Brier score, decision-curve analysis, and SHAP-based interpretation. Bootstrap optimism correction was additionally used to assess overfitting.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>All 1,842 included patients were event-free at triage completion, and 214 experienced a short-term critical outcome during the subsequent 72-hour period, corresponding to an event rate of 11.6%. The development cohort included 1,289 patients with 150 events, and the previously examined exploratory hold-out cohort included 553 patients with 64 events. In 10 repetitions of nested cross-validation, the mean AUCs were 0.821\u2009\u00b1\u20090.012 for logistic regression, 0.817\u2009\u00b1\u20090.015 for random forest, 0.752\u2009\u00b1\u20090.018 for support vector machine, 0.831\u2009\u00b1\u20090.013 for XGBoost, and 0.833\u2009\u00b1\u20090.014 for LightGBM. XGBoost was highest ranked in 42% of repeated outer-validation results, LightGBM in 48%, and logistic regression in 10%. These primary internal-validation results supported broadly comparable performance among XGBoost, LightGBM, and logistic regression rather than consistent superiority of one algorithm. In the secondary exploratory analysis, the representative XGBoost model achieved an AUC of 0.828 in the previously examined exploratory hold-out cohort. At the development-derived threshold of 0.106, its accuracy was 0.770, sensitivity 0.656, specificity 0.785, positive predictive value 0.286, negative predictive value 0.946, F1 score 0.398, and Brier score 0.080. Its paired comparisons, sensitivity analyses, calibration, decision-curve analysis, and SHAP findings in this cohort were interpreted as exploratory.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Repeated nested cross-validation showed broadly comparable discrimination for XGBoost, LightGBM, and logistic regression in predicting post-triage short-term critical outcomes. XGBoost was retained as a representative explainable model, whereas its performance and benchmark comparisons in the previously examined exploratory hold-out cohort should be regarded as secondary exploratory analyses. The study does not establish the superiority or independent validation of XGBoost. Evaluation in a genuinely unused temporal cohort and external prospective validation remain necessary before clinical implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1921692","type":"journal-article","created":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T05:53:14Z","timestamp":1791352394000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Development and internal validation of triage-stage prediction models for short-term critical outcomes among adult emergency department patients entering observation or hospital-based care: a retrospective study"],"prefix":"10.3389","volume":"13","author":[{"given":"Mingwei","family":"Wu","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, The Affiliated Hospital, Southwest Medical University","place":["Luzhou, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yao","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Dermatology, The Affiliated Hospital, Southwest Medical University","place":["Luzhou, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengli","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, The Affiliated Hospital, Southwest Medical University","place":["Luzhou, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jinfeng","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, The Affiliated Hospital, Southwest Medical University","place":["Luzhou, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,10,7]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e258498","DOI":"10.1001\/jamanetworkopen.2025.8498","article-title":"Emergency department triage accuracy and delays in care for high-risk conditions","volume":"8","author":"Sax","year":"2025","journal-title":"JAMA Netw Open"},{"key":"B2","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1177\/15533506241273449","article-title":"Advancing emergency department triage prediction with machine learning to optimize triage for abdominal pain surgery patients","volume":"31","author":"Chai","year":"2024","journal-title":"Surg Innov"},{"key":"B3","doi-asserted-by":"publisher","first-page":"31212","DOI":"10.1038\/s41598-025-17180-1","article-title":"Development and internal validation of an AI-based emergency triage model for predicting critical outcomes in emergency department","volume":"15","author":"Sitthiprawiat","year":"2025","journal-title":"Sci Rep"},{"key":"B4","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1186\/s13054-019-2351-7","article-title":"Emergency department triage prediction of clinical outcomes using machine learning models","volume":"23","author":"Raita","year":"2019","journal-title":"Crit Care"},{"key":"B5","doi-asserted-by":"publisher","first-page":"23","DOI":"10.4258\/hir.2025.31.1.23","article-title":"Triage data-driven prediction models for hospital admission of emergency department patients: a systematic review","volume":"31","author":"Shin","year":"2025","journal-title":"Healthc Inform Res"},{"key":"B6","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/s12245-025-00861-z","article-title":"Predicting triage of pediatric patients in the emergency department using machine learning approach","volume":"18","author":"Halwani","year":"2025","journal-title":"Int J Emerg Med"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e30770","DOI":"10.2196\/30770","article-title":"Prediction of critical care outcome for adult patients presenting to emergency department using initial triage information: an XGBoost algorithm analysis","volume":"9","author":"Yun","year":"2021","journal-title":"JMIR Med Inform"},{"key":"B8","doi-asserted-by":"publisher","first-page":"10537","DOI":"10.1038\/s41598-022-14422-4","article-title":"Machine learning-based suggestion for critical interventions in the management of potentially severe conditioned patients in emergency department triage","volume":"12","author":"Chang","year":"2022","journal-title":"Sci Rep"},{"key":"B9","doi-asserted-by":"publisher","first-page":"e259639","DOI":"10.1001\/jamanetworkopen.2025.9639","article-title":"A machine learning trauma triage model for critical care transport","volume":"8","author":"Weidman","year":"2025","journal-title":"JAMA Netw Open"},{"key":"B10","doi-asserted-by":"publisher","first-page":"e074820","DOI":"10.1136\/bmj-2023-074820","article-title":"Evaluation of clinical prediction models (part 2): how to undertake an external validation study","volume":"384","author":"Riley","year":"2024","journal-title":"Br Med J"},{"key":"B11","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1186\/s41512-023-00148-y","article-title":"Decision curve analysis: confidence intervals and hypothesis testing for net benefit","volume":"7","author":"Vickers","year":"2023","journal-title":"Diagn Progn Res"},{"key":"B12","doi-asserted-by":"publisher","first-page":"354","DOI":"10.15441\/ceem.23.145","article-title":"Explainable artificial intelligence in emergency medicine: an overview","volume":"10","author":"Okada","year":"2023","journal-title":"Clin Exp Emerg Med"},{"key":"B13","doi-asserted-by":"publisher","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"Br Med J"},{"key":"B14","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1186\/1757-7241-20-28","article-title":"Abnormal vital signs are strong predictors for intensive care unit admission and in-hospital mortality in adults triaged in the emergency department - a prospective cohort study","volume":"20","author":"Barfod","year":"2012","journal-title":"Scand J Trauma Resusc Emerg Med"},{"key":"B15","doi-asserted-by":"publisher","first-page":"453","DOI":"10.5811\/westjem.2013.5.13411","article-title":"Predictive value of initial triage vital signs for critically ill older adults","volume":"14","author":"Lamantia","year":"2013","journal-title":"West J Emerg Med"},{"key":"B16","doi-asserted-by":"publisher","first-page":"109876","DOI":"10.1016\/j.resuscitation.2023.109876","article-title":"Predicting ICU admission and death in the emergency department: a comparison of six early warning scores","volume":"190","author":"Covino","year":"2023","journal-title":"Resuscitation"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2441","DOI":"10.1007\/s10096-024-04961-1","article-title":"Validation of MEWS, NEWS, NEWS-2 and qSOFA for different infection foci at the emergency department, the acutelines cohort","volume":"43","author":"Hincapi\u00e9-Osorno","year":"2024","journal-title":"Eur J Clin Microbiol Infect Dis"},{"key":"B18","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1186\/s12873-024-01135-2","article-title":"Improving triage performance in emergency departments using machine learning and natural language processing: a systematic review","volume":"24","author":"Porto","year":"2024","journal-title":"BMC Emerg Med"},{"key":"B19","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1186\/s12873-024-01152-1","article-title":"Using machine learning and natural language processing in triage for prediction of clinical disposition in the emergency department","volume":"24","author":"Chang","year":"2024","journal-title":"BMC Emerg Med"},{"key":"B20","doi-asserted-by":"publisher","first-page":"198","DOI":"10.5455\/medarh.2024.78.198-206","article-title":"Applications of artificial intelligence and machine learning in emergency medicine triage - A systematic review","volume":"78","author":"Almulihi","year":"2024","journal-title":"Med Arch"},{"key":"B21","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1186\/s12911-024-02793-9","article-title":"Integrating structured and unstructured data for predicting emergency severity: an association and predictive study using transformer-based natural language processing models","volume":"24","author":"Zhang","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e0284315","DOI":"10.1371\/journal.pone.0284315","article-title":"Interpretable machine learning with tree-based shapley additive explanations: application to metabolomics datasets for binary classification","volume":"18","author":"Bifarin","year":"2023","journal-title":"PLoS One"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1129380","DOI":"10.3389\/fonc.2023.1129380","article-title":"On the importance of interpretable machine learning predictions to inform clinical decision making in oncology","volume":"13","author":"Lu","year":"2023","journal-title":"Front Oncol"},{"key":"B24","doi-asserted-by":"publisher","first-page":"2244","DOI":"10.3390\/healthcare11162244","article-title":"Optimizing clinical decision making with decision curve analysis: insights for clinical investigators","volume":"11","author":"Piovani","year":"2023","journal-title":"Healthcare (Basel)"},{"key":"B25","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1186\/s41512-022-00136-8","article-title":"Targeted validation: validating clinical prediction models in their intended population and setting","volume":"6","author":"Sperrin","year":"2022","journal-title":"Diagn Progn Res"},{"key":"B26","doi-asserted-by":"publisher","first-page":"e074819","DOI":"10.1136\/bmj-2023-074819","article-title":"Evaluation of clinical prediction models (part 1): from development to external validation","volume":"384","author":"Collins","year":"2024","journal-title":"Br Med J"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1921692\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T05:53:22Z","timestamp":1791352402000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1921692\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,7]]},"references-count":26,"alternative-id":["10.3389\/fmed.2026.1921692"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1921692","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,7]]},"article-number":"1921692"}}