{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T17:21:10Z","timestamp":1793380870778,"version":"4.5.0"},"reference-count":42,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T00:00:00Z","timestamp":1751414400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T00:00:00Z","timestamp":1751414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003710","name":"Korea Health Industry Development Institute","doi-asserted-by":"crossref","award":["RS-2024-00335937"],"award-info":[{"award-number":["RS-2024-00335937"]}],"id":[{"id":"10.13039\/501100003710","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003710","name":"Korea Health Industry Development Institute","doi-asserted-by":"crossref","award":["RS-2023-KH134607"],"award-info":[{"award-number":["RS-2023-KH134607"]}],"id":[{"id":"10.13039\/501100003710","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100024838","name":"Samsung Medical Center","doi-asserted-by":"crossref","award":["SMX1240281"],"award-info":[{"award-number":["SMX1240281"]}],"id":[{"id":"10.13039\/501100024838","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This study aimed to develop and validate a transformer-based early warning score (TEWS) system for predicting adverse events (AEs) in the emergency department (ED). We conducted a retrospective study analyzing adult ED visits at a tertiary hospital. The TEWS was developed to predict five AEs within 24\u00a0h: vasopressor use, respiratory support, intensive care unit admission, septic shock, and cardiac arrest. Performance was evaluated and compared using the area under the receiver operating characteristic curve (AUROC) and bootstrap-based t-test. External validation was performed using the Marketplace for Medical Information in Intensive Care (MIMIC)-IV-ED database. Transfer learning was applied using 1% and 5% of the external data. A total of 414,748 patients was analyzed in the development cohort (AEs, 3.7%), and 410,880 patients (AEs, 6.7%) were included in the external validation cohort. Compared to the modified early warning score (MEWS), the TEWS incorporating 13 variables and the vital signs-only TEWS demonstrated superior prognostic performance across all AEs. The AUROC ranged from 0.833 to 0.936 for TEWS and 0.688 to 0.874 for MEWS. In external validation, the TEWS also showed acceptable discrimination with AUROC values of 0.759 to 0.905. Transfer learning significantly improved the performance, increasing AUROC values to 0.846\u20130.911. The TEWS system was successfully integrated into the electronic health record (EHR) system of the study hospital, providing real-time risk assessment for ED patients. We developed and validated an artificial intelligence-based early warning score system that predicts multiple adverse outcomes in the ED and was successfully integrated into the EHR system.<\/jats:p>","DOI":"10.1038\/s41598-025-07511-7","type":"journal-article","created":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T22:31:06Z","timestamp":1751409066000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Development and validation of a transformer model-based early warning score for real-time prediction of adverse outcomes in the emergency department"],"prefix":"10.1038","volume":"15","author":[{"given":"Hansol","family":"Chang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jong Eun","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daehwan","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kiwon","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Se Yong","family":"Jekal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ki Tae","family":"Moon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sejin","family":"Heo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Doyeop","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gun Tak","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sung Yeon","family":"Hwang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Won Chul","family":"Cha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wonhee","family":"Kim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tae Ho","family":"Lim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tae Gun","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,7,2]]},"reference":[{"issue":"7","key":"7511_CR1","doi-asserted-by":"publisher","first-page":"1023","DOI":"10.1016\/j.ajem.2017.02.012","volume":"35","author":"BA Lentz","year":"2017","unstructured":"Lentz, B. A. et al. Validity of ED: addressing heterogeneous definitions of over-triage and under-triage. Am. J. Emerg. Med. 35(7), 1023\u20131025 (2017).","journal-title":"Am. J. Emerg. Med."},{"issue":"3","key":"7511_CR2","doi-asserted-by":"publisher","first-page":"384","DOI":"10.1513\/AnnalsATS.201611-905OC","volume":"14","author":"L Chen","year":"2017","unstructured":"Chen, L. et al. Dynamic and personalized risk forecast in Step-Down units. Implications for monitoring paradigms. Ann. Am. Thorac. Soc. 14(3), 384\u2013391 (2017).","journal-title":"Ann. Am. Thorac. Soc."},{"issue":"50","key":"7511_CR3","first-page":"892","volume":"107","author":"M Christ","year":"2010","unstructured":"Christ, M., Grossmann, F., Winter, D., Bingisser, R. & Platz, E. Modern triage in the emergency department. Dtsch. Arztebl Int. 107(50), 892\u2013898 (2010).","journal-title":"Dtsch. Arztebl Int."},{"issue":"3","key":"7511_CR4","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/j.annemergmed.2006.05.019","volume":"49","author":"KV Iserson","year":"2007","unstructured":"Iserson, K. V. & Moskop, J. C. Triage in medicine, part I: concept, history, and types. Ann. Emerg. Med. 49(3), 275\u2013281 (2007).","journal-title":"Ann. Emerg. Med."},{"issue":"10","key":"7511_CR5","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1093\/qjmed\/94.10.521","volume":"94","author":"CP Subbe","year":"2001","unstructured":"Subbe, C. P., Kruger, M., Rutherford, P. & Gemmel, L. Validation of a modified early warning score in medical admissions. QJM 94(10), 521\u2013526 (2001).","journal-title":"QJM"},{"issue":"40","key":"7511_CR6","doi-asserted-by":"publisher","first-page":"e27325","DOI":"10.1097\/MD.0000000000027325","volume":"100","author":"P Vergara","year":"2021","unstructured":"Vergara, P. et al. Validation of the National early warning score (NEWS)-2 for adults in the emergency department in a tertiary-level clinic in colombia: cohort study. Med. (Baltim). 100(40), e27325 (2021).","journal-title":"Med. (Baltim)."},{"issue":"1","key":"7511_CR7","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1055\/s-0039-1677903","volume":"28","author":"F Magrabi","year":"2019","unstructured":"Magrabi, F. et al. Artificial intelligence in clinical decision support: challenges for evaluating AI and practical implications. Yearb Med. Inf. 28(1), 128\u2013134 (2019).","journal-title":"Yearb Med. Inf."},{"key":"7511_CR8","doi-asserted-by":"publisher","first-page":"101762","DOI":"10.1016\/j.artmed.2019.101762","volume":"102","author":"M Fernandes","year":"2020","unstructured":"Fernandes, M. et al. Clinical decision support systems for triage in the emergency department using intelligent systems: a review. Artif. Intell. Med. 102, 101762 (2020).","journal-title":"Artif. Intell. Med."},{"issue":"4","key":"7511_CR9","doi-asserted-by":"publisher","first-page":"354","DOI":"10.15441\/ceem.23.145","volume":"10","author":"Y Okada","year":"2023","unstructured":"Okada, Y., Ning, Y. & Ong, M. E. H. Explainable artificial intelligence in emergency medicine: an overview. Clin. Exp. Emerg. Med. 10(4), 354\u2013362 (2023).","journal-title":"Clin. Exp. Emerg. Med."},{"issue":"2","key":"7511_CR10","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1007\/s10877-016-9849-1","volume":"31","author":"A Belard","year":"2017","unstructured":"Belard, A. et al. Precision diagnosis: a view of the clinical decision support systems (CDSS) landscape through the lens of critical care. J. Clin. Monit. Comput. 31(2), 261\u2013271 (2017).","journal-title":"J. Clin. Monit. Comput."},{"issue":"4","key":"7511_CR11","first-page":"77","volume":"31","author":"D Lee","year":"2016","unstructured":"Lee, D. et al. Trends of clinical decision support system (CDSS). Electron. Telecommunications Trends. 31(4), 77\u201385 (2016).","journal-title":"Electron. Telecommunications Trends"},{"issue":"3","key":"7511_CR12","doi-asserted-by":"publisher","first-page":"165","DOI":"10.15441\/ceem.22.366","volume":"9","author":"H Chang","year":"2022","unstructured":"Chang, H. & Cha, W. C. Artificial intelligence decision points in an emergency department. Clin. Experimental Emerg. Med. 9(3), 165\u2013168 (2022).","journal-title":"Clin. Experimental Emerg. Med."},{"issue":"1","key":"7511_CR13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-021-99269-x","volume":"12","author":"H Chang","year":"2022","unstructured":"Chang, H., Yu, J. Y., Yoon, S., Kim, T. & Cha, W. C. Machine learning-based suggestion for critical interventions in the management of potentially severe conditioned patients in emergency department triage. Sci. Rep. 12(1), 1\u201310 (2022).","journal-title":"Sci. Rep."},{"key":"7511_CR14","first-page":"11","volume":"1","author":"JY Yu","year":"2022","unstructured":"Yu, J. Y. et al. Predicting mid-term survival of patients during emergency department triage for resuscitation decision. Signa Vitae. 1, 11 (2022).","journal-title":"Signa Vitae"},{"issue":"6","key":"7511_CR15","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1038\/s41551-023-01045-x","volume":"7","author":"H-Y Zhou","year":"2023","unstructured":"Zhou, H-Y. et al. A transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics. Nat. Biomedical Eng. 7(6), 743\u2013755 (2023).","journal-title":"Nat. Biomedical Eng."},{"key":"7511_CR16","unstructured":"Vaswani, A. Attention is all you need. Advances Neural Inform. Process. Systems (2017)."},{"key":"7511_CR17","unstructured":"Devlin, J. Bert: Pre-training of deep bidirectional Transformers for Language Understanding. ArXiv Preprint arXiv 181004805 (2018)."},{"key":"7511_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.dsp.2017.10.011","volume":"73","author":"G Montavon","year":"2018","unstructured":"Montavon, G., Samek, W. & M\u00fcller, K-R. Methods for interpreting and Understanding deep neural networks. Digit. Signal Proc. 73, 1\u201315 (2018).","journal-title":"Digit. Signal Proc."},{"key":"7511_CR19","unstructured":"Rubanova, Y., Chen, R. T. & Duvenaud, D. K. Latent ordinary differential equations for irregularly-sampled time series. Adv. Neural Inform. Process. Syst. 32  (2019)."},{"key":"7511_CR20","doi-asserted-by":"crossref","unstructured":"Zerveas, G., Jayaraman, S., Patel, D., Bhamidipaty, A. & Eickhoff, C. A transformer-based framework for multivariate time series representation learning. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining 2114\u20132124  (2021).","DOI":"10.1145\/3447548.3467401"},{"key":"7511_CR21","first-page":"27730","volume":"35","author":"L Ouyang","year":"2022","unstructured":"Ouyang, L. et al. Training Language models to follow instructions with human feedback. Adv. Neural. Inf. Process. Syst. 35, 27730\u201327744 (2022).","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"7511_CR22","unstructured":"Choi, E. et al. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. Adv. Neural Inform. Process. Syst. 29  (2016)."},{"key":"7511_CR23","unstructured":"Lin, Z. et al. A structured self-attentive sentence embedding. ArXiv Preprint arXiv 170303130 (2017)."},{"issue":"1","key":"7511_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","volume":"10","author":"AEW Johnson","year":"2023","unstructured":"Johnson, A. E. W. et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data. 10(1), 1 (2023).","journal-title":"Sci. Data"},{"key":"7511_CR25","doi-asserted-by":"publisher","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","volume":"385","author":"GS Collins","year":"2024","unstructured":"Collins, G. S. et al. TRIPOD\u2009+\u2009AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 385, e078378 (2024).","journal-title":"BMJ"},{"issue":"8","key":"7511_CR26","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","volume":"315","author":"M Singer","year":"2016","unstructured":"Singer, M. et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA 315(8), 801\u2013810 (2016).","journal-title":"JAMA"},{"issue":"1","key":"7511_CR27","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1093\/jamia\/ocx084","volume":"25","author":"AE Johnson","year":"2018","unstructured":"Johnson, A. E., Stone, D. J., Celi, L. A. & Pollard, T. J. The MIMIC code repository: enabling reproducibility in critical care research. J. Am. Med. Inf. Assoc. 25(1), 32\u201339 (2018).","journal-title":"J. Am. Med. Inf. Assoc."},{"key":"7511_CR28","doi-asserted-by":"crossref","unstructured":"Selvaraju, R. R. et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In IEEE International Conference on Computer Vision (ICCV) 618\u2013626 (2017).","DOI":"10.1109\/ICCV.2017.74"},{"issue":"3","key":"7511_CR29","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/S0304-3800(02)00257-0","volume":"160","author":"M Gevrey","year":"2003","unstructured":"Gevrey, M., Dimopoulos, I. & Lek, S. Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecol. Model. 160(3), 249\u2013264 (2003).","journal-title":"Ecol. Model."},{"issue":"1","key":"7511_CR30","doi-asserted-by":"publisher","first-page":"60","DOI":"10.15441\/ceem.22.339","volume":"10","author":"S Ryu","year":"2023","unstructured":"Ryu, S., Oh, S. K., Lee, B. K. & Jeon, S. Y. A combination of the modified early warning score and the Korean triage and acuity scale as a triage tool in patients with infection. Clin. Exp. Emerg. Med. 10(1), 60\u201367 (2023).","journal-title":"Clin. Exp. Emerg. Med."},{"key":"7511_CR31","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.resuscitation.2021.04.013","volume":"163","author":"YJ Lee","year":"2021","unstructured":"Lee, Y. J. et al. A multicentre validation study of the deep learning-based early warning score for predicting in-hospital cardiac arrest in patients admitted to general wards. Resuscitation 163, 78\u201385 (2021).","journal-title":"Resuscitation"},{"issue":"10","key":"7511_CR32","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","volume":"22","author":"SJ Pan","year":"2010","unstructured":"Pan, S. J. & Yang, Q. A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345\u20131359 (2010).","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"7511_CR33","doi-asserted-by":"publisher","first-page":"106288","DOI":"10.1016\/j.cmpb.2021.106288","volume":"208","author":"F Cabitza","year":"2021","unstructured":"Cabitza, F. et al. The importance of being external. Methodological insights for the external validation of machine learning models in medicine. Comput. Methods Programs Biomed. 208, 106288 (2021).","journal-title":"Comput. Methods Programs Biomed."},{"issue":"2","key":"7511_CR34","doi-asserted-by":"publisher","first-page":"132","DOI":"10.15441\/ceem.23.041","volume":"10","author":"C-Y Kang","year":"2023","unstructured":"Kang, C-Y. & Yoon, J. H. Current challenges in adopting machine learning to critical care and emergency medicine. Clin. Exp. Emerg. Med. 10(2), 132\u2013137 (2023).","journal-title":"Clin. Exp. Emerg. Med."},{"key":"7511_CR35","doi-asserted-by":"publisher","first-page":"115310","DOI":"10.1016\/j.engstruct.2022.115310","volume":"276","author":"Q Li","year":"2023","unstructured":"Li, Q., Wang, Y., Shao, Y., Li, L. & Hao, H. A comparative study on the most effective machine learning model for blast loading prediction: from GBDT to transformer. Eng. Struct. 276, 115310 (2023).","journal-title":"Eng. Struct."},{"key":"7511_CR36","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1016\/j.ajem.2021.10.049","volume":"51","author":"H Ouyang","year":"2022","unstructured":"Ouyang, H., Wang, J., Sun, Z. & Lang, E. The impact of emergency department crowding on admission decisions and patient outcomes. Am. J. Emerg. Med. 51, 163\u2013168 (2022).","journal-title":"Am. J. Emerg. Med."},{"issue":"5","key":"7511_CR37","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1007\/s00134-023-07031-w","volume":"49","author":"TS Valley","year":"2023","unstructured":"Valley, T. S. et al. Hospital factors that influence ICU admission decision-making: a qualitative study of eight hospitals. Intensive Care Med. 49(5), 505\u2013516 (2023).","journal-title":"Intensive Care Med."},{"issue":"1","key":"7511_CR38","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1186\/s13054-022-03899-0","volume":"26","author":"JS You","year":"2022","unstructured":"You, J. S. et al. Relationship between time of emergency department admission and adherence to the surviving Sepsis campaign bundle in patients with septic shock. Crit. Care. 26(1), 43 (2022).","journal-title":"Crit. Care"},{"issue":"10","key":"7511_CR39","doi-asserted-by":"publisher","first-page":"2007","DOI":"10.1097\/00003246-200110000-00026","volume":"29","author":"RJ Brilli","year":"2001","unstructured":"Brilli, R. J. et al. Critical care delivery in the intensive care unit: defining clinical roles and the best practice model. Crit. Care Med. 29(10), 2007\u20132019 (2001).","journal-title":"Crit. Care Med."},{"key":"7511_CR40","doi-asserted-by":"crossref","unstructured":"Soares, J. et al. Intensive care unit admission criteria: A scoping review. J. Intensive Care Soc. 17511437241246901 (2024).","DOI":"10.1177\/17511437241246901"},{"issue":"2","key":"7511_CR41","doi-asserted-by":"publisher","first-page":"e1920733","DOI":"10.1001\/jamanetworkopen.2019.20733","volume":"3","author":"N Brajer","year":"2020","unstructured":"Brajer, N. et al. Prospective and external evaluation of a machine learning model to predict In-Hospital mortality of adults at time of admission. JAMA Netw. Open. 3(2), e1920733\u2013e1920733 (2020).","journal-title":"JAMA Netw. Open."},{"issue":"1","key":"7511_CR42","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.artmed.2007.09.005","volume":"42","author":"BH Cho","year":"2008","unstructured":"Cho, B. H. et al. Application of irregular and unbalanced data to predict diabetic nephropathy using visualization and feature selection methods. Artif. Intell. Med. 42(1), 37\u201353 (2008).","journal-title":"Artif. Intell. Med."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-07511-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-07511-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-07511-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T22:31:11Z","timestamp":1751409071000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-07511-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,2]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["7511"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-07511-7","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,2]]},"assertion":[{"value":"2 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"23021"}}