{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T12:57:31Z","timestamp":1790600251168,"version":"4.1.0"},"reference-count":41,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T00:00:00Z","timestamp":1790553600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Smartphone-based analysis of electrocardiograms (ECGs) and vital signs may support risk assessment without an integrated electronic health record feed. We evaluated MEDI-AI for 24 h mortality and unplanned intensive care unit (ICU) admission, with secondary diagnostic and triage assessments. Methods: This single-center retrospective evaluation included 2000 encounters from 2025 after supplementary development on 3000 encounters from 2024, with patient-level separation. Stored probabilities were assessed for area under the receiver-operating-characteristic curve (AUROC), calibration, and threshold performance. Post hoc exploratory NEWS2 and paired comparisons were performed; accepted inputs were also analyzed separately. Results: There were 189 deaths and 378 ICU admissions. Fusion AUROCs were 0.9994 and 0.9982; NEWS2 AUROCs were 0.8801 and 0.8131. Fusion-minus-vital-only AUROC differences were \u22120.000039 (95% CI, \u22120.000450 to 0.000366) and 0.002114 (0.000632 to 0.003920). Fusion calibration slopes were 5.62 and 6.09. STEMI and ACS sensitivities were 90.9% (58.7\u201399.8%) and 92.3% (74.9\u201399.1%). Triage critical-class recall was 24.9%. Input rejection occurred in 48 encounters; median recorded latency was 1235 ms. These stored-score estimates do not measure end-to-end deployable performance. Conclusions: Near-ceiling stored-score discrimination did not establish reliable absolute risks or a clinically usable workflow. Incomplete predictor-timing and model-development records limit reproducibility and assessment of bias. The evaluated triage configuration is unsuitable for clinical deployment, and the diagnostic analyses do not validate rule-out use. Independent external and prospective validation are required.<\/jats:p>","DOI":"10.3390\/diagnostics16193153","type":"journal-article","created":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T11:35:59Z","timestamp":1790595359000},"page":"3153","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["On-Device ECG and Vital-Sign Prediction of 24-Hour Mortality and Unplanned ICU Admission in Emergency Department Patients: A Retrospective Evaluation of MEDI-AI"],"prefix":"10.3390","volume":"16","author":[{"given":"Hyunjin","family":"Kyung","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam 13620, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2449-1006","authenticated-orcid":false,"given":"Dongbum","family":"Suh","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam 13620, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2385-2834","authenticated-orcid":false,"given":"Jin Hee","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam 13620, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0960-0198","authenticated-orcid":false,"given":"Hyuksool","family":"Kwon","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Seoul National University Bundang Hospital, Seongnam 13620, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,28]]},"reference":[{"key":"ref_1","unstructured":"Royal College of Physicians (2017). National Early Warning Score (NEWS) 2: Standardising the Assessment of Acute-Illness Severity in the NHS, RCP. Updated Report of a Working Party."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"109876","DOI":"10.1016\/j.resuscitation.2023.109876","article-title":"Predicting ICU admission and death in the Emergency Department: A comparison of six early warning scores","volume":"190","author":"Covino","year":"2023","journal-title":"Resuscitation"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1186\/s13054-019-2351-7","article-title":"Emergency department triage prediction of clinical outcomes using machine learning models","volume":"23","author":"Raita","year":"2019","journal-title":"Crit. Care"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mhd Yunin, N.S., and Tan, T.L. (2025). National Early Warning Score 2 (NEWS2) as a prognostic tool for adult patients in emergency department: A retrospective observational study. PLoS ONE, 20.","DOI":"10.1371\/journal.pone.0326058"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"m1501","DOI":"10.1136\/bmj.m1501","article-title":"Early warning scores for detecting deterioration in adult hospital patients: Systematic review and critical appraisal of methodology","volume":"369","author":"Gerry","year":"2020","journal-title":"BMJ"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"20963","DOI":"10.1038\/s41598-022-25284-1","article-title":"A fully-automated paper ECG digitisation algorithm using deep learning","volume":"12","author":"Wu","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1074","DOI":"10.1136\/heartjnl-2023-323822","article-title":"Automatic detection of cardiac conditions from photos of electrocardiogram captured by smartphones","volume":"110","author":"Wong","year":"2024","journal-title":"Heart"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1038\/s41746-025-02327-1","article-title":"Digitizing paper ECGs at scale: An open-source algorithm for clinical research","volume":"9","author":"Stenhede","year":"2026","journal-title":"npj Digit. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1093\/ehjdh\/ztae090","article-title":"A comparison of artificial intelligence-enhanced electrocardiography approaches for the prediction of time to mortality using electrocardiogram images","volume":"6","author":"Sau","year":"2025","journal-title":"Eur. Heart J.-Digit. Health"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Choi, H.Y., Kim, W., Kang, G.H., Jang, Y.S., Lee, Y., Kim, J.G., Lee, N., Shin, D.G., Bae, W., and Song, Y. (2022). Diagnostic Accuracy of the Deep Learning Model for the Detection of ST Elevation Myocardial Infarction on Electrocardiogram. J. Pers. Med., 12.","DOI":"10.3390\/jpm12030336"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"e004211","DOI":"10.1136\/openhrt-2026-004211","article-title":"AI-enhanced electrocardiography as a digital biomarker platform in cardiovascular medicine: Clinical applications, validation gaps and future implementation pathways","volume":"13","author":"Venturiello","year":"2026","journal-title":"Open Heart"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1038\/s41591-018-0240-2","article-title":"Screening for cardiac contractile dysfunction using an artificial intelligence-enabled electrocardiogram","volume":"25","author":"Attia","year":"2019","journal-title":"Nat. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"886","DOI":"10.1038\/s41591-020-0870-z","article-title":"Prediction of mortality from 12-lead electrocardiogram voltage data using a deep neural network","volume":"26","author":"Raghunath","year":"2020","journal-title":"Nat. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e791","DOI":"10.1016\/S2589-7500(24)00172-9","article-title":"Artificial intelligence-enabled electrocardiogram for mortality and cardiovascular risk estimation: A model development and validation study","volume":"6","author":"Sau","year":"2024","journal-title":"Lancet Digit. Health"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chang, P.C., Liu, Z.Y., Huang, Y.C., Hsu, Y.C., Chen, J.S., Lin, C.H., Tsai, R., Chou, C.C., Wen, M.S., and Wo, H.T. (2023). Machine learning-based prediction of acute mortality in emergency department patients using twelve-lead electrocardiogram. Front. Cardiovasc. Med., 10.","DOI":"10.3389\/fcvm.2023.1245614"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e003343","DOI":"10.1136\/openhrt-2025-003343","article-title":"Risk stratification of chest pain in the emergency department using artificial intelligence applied to electrocardiograms","volume":"12","author":"Haimovich","year":"2025","journal-title":"Open Heart"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"110196","DOI":"10.1016\/j.compbiomed.2025.110196","article-title":"Enhancing clinical decision support with physiological waveforms\u2014A multimodal benchmark in emergency care","volume":"192","author":"Alcaraz","year":"2025","journal-title":"Comput. Biol. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e80569","DOI":"10.2196\/80569","article-title":"Serial 12-Lead Electrocardiogram-Based Deep-Learning Model for Hospital Admission Prediction in Emergency Department Cardiac Presentations: Retrospective Cohort Study","volume":"9","author":"Altintepe","year":"2025","journal-title":"JMIR Cardio"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e67576","DOI":"10.2196\/67576","article-title":"Deep Learning-Based Electrocardiogram Model (EIANet) to Predict Emergency Department Cardiac Arrest: Development and External Validation Study","volume":"27","author":"Lu","year":"2025","journal-title":"J. Med. Internet Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1038\/s41746-026-02682-7","article-title":"External validation of ECG artificial intelligence for emergency and cardiac assessment across a large-scale U.S. healthcare system","volume":"9","author":"Lee","year":"2026","journal-title":"npj Digit. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e026471","DOI":"10.1136\/bmjopen-2018-026471","article-title":"Performance of triage systems in emergency care: A systematic review and meta-analysis","volume":"9","author":"Zachariasse","year":"2019","journal-title":"BMJ Open"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1136\/emermed-2024-214562","article-title":"Accuracy of the National Early Warning Score version 2 (NEWS2) in predicting need for time-critical treatment: Retrospective observational cohort study","volume":"42","author":"Goodacre","year":"2025","journal-title":"Emerg. Med. J."},{"key":"ref_23","unstructured":"Apple (2026, September 10). Core ML. Apple Developer Documentation. Available online: https:\/\/developer.apple.com\/documentation\/coreml."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wu, H., Sawada, T., Goto, T., Yoneyama, T., Sasano, T., and Asada, K. (2024). Edge AI Model Deployed for Real-Time Detection of Atrial Fibrillation Risk during Sinus Rhythm. J. Clin. Med., 13.","DOI":"10.20944\/preprints202403.1575.v1"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kartali, A., Koshtrevska, M., Joki\u0107, S., Gligori\u0107, N., and Machidon, O.M. (2026). Privacy-preserving adaptive arrhythmia monitoring with wearable ECG, smartphone PPG, and federated on-device learning. Front. Digit. Health, 8.","DOI":"10.3389\/fdgth.2026.1900115"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3283","DOI":"10.1038\/s41591-025-03953-8","article-title":"The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence","volume":"31","author":"Sounderajah","year":"2025","journal-title":"Nat. Med."},{"key":"ref_28","unstructured":"Gilboy, N., Tanabe, P., Travers, D., and Rosenau, A.M. (2011). Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4. Implementation Handbook 2012 Edition, Agency for Healthcare Research and Quality. AHRQ Publication No. 12-0014."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1093\/biomet\/26.4.404","article-title":"The use of confidence or fiducial limits illustrated in the case of the binomial","volume":"26","author":"Clopper","year":"1934","journal-title":"Biometrika"},{"key":"ref_30","unstructured":"National Institute of Standards and Technology (2026, September 10). Proportion Confidence Interval. Dataplot Reference Manual, Available online: https:\/\/www.itl.nist.gov\/div898\/software\/dataplot\/refman1\/auxillar\/propconf.htm."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1214\/aos\/1176344552","article-title":"Bootstrap Methods: Another Look at the Jackknife","volume":"7","author":"Efron","year":"1979","journal-title":"Ann. Stat."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Van Calster, B., McLernon, D.J., van Smeden, M., Wynants, L., Steyerberg, E.W., and Topic Group \u2018Evaluating Diagnostic Tests and Prediction Models\u2019 of the STRATOS Initiative (2019). Calibration: The Achilles heel of predictive analytics. BMC Med., 17.","DOI":"10.1186\/s12916-019-1466-7"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1016\/j.jclinepi.2015.12.005","article-title":"A calibration hierarchy for risk models was defined: From utopia to empirical data","volume":"74","author":"Nieboer","year":"2016","journal-title":"J. Clin. Epidemiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision curve analysis: A novel method for evaluating prediction models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med. Decis. Mak."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"e074820","DOI":"10.1136\/bmj-2023-074820","article-title":"Evaluation of clinical prediction models (part 2): How to undertake an external validation study","volume":"384","author":"Riley","year":"2024","journal-title":"BMJ"},{"key":"ref_36","first-page":"YE01","article-title":"Requirements for Minimum Sample Size for Sensitivity and Specificity Analysis","volume":"10","author":"Bujang","year":"2016","journal-title":"J. Clin. Diagn. Res."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"e074821","DOI":"10.1136\/bmj-2023-074821","article-title":"Evaluation of clinical prediction models (part 3): Calculating the sample size required for an external validation study","volume":"384","author":"Riley","year":"2024","journal-title":"BMJ"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1038\/s41591-022-01772-9","article-title":"Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI","volume":"28","author":"Vasey","year":"2022","journal-title":"Nat. Med."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1038\/s41591-020-1034-x","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: The CONSORT-AI extension","volume":"26","author":"Liu","year":"2020","journal-title":"Nat. Med."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"100804","DOI":"10.1016\/j.patter.2023.100804","article-title":"Leakage and the reproducibility crisis in machine-learning-based science","volume":"4","author":"Kapoor","year":"2023","journal-title":"Patterns"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"e082505","DOI":"10.1136\/bmj-2024-082505","article-title":"PROBAST+AI: An updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods","volume":"388","author":"Moons","year":"2025","journal-title":"BMJ"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3153\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T11:58:46Z","timestamp":1790596726000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,28]]},"references-count":41,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["diagnostics16193153"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16193153","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,28]]}}}