{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T12:06:29Z","timestamp":1790769989317,"version":"4.1.0"},"reference-count":38,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"am","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Electronic health record (EHR) time series are irregularly sampled and frequently incomplete, with missingness potentially carrying clinically informative signals. We developed Masked-Attention-based timestamp-wise Data Embedding (MADE), a missingness-aware model for continuous prediction of deterioration in general wards, and evaluated it across heterogeneous hospitals.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>Observed variables were tokenized into feature-specific embeddings and processed by a masked-attention Transformer, followed by bi-LSTM temporal aggregation for risk prediction. MADE was evaluated for major adverse event (MAE) and sepsis in 1 internal (SVH) and 2 external cohorts (AMC, HUMC) against conventional and deep learning baselines. Ablation studies compared no imputation with last observation carried forward (LOCF) and multiple imputations by chained equations (MICE). Discriminative performance was assessed using the area under the receiver operating characteristic curve (AUROC) and precision\u2013recall curve (AUPRC). Clinical utility and reliability were assessed using decision curve analysis, Brier score, and expected calibration error (ECE).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>MADE achieved strong discrimination for MAE (AUROC 0.966\/0.902\/0.825 for SVH\/AMC\/HUMC) and sepsis (AUROC 0.934\/0.874\/0.776). The no-imputation configuration generally outperformed LOCF and MICE, particularly for MAE and for sepsis AUPRC, supporting preservation of informative missingness. Decision curves showed favorable net benefit for MAE and competitive benefit for sepsis. Calibration indicated modest underestimation at higher risks, suggesting site-specific recalibration.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>MADE mitigates limitations of fixed grid and imputation-dependent EHR modeling by jointly leveraging observed values and missingness patterns under heterogeneous sampling.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>MADE supports scalable observation-aligned risk monitoring, particularly for MAE, while sepsis prediction may require site-specific recalibration or feature-set adaptation under cross-institution domain shifts.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocag124","type":"journal-article","created":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T11:30:38Z","timestamp":1783337438000},"source":"Crossref","is-referenced-by-count":0,"title":["MADE: leveraging informative missingness and temporal evaluation for electronic health record-based deterioration prediction in general wards"],"prefix":"10.1093","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-5402-6847","authenticated-orcid":false,"given":"Yechan","family":"Mun","sequence":"first","affiliation":[{"name":"AITRICS. Inc Research Department, , Seoul 06627,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jee Hwan","family":"Ahn","sequence":"additional","affiliation":[{"name":"Department of Pulmonary and Critical Care Medicine, Asan Medical Center, University of Ulsan College of Medicine , Seoul 05505,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chang Youl","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Chuncheon Sacred Heart Hospital, Hallym University Medical Center , Seoul 24253,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Byungeun","family":"Ahn","sequence":"additional","affiliation":[{"name":"AITRICS. Inc Research Department, , Seoul 06627,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jin Won","family":"Huh","sequence":"additional","affiliation":[{"name":"Department of Pulmonary and Critical Care Medicine, Asan Medical Center, University of Ulsan College of Medicine , Seoul 05505,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1604-8730","authenticated-orcid":false,"given":"Kyung Soo","family":"Chung","sequence":"additional","affiliation":[{"name":"Division of Pulmonary and Critical Care Medicine, Department of Internal Medicine, Severance Hospital, Yonsei University College of Medicine , Seoul 03722,","place":["Republic of Korea"]},{"name":"Institute for Innovation in Digital Healthcare, Yonsei University , Seoul 03722,","place":["Republic of Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"2026093007504989800_ocag124-B1","doi-asserted-by":"publisher","first-page":"30116","DOI":"10.1038\/s41598-024-80268-7","article-title":"A novel deep learning algorithm for real-time prediction of clinical deterioration in the emergency department for a multimodal clinical decision support system","volume":"14","author":"Choi","year":"2024","journal-title":"Sci Rep."},{"key":"2026093007504989800_ocag124-B2","doi-asserted-by":"publisher","first-page":"e2438986","DOI":"10.1001\/jamanetworkopen.2024.38986","article-title":"Early warning scores with and without artificial intelligence","volume":"7","author":"Edelson","year":"2024","journal-title":"JAMA Netw Open"},{"key":"2026093007504989800_ocag124-B3","doi-asserted-by":"publisher","first-page":"13563","DOI":"10.1038\/s41598-023-40661-0","article-title":"Predicting acute clinical deterioration with interpretable machine learning to support emergency care decision making","volume":"13","author":"Boulitsakis Logothetis","year":"2023","journal-title":"Sci Rep."},{"key":"2026093007504989800_ocag124-B4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s00392-016-1025-6","article-title":"Electronic health records to facilitate clinical research","volume":"106","author":"Cowie","year":"2017","journal-title":"Clin Res Cardiol."},{"key":"2026093007504989800_ocag124-B5","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1007\/s41666-022-00123-0","article-title":"Electronic health records that support health professional reflective practice: A missed opportunity in digital health","volume":"6","author":"Janssen","year":"2022","journal-title":"J Healthc Inform Res."},{"key":"2026093007504989800_ocag124-B6","doi-asserted-by":"publisher","first-page":"197","DOI":"10.4266\/acc.000525","article-title":"Prospective external validation of a deep-learning-based early-warning system for major adverse events in general wards in South Korea","volume":"40","author":"Sim","year":"2025","journal-title":"Acute Crit Care."},{"key":"2026093007504989800_ocag124-B7","doi-asserted-by":"publisher","first-page":"e101353","DOI":"10.1136\/bmjhci-2024-101353","article-title":"Validation of an artificial intelligence-based algorithm for predictive performance and risk stratification of sepsis using real-world data from hospitalised patients: A prospective observational study","volume":"32","author":"Kim","year":"2025","journal-title":"BMJ Health Care Inform"},{"key":"2026093007504989800_ocag124-B8","doi-asserted-by":"publisher","first-page":"17903","DOI":"10.1007\/s10489-022-04425-z","article-title":"Early prediction of sepsis using double fusion of deep features and handcrafted features","volume":"53","author":"Duan","year":"2023","journal-title":"Appl Intell."},{"key":"2026093007504989800_ocag124-B9","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1186\/s13054-023-04609-0","article-title":"Prospective, multicenter validation of the deep learning-based cardiac arrest risk management system for predicting in-hospital cardiac arrest or unplanned intensive care unit transfer in patients admitted to general wards","volume":"27","author":"Cho","year":"2023","journal-title":"Crit Care."},{"key":"2026093007504989800_ocag124-B10","doi-asserted-by":"publisher","first-page":"e72232","DOI":"10.2196\/72232","article-title":"Deep learning\u2013based early warning systems in hospitalized patients at risk of code blue events and length of stay: Retrospective real-world implementation study","volume":"13","author":"Kim","year":"2025","journal-title":"JMIR Med Inform"},{"key":"2026093007504989800_ocag124-B11","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1186\/s12911-022-01787-9","article-title":"Machine learning based forecast for the prediction of inpatient bed demand","volume":"22","author":"Tello","year":"2022","journal-title":"BMC Med Inform Decis Mak."},{"key":"2026093007504989800_ocag124-B12","first-page":"41300","author":"Zhang"},{"key":"2026093007504989800_ocag124-B13","doi-asserted-by":"publisher","first-page":"1246","DOI":"10.1093\/jamia\/ocad066","article-title":"Missing data matter: An empirical evaluation of the impacts of missing EHR data in comparative effectiveness research","volume":"30","author":"Zhou","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2026093007504989800_ocag124-B14","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1186\/s12911-025-03058-9","article-title":"Implicit bias in ICU electronic health record data: measurement frequencies and missing data rates of clinical variables","volume":"25","author":"Shi","year":"2025","journal-title":"BMC Med Inform Decis Mak."},{"key":"2026093007504989800_ocag124-B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3516367","article-title":"Self-supervised transformer for sparse and irregularly sampled multivariate clinical time-series","volume":"16","author":"Tipirneni","year":"2022","journal-title":"ACM Trans Knowl Discov Data."},{"key":"2026093007504989800_ocag124-B16","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1186\/s12911-024-02514-2","article-title":"Continuous patient state attention model for addressing irregularity in electronic health records","volume":"24","author":"Chauhan","year":"2024","journal-title":"BMC Med Inform Decis Mak."},{"key":"2026093007504989800_ocag124-B17","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1038\/s41746-021-00518-0","article-title":"Imputation of missing values for electronic health record laboratory data","volume":"4","author":"Li","year":"2021","journal-title":"NPJ Digit Med."},{"key":"2026093007504989800_ocag124-B18","doi-asserted-by":"publisher","first-page":"102587","DOI":"10.1016\/j.artmed.2023.102587","article-title":"Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques","volume":"142","author":"Liu","year":"2023","journal-title":"Artif Intell Med."},{"key":"2026093007504989800_ocag124-B19","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1186\/s12874-024-02310-6","article-title":"Identify the most appropriate imputation method for handling missing values in clinical structured datasets: A systematic review","volume":"24","author":"Afkanpour","year":"2024","journal-title":"BMC Med Res Methodol."},{"key":"2026093007504989800_ocag124-B20","doi-asserted-by":"publisher","first-page":"2213","DOI":"10.3390\/jcm14072213","article-title":"Preserving informative presence: How missing data and imputation strategies affect the performance of an AI-based early warning score","volume":"14","author":"Sim","year":"2025","journal-title":"J Clin Med."},{"key":"2026093007504989800_ocag124-B21","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s13721-024-00490-1","article-title":"Motif discovery in hospital ward vital signs observation networks","volume":"13","author":"Ironside-Smith","year":"2024","journal-title":"Netw Model Anal Health Inform Bioinform."},{"key":"2026093007504989800_ocag124-B22","doi-asserted-by":"publisher","first-page":"1526","DOI":"10.3390\/s24051526","article-title":"Missing data statistics provide causal insights into data loss in diabetes health monitoring by wearable sensors","volume":"24","author":"Braem","year":"2024","journal-title":"Sensors (Basel)."},{"key":"2026093007504989800_ocag124-B23","doi-asserted-by":"publisher","first-page":"194","DOI":"10.4103\/mamcjms.mamcjms_54_20","article-title":"Methods to handle incomplete data","volume":"6","author":"Johny","year":"2020","journal-title":"MAMC J Med Sci."},{"key":"2026093007504989800_ocag124-B24","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/s13040-024-00372-2","article-title":"Electronic medical records imputation by temporal generative adversarial network","volume":"17","author":"Yin","year":"2024","journal-title":"BioData Min."},{"key":"2026093007504989800_ocag124-B25","doi-asserted-by":"publisher","first-page":"6085","DOI":"10.1038\/s41598-018-24271-9","article-title":"Recurrent neural networks for multivariate time series with missing values","volume":"8","author":"Che","year":"2018","journal-title":"Sci Rep."},{"key":"2026093007504989800_ocag124-B26","first-page":"6775\u2011","article-title":"Brits: Bidirectional recurrent imputation for time series","volume":"31","author":"Cao","year":"2018","journal-title":"Adv Neural Inf Process Syst"},{"key":"2026093007504989800_ocag124-B27","doi-asserted-by":"publisher","first-page":"39119","DOI":"10.1038\/s41598-025-26611-y","article-title":"The impact of feature combinations on machine learning models for in-hospital mortality prediction","volume":"15","author":"Stenwig","year":"2025","journal-title":"Sci Rep."},{"key":"2026093007504989800_ocag124-B28","doi-asserted-by":"publisher","first-page":"e074819","DOI":"10.1136\/bmj-2023-074819","article-title":"Evaluation of clinical prediction models (part 1): From development to external validation","volume":"384","author":"Collins","year":"2024","journal-title":"Bmj"},{"key":"2026093007504989800_ocag124-B29","doi-asserted-by":"publisher","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"Bmj."},{"key":"2026093007504989800_ocag124-B30","doi-asserted-by":"publisher","first-page":"e26426","DOI":"10.2196\/26426","article-title":"Event prediction model considering time and input error using electronic medical records in the intensive care unit: Retrospective study","volume":"9","author":"Sung","year":"2021","journal-title":"JMIR Med Inform"},{"key":"2026093007504989800_ocag124-B31","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","article-title":"The third international consensus definitions for sepsis and septic shock (Sepsis-3)","volume":"315","author":"Singer","year":"2016","journal-title":"Jama."},{"key":"2026093007504989800_ocag124-B32","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1308\/003588406X130615","article-title":"The value of modified early warning score (MEWS) in surgical in-patients: A prospective observational study","volume":"88","author":"Gardner-Thorpe","year":"2006","journal-title":"Ann R Coll Surg Engl."},{"key":"2026093007504989800_ocag124-B33","doi-asserted-by":"publisher","first-page":"499","DOI":"10.7861\/clinmed.2022-news-concept","article-title":"The national early warning score: From concept to NHS implementation","volume":"22","author":"Williams","year":"2022","journal-title":"Clin Med (Lond)"},{"key":"2026093007504989800_ocag124-B34","doi-asserted-by":"publisher","first-page":"357","DOI":"10.2307\/2280041","article-title":"Application of the logistic function to bio-assay","volume":"39","author":"Berkson","year":"1944","journal-title":"J Am Stat Assoc"},{"key":"2026093007504989800_ocag124-B35","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"2026093007504989800_ocag124-B36","first-page":"785","author":"Chen","year":"2016"},{"key":"2026093007504989800_ocag124-B37","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"2026093007504989800_ocag124-B38","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocag124\/71518566\/ocag124.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocag124\/71518566\/ocag124.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T11:50:59Z","timestamp":1790769059000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article\/doi\/10.1093\/jamia\/ocag124\/8853718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":38,"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocag124","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]},"article-number":"ocag124"}}