{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T03:15:27Z","timestamp":1774062927869,"version":"3.50.1"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,11,9]],"date-time":"2022-11-09T00:00:00Z","timestamp":1667952000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["R01GM110240"],"award-info":[{"award-number":["R01GM110240"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["OT2OD032701"],"award-info":[{"award-number":["OT2OD032701"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["K01DK120784, R01DK123078, and R01DK1217300"],"award-info":[{"award-number":["K01DK120784, R01DK123078, and R01DK1217300"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01NS120924"],"award-info":[{"award-number":["R01NS120924"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["OT2OD032701"],"award-info":[{"award-number":["OT2OD032701"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["R01GM110240"],"award-info":[{"award-number":["R01GM110240"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000070","name":"National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","award":["R01EB029699 and R21EB027344"],"award-info":[{"award-number":["R01EB029699 and R21EB027344"]}],"id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01NS120924"],"award-info":[{"award-number":["R01NS120924"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["R01DK121730"],"award-info":[{"award-number":["R01DK121730"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","award":["OT2OD032701"],"award-info":[{"award-number":["OT2OD032701"]}],"id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1750192, R01EB029699 and R21EB027344"],"award-info":[{"award-number":["1750192, R01EB029699 and R21EB027344"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000070","name":"National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Science","doi-asserted-by":"publisher","award":["R01GM110240"],"award-info":[{"award-number":["R01GM110240"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01NS120924"],"award-info":[{"award-number":["R01NS120924"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000065","name":"National Institute of Neurological Disorders and Stroke","doi-asserted-by":"publisher","award":["R01DK121730"],"award-info":[{"award-number":["R01DK121730"]}],"id":[{"id":"10.13039\/100000065","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["OT2OD032701"],"award-info":[{"award-number":["OT2OD032701"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000052","name":"NIH Office of the Director","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:p>Transformer model architectures have revolutionized the natural language processing (NLP) domain and continue to produce state-of-the-art results in text-based applications. Prior to the emergence of transformers, traditional NLP models such as recurrent and convolutional neural networks demonstrated promising utility for patient-level predictions and health forecasting from longitudinal datasets. However, to our knowledge only few studies have explored transformers for predicting clinical outcomes from electronic health record (EHR) data, and in our estimation, none have adequately derived a health-specific tokenization scheme to fully capture the heterogeneity of EHR systems. In this study, we propose a dynamic method for tokenizing both discrete and continuous patient data, and present a transformer-based classifier utilizing a joint embedding space for integrating disparate temporal patient measurements. We demonstrate the feasibility of our clinical AI framework through multi-task ICU patient acuity estimation, where we simultaneously predict six mortality and readmission outcomes. Our longitudinal EHR tokenization and transformer modeling approaches resulted in more accurate predictions compared with baseline machine learning models, which suggest opportunities for future multimodal data integrations and algorithmic support tools using clinical transformer networks.<\/jats:p>","DOI":"10.3389\/fdgth.2022.1029191","type":"journal-article","created":{"date-parts":[[2022,11,9]],"date-time":"2022-11-09T08:21:01Z","timestamp":1667982061000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Multi-dimensional patient acuity estimation with longitudinal EHR tokenization and flexible transformer networks"],"prefix":"10.3389","volume":"4","author":[{"given":"Benjamin","family":"Shickel","sequence":"first","affiliation":[]},{"given":"Brandon","family":"Silva","sequence":"additional","affiliation":[]},{"given":"Tezcan","family":"Ozrazgat-Baslanti","sequence":"additional","affiliation":[]},{"given":"Yuanfang","family":"Ren","sequence":"additional","affiliation":[]},{"given":"Kia","family":"Khezeli","sequence":"additional","affiliation":[]},{"given":"Ziyuan","family":"Guan","sequence":"additional","affiliation":[]},{"given":"Patrick J.","family":"Tighe","sequence":"additional","affiliation":[]},{"given":"Azra","family":"Bihorac","sequence":"additional","affiliation":[]},{"given":"Parisa","family":"Rashidi","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2022,11,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"707","DOI":"10.1007\/BF01709751","article-title":"The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction\/failure. On behalf of the working group on sepsis-related problems of the european society of intensive care medicine","volume":"22","author":"Vincent","year":"1996","journal-title":"Intensive Care Med"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1793","DOI":"10.1097\/00003246-199811000-00016","article-title":"Use of the SOFA score to assess the incidence of organ dysfunction\/failure in intensive care units","volume":"26","author":"Vincent","year":"1998","journal-title":"Crit Care Med"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1879","DOI":"10.1038\/s41598-019-38491-0","article-title":"DeepSOFA: a continuous acuity score for critically ill patients using clinically interpretable deep learning","volume":"9","author":"Shickel","year":"2019","journal-title":"Sci Rep"},{"key":"B4","author":"Choi","year":""},{"key":"B5","author":"Choi","year":""},{"key":"B6","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1093\/jamia\/ocw112","article-title":"Using recurrent neural network models for early detection of heart failure onset","volume":"292","author":"Choi","year":"2016","journal-title":"J Am Med Inform Assoc"},{"key":"B7","author":"Sha","year":""},{"key":"B8","author":"Lipton","year":""},{"key":"B9","author":"Lin","year":""},{"key":"B10","author":"Nguyen","year":""},{"key":"B11","doi-asserted-by":"publisher","first-page":"5998","DOI":"10.1017\/S0952523813000308","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"B12","author":"Devlin","year":""},{"key":"B13","author":"Zhou","year":""},{"key":"B14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-020-62922-y","article-title":"BEHRT: transformer for electronic health records","volume":"10","author":"Li","year":"2020","journal-title":"Sci Rep"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-021-00455-y","article-title":"Med-BERT: pretrained contextualized embeddings on large- scale structured electronic health records for disease prediction","volume":"4","author":"Rasmy","year":"2021","journal-title":"NPJ Digit Med"},{"key":"B16","author":"Li","year":""},{"key":"B17","author":"Song","year":""},{"key":"B18","author":"Beltagy","year":""},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-022-11012-2","article-title":"Interpretability, fairness evaluation of deep learning models on MIMIC-IV dataset","volume":"12","author":"Meng","year":"2022","journal-title":"Sci Rep"},{"key":"B20","author":"Dorogush","year":""},{"key":"B21","author":"Chen","year":""},{"key":"B22","author":"Qin","year":""},{"key":"B23","author":"Sukhbaatar","year":""},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TPAMI.2022.3152247","article-title":"A survey on vision transformer","author":"Han","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"B25","author":"Patrick","year":""},{"key":"B26","author":"Huang","year":""},{"key":"B27","author":"Zhang","year":""},{"key":"B28","author":"Ren","year":""},{"key":"B29","author":"Peng","year":""},{"key":"B30","author":"Prakash","year":""},{"key":"B31","doi-asserted-by":"publisher","first-page":"3121","DOI":"10.1109\/JBHI.2021.3063721","article-title":"Transformers using multimodal electronic health record data to predict depression","volume":"25","author":"Meng","year":"2021","journal-title":"IEEE J Biomed Health Inform"},{"key":"B32","author":"Choi","year":""},{"key":"B33","author":"Shang","year":""},{"key":"B34","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1093\/jamia\/ocx084","article-title":"The mimic code repository: enabling reproducibility in critical care research","volume":"25","author":"Johnson","year":"2018","journal-title":"J Am Med Inform Assoc"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2022.1029191\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,9]],"date-time":"2022-11-09T08:21:06Z","timestamp":1667982066000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2022.1029191\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,9]]},"references-count":34,"alternative-id":["10.3389\/fdgth.2022.1029191"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2022.1029191","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,9]]},"article-number":"1029191"}}