{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,30]],"date-time":"2025-12-30T08:45:56Z","timestamp":1767084356481,"version":"3.37.3"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2021,11,19]],"date-time":"2021-11-19T00:00:00Z","timestamp":1637280000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000057","name":"NIGMS","doi-asserted-by":"publisher","award":["R01 GM120079"],"award-info":[{"award-number":["R01 GM120079"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000133","name":"AHRQ","doi-asserted-by":"publisher","award":["R01 HS024532"],"award-info":[{"award-number":["R01 HS024532"]}],"id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"publisher"}]},{"name":"NCATS University of Minnesota","award":["CTSA UL1 TR002494"],"award-info":[{"award-number":["CTSA UL1 TR002494"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,12,28]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Hospital-acquired infections (HAIs) are associated with significant morbidity, mortality, and prolonged hospital length of stay. Risk prediction models based on pre- and intraoperative data have been proposed to assess the risk of HAIs at the end of the surgery, but the performance of these models lag behind HAI detection models based on postoperative data. Postoperative data are more predictive than pre- or interoperative data since it is closer to the outcomes in time, but it is unavailable when the risk models are applied (end of surgery). The objective is to study whether such data, which is temporally unavailable at prediction time (TUP) (and thus cannot directly enter the model), can be used to improve the performance of the risk model.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>An extensive array of 12 methods based on logistic\/linear regression and deep learning were used to incorporate the TUP data using a variety of intermediate representations of the data. Due to the hierarchical structure of different HAI outcomes, a comparison of single and multi-task learning frameworks is also presented.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results and Discussion<\/jats:title>\n                  <jats:p>The use of TUP data was always advantageous as baseline methods, which cannot utilize TUP data, never achieved the top performance. The relative performances of the different models vary across the different outcomes. Regarding the intermediate representation, we found that its complexity was key and that incorporating label information was helpful.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Using TUP data significantly helped predictive performance irrespective of the model complexity.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocab229","type":"journal-article","created":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T20:33:14Z","timestamp":1634157194000},"page":"72-79","source":"Crossref","is-referenced-by-count":13,"title":["Strategies for building robust prediction models using data unavailable at prediction time"],"prefix":"10.1093","volume":"29","author":[{"given":"Haoyu","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roshan","family":"Tourani","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Zhu","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vipin","family":"Kumar","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5193-1663","authenticated-orcid":false,"given":"Genevieve B","family":"Melton","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"},{"name":"Department of Surgery, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Steinbach","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gyorgy","family":"Simon","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"},{"name":"Department of Internal Medicine, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2021,11,19]]},"reference":[{"key":"2021122823071363600_ocab229-B1","doi-asserted-by":"crossref","first-page":"105581","DOI":"10.1016\/j.cmpb.2020.105581","article-title":"CoroNet: a deep neural network for detection and diagnosis of COVID-19 from chest x-ray images","volume":"196","author":"Khan","year":"2020","journal-title":"Comput Methods Programs Biomed"},{"issue":"2","key":"2021122823071363600_ocab229-B2","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1093\/jamiaopen\/ooaa006","article-title":"Machine learning for early detection of sepsis: an internal and temporal validation study","volume":"3","author":"Bedoya","year":"2020","journal-title":"JAMIA Open"},{"issue":"1","key":"2021122823071363600_ocab229-B3","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1093\/jamiaopen\/ooy011","article-title":"Predictive modeling in urgent care: a comparative study of machine learning approaches","volume":"1","author":"Tang","year":"2018","journal-title":"JAMIA Open"},{"key":"2021122823071363600_ocab229-B4","doi-asserted-by":"crossref","DOI":"10.1109\/TKDE.2021.3052992","article-title":"Prediction of treatment medicines with dual adaptive sequential networks","author":"An","year":"2021","journal-title":"IEEE Trans Knowl Data Eng"},{"volume-title":"Risk Modeling, Assessment, and Management","year":"2005","author":"Haimes","key":"2021122823071363600_ocab229-B5"},{"issue":"1","key":"2021122823071363600_ocab229-B6","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/S2213-2600(20)30520-8","article-title":"Sepsis: the importance of an accurate final diagnosis","volume":"9","author":"Tidswell","year":"2021","journal-title":"Lancet Respir Med"},{"issue":"7","key":"2021122823071363600_ocab229-B7","doi-asserted-by":"crossref","first-page":"e207750","DOI":"10.1001\/jamanetworkopen.2020.7750","article-title":"Assessment of the accuracy of using ICD-9 diagnosis codes to identify pneumonia etiology in patients hospitalized with pneumonia","volume":"3","author":"Higgins","year":"2020","journal-title":"JAMA Netw Open"},{"key":"2021122823071363600_ocab229-B8"},{"issue":"2","key":"2021122823071363600_ocab229-B9","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1007\/s10994-019-05855-6","article-title":"A survey on semi-supervised learning","volume":"109","author":"Van Engelen","year":"2020","journal-title":"Mach Learn"},{"issue":"1","key":"2021122823071363600_ocab229-B10","first-page":"1","article-title":"Introduction to semi-supervised learning","volume":"3","author":"Zhu","year":"2009","journal-title":"Synth Lect Artif Intell Mach Learn"},{"issue":"10","key":"2021122823071363600_ocab229-B11","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans Knowl Data Eng"},{"issue":"2","key":"2021122823071363600_ocab229-B12","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1007\/s41666-019-00062-3","article-title":"Transfer learning for clinical time series analysis using deep neural networks","volume":"4","author":"Gupta","year":"2020","journal-title":"J Healthc Inform Res"},{"key":"2021122823071363600_ocab229-B13","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.neunet.2019.08.032","article-title":"Named entity recognition in electronic health records using transfer learning bootstrapped neural networks","volume":"121","author":"Gligic","year":"2020","journal-title":"Neural Netw"},{"article-title":"Distilling the knowledge in a neural network","year":"2015","author":"Hinton","key":"2021122823071363600_ocab229-B14"},{"article-title":"Fitnets: Hints for thin deep nets","year":"2014","author":"Romero","key":"2021122823071363600_ocab229-B15"},{"key":"2021122823071363600_ocab229-B16","first-page":"3967","article-title":"Relational knowledge distillation","author":"Park","year":"2019","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"2021122823071363600_ocab229-B17","first-page":"1365","article-title":"Similarity-preserving knowledge distillation","author":"Tung","year":"2019","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"2021122823071363600_ocab229-B18","article-title":"Unifying distillation and privileged information","author":"Lopez-Paz","year":"2016","journal-title":"International Conference on Learning Representations"},{"key":"2021122823071363600_ocab229-B19","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1007\/978-3-030-59137-3_16","volume-title":"International Conference on Artificial Intelligence in Medicine","author":"Zhu","year":"2020"},{"issue":"12","key":"2021122823071363600_ocab229-B20","doi-asserted-by":"crossref","first-page":"2639","DOI":"10.1162\/0899766042321814","article-title":"Canonical correlation analysis: an overview with application to learning methods","volume":"16","author":"Hardoon","year":"2004","journal-title":"Neural Comput"},{"key":"2021122823071363600_ocab229-B21","first-page":"107","article-title":"Supervised autoencoders: improving generalization performance with unsupervised regularizers","volume":"31","author":"Le","year":"2018","journal-title":"Adv Neural Inf Process Syst"},{"year":"2000","author":"Tishby","key":"2021122823071363600_ocab229-B22"},{"key":"2021122823071363600_ocab229-B23","first-page":"165","article-title":"Information bottleneck for gaussian variables","volume":"6","author":"Chechik","year":"2005","journal-title":"J Mach Learn Res"},{"key":"2021122823071363600_ocab229-B24","first-page":"1","volume-title":"2015 IEEE Information Theory Workshop (ITW)","author":"Tishby","year":"2015"},{"volume-title":"Causation, Prediction, and Search","year":"2000","author":"Spirtes","key":"2021122823071363600_ocab229-B25"},{"key":"2021122823071363600_ocab229-B26","first-page":"3741","article-title":"Order-independent constraint-based causal structure learning","volume":"15","author":"Colombo","year":"2014","journal-title":"J Mach Learn Res"},{"key":"2021122823071363600_ocab229-B27","doi-asserted-by":"crossref","DOI":"10.1201\/9780429246593","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1994"},{"key":"2021122823071363600_ocab229-B28","first-page":"1321","volume-title":"International Conference on Machine Learning","author":"Guo","year":"2017"},{"key":"2021122823071363600_ocab229-B29","first-page":"5142","volume-title":"International Conference on Machine Learning","author":"Phuong","year":"2019"},{"issue":"6","key":"2021122823071363600_ocab229-B30","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1007\/s11263-021-01453-z","article-title":"Knowledge distillation: a survey","volume":"129","author":"Gou","year":"2021","journal-title":"Int J Comput Vis"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/1\/72\/41955533\/ocab229.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/1\/72\/41955533\/ocab229.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,28]],"date-time":"2021-12-28T23:10:52Z","timestamp":1640733052000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/1\/72\/6432059"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,19]]},"references-count":30,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,11,19]]},"published-print":{"date-parts":[[2021,12,28]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab229","relation":{},"ISSN":["1527-974X"],"issn-type":[{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2022,1,1]]},"published":{"date-parts":[[2021,11,19]]}}}