{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T02:04:53Z","timestamp":1774922693464,"version":"3.50.1"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T00:00:00Z","timestamp":1658534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"National Institutes of Health (NIH) National Heart, Lung, and Blood Institute","award":["R01HL157262"],"award-info":[{"award-number":["R01HL157262"]}]},{"name":"National Institutes of Health (NIH) National Heart, Lung, and Blood Institute","award":["K01HL148390"],"award-info":[{"award-number":["K01HL148390"]}]},{"DOI":"10.13039\/100000057","name":"NIH National Institute of General Medical Sciences","doi-asserted-by":"crossref","award":["R01GM123193"],"award-info":[{"award-number":["R01GM123193"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000057","name":"NIH National Institute of General Medical Sciences","doi-asserted-by":"crossref","award":["K23GM144867"],"award-info":[{"award-number":["K23GM144867"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000027","name":"NIH National Institute on Alcohol Abuse and Alcoholism","doi-asserted-by":"crossref","award":["K23AA024503"],"award-info":[{"award-number":["K23AA024503"]}],"id":[{"id":"10.13039\/100000027","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000026","name":"NIH National Institute on Drug Abuse","doi-asserted-by":"crossref","award":["R01DA051464"],"award-info":[{"award-number":["R01DA051464"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,9,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objectives<\/jats:title><jats:p>Early identification of infection improves outcomes, but developing models for early identification requires determining infection status with manual chart review, limiting sample size. Therefore, we aimed to compare semi-supervised and transfer learning algorithms with algorithms based solely on manual chart review for identifying infection in hospitalized patients.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>This multicenter retrospective study of admissions to 6 hospitals included \u201cgold-standard\u201d labels of infection from manual chart review and \u201csilver-standard\u201d labels from nonchart-reviewed patients using the Sepsis-3 infection criteria based on antibiotic and culture orders. \u201cGold-standard\u201d labeled admissions were randomly allocated to training (70%) and testing (30%) datasets. Using patient characteristics, vital signs, and laboratory data from the first 24 hours of admission, we derived deep learning and non-deep learning models using transfer learning and semi-supervised methods. Performance was compared in the gold-standard test set using discrimination and calibration metrics.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The study comprised 432\u200a965 admissions, of which 2724 underwent chart review. In the test set, deep learning and non-deep learning approaches had similar discrimination (area under the receiver operating characteristic curve of 0.82). Semi-supervised and transfer learning approaches did not improve discrimination over models fit using only silver- or gold-standard data. Transfer learning had the best calibration (unreliability index P value: .997, Brier score: 0.173), followed by self-learning gradient boosted machine (P value: .67, Brier score: 0.170).<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Deep learning and non-deep learning models performed similarly for identifying infection, as did models developed using Sepsis-3 and manual chart review labels.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>In a multicenter study of almost 3000 chart-reviewed patients, semi-supervised and transfer learning models showed similar performance for model discrimination as baseline XGBoost, while transfer learning improved calibration.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocac109","type":"journal-article","created":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T14:20:28Z","timestamp":1658586028000},"page":"1696-1704","source":"Crossref","is-referenced-by-count":7,"title":["Identifying infected patients using semi-supervised and transfer learning"],"prefix":"10.1093","volume":"29","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0153-4958","authenticated-orcid":false,"given":"Fereshteh S","family":"Bashiri","sequence":"first","affiliation":[{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, Wisconsin, USA"}]},{"given":"John R","family":"Caskey","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, Wisconsin, USA"}]},{"given":"Anoop","family":"Mayampurath","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison , Madison, Wisconsin, USA"}]},{"given":"Nicole","family":"Dussault","sequence":"additional","affiliation":[{"name":"Pritzker School of Medicine, University of Chicago , Chicago, Illinois, USA"}]},{"given":"Jay","family":"Dumanian","sequence":"additional","affiliation":[{"name":"Pritzker School of Medicine, University of Chicago , Chicago, Illinois, USA"}]},{"given":"Sivasubramanium V","family":"Bhavani","sequence":"additional","affiliation":[{"name":"Department of Medicine, Emory University , Atlanta, Georgia, USA"}]},{"given":"Kyle A","family":"Carey","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago , Chicago, Illinois, USA"}]},{"given":"Emily R","family":"Gilbert","sequence":"additional","affiliation":[{"name":"Department of Medicine, Loyola University , Chicago, Illinois, USA"}]},{"given":"Christopher J","family":"Winslow","sequence":"additional","affiliation":[{"name":"Department of Medicine, NorthShore University HealthSystem , Evanston, Illinois, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9107-7788","authenticated-orcid":false,"given":"Nirav S","family":"Shah","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago , Chicago, Illinois, USA"},{"name":"Department of Medicine, NorthShore University HealthSystem , Evanston, Illinois, USA"}]},{"given":"Dana P","family":"Edelson","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago , Chicago, Illinois, USA"}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, Wisconsin, USA"},{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison , Madison, Wisconsin, USA"}]},{"given":"Matthew M","family":"Churpek","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, Wisconsin, USA"},{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison , Madison, Wisconsin, USA"}]}],"member":"286","published-online":{"date-parts":[[2022,7,23]]},"reference":[{"issue":"13","key":"2022091407545455600_ocac109-B1","doi-asserted-by":"crossref","first-page":"1241","DOI":"10.1001\/jama.2017.13836","article-title":"Incidence and trends of sepsis in US Hospitals using clinical vs claims data, 2009-2014","volume":"318","author":"Rhee","year":"2017","journal-title":"JAMA"},{"issue":"5","key":"2022091407545455600_ocac109-B2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/s11908-010-0122-3","article-title":"Early identification of sepsis","volume":"12","author":"Gonsalves","year":"2010","journal-title":"Curr Infect Dis Rep"},{"issue":"1","key":"2022091407545455600_ocac109-B3","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1186\/s13054-016-1423-1","article-title":"Early identification of sepsis in hospital inpatients by ward nurses increases 30-day survival","volume":"20","author":"Torsvik","year":"2016","journal-title":"Crit Care"},{"issue":"7","key":"2022091407545455600_ocac109-B4","doi-asserted-by":"crossref","first-page":"856","DOI":"10.1164\/rccm.201609-1848OC","article-title":"The timing of early antibiotics and hospital mortality in sepsis","volume":"196","author":"Liu","year":"2017","journal-title":"Am J Respir Crit Care Med"},{"issue":"3","key":"2022091407545455600_ocac109-B5","doi-asserted-by":"crossref","first-page":"1007","DOI":"10.21037\/jtd.2020.01.47","article-title":"An approach to antibiotic treatment in patients with sepsis","volume":"12","author":"Mart\u00ednez","year":"2020","journal-title":"J Thorac Dis"},{"issue":"13","key":"2022091407545455600_ocac109-B6","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"issue":"5","key":"2022091407545455600_ocac109-B7","doi-asserted-by":"crossref","first-page":"e205191","DOI":"10.1001\/jamanetworkopen.2020.5191","article-title":"Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit","volume":"3","author":"Liu","year":"2020","journal-title":"JAMA Netw Open"},{"issue":"1","key":"2022091407545455600_ocac109-B8","doi-asserted-by":"crossref","first-page":"e0245157","DOI":"10.1371\/journal.pone.0245157","article-title":"A comparison of machine learning models versus clinical evaluation for mortality prediction in patients with sepsis","volume":"16","author":"van Doorn","year":"2021","journal-title":"PLoS One"},{"issue":"3","key":"2022091407545455600_ocac109-B9","first-page":"21","article-title":"Risk stratification: a two-step process for identifying your sickest patients","volume":"26","author":"Dom Dera","year":"2019","journal-title":"Fam Pract Manag"},{"issue":"6","key":"2022091407545455600_ocac109-B10","doi-asserted-by":"crossref","first-page":"e39","DOI":"10.1097\/MLR.0b013e318268ac86","article-title":"Identifying patients with severe sepsis using administrative claims: patient-level validation of the angus implementation of the international consensus conference definition of severe sepsis","volume":"52","author":"Iwashyna","year":"2014","journal-title":"Med Care"},{"issue":"7","key":"2022091407545455600_ocac109-B11","doi-asserted-by":"crossref","first-page":"1303","DOI":"10.1097\/00003246-200107000-00002","article-title":"Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care","volume":"29","author":"Angus","year":"2001","journal-title":"Crit Care Med"},{"issue":"7","key":"2022091407545455600_ocac109-B12","doi-asserted-by":"crossref","first-page":"e673","DOI":"10.1097\/CCM.0000000000004968","article-title":"Determining the electronic signature of infection in electronic health record data","volume":"49","author":"Churpek","year":"2021","journal-title":"Crit Care Med"},{"issue":"8","key":"2022091407545455600_ocac109-B13","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1001\/jama.2016.0288","article-title":"Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3)","volume":"315","author":"Seymour","year":"2016","journal-title":"JAMA"},{"issue":"4","key":"2022091407545455600_ocac109-B14","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1007\/s00134-003-1662-x","article-title":"2001 SCCM\/ESICM\/ACCP\/ATS\/SIS International Sepsis Definitions Conference","volume":"29","author":"Levy","year":"2003","journal-title":"Intensive Care Med"},{"issue":"6","key":"2022091407545455600_ocac109-B15","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1164\/rccm.201406-1022OC","article-title":"Multicenter development and validation of a risk stratification tool for ward patients","volume":"190","author":"Churpek","year":"2014","journal-title":"Am J Respir Crit Care Med"},{"issue":"4","key":"2022091407545455600_ocac109-B16","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1097\/CCM.0000000000002936","article-title":"An interpretable machine learning model for accurate prediction of sepsis in the ICU","volume":"46","author":"Nemati","year":"2018","journal-title":"Crit Care Med"},{"issue":"7","key":"2022091407545455600_ocac109-B17","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1097\/CCM.0000000000003123","article-title":"The development of a machine learning inpatient acute kidney injury prediction model","volume":"46","author":"Koyner","year":"2018","journal-title":"Crit Care Med"},{"issue":"2","key":"2022091407545455600_ocac109-B18","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/BF00058655","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach Learn"},{"issue":"6","key":"2022091407545455600_ocac109-B19","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1002\/sam.11348","article-title":"Random Forest missing data algorithms","volume":"10","author":"Tang","year":"2017","journal-title":"Stat Anal Data Min"},{"issue":"2","key":"2022091407545455600_ocac109-B20","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1097\/CCM.0000000000001571","article-title":"Multicenter comparison of machine learning methods and conventional regression for predicting clinical deterioration on the wards","volume":"44","author":"Churpek","year":"2016","journal-title":"Crit Care Med"},{"issue":"5\u20136","key":"2022091407545455600_ocac109-B21","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw"},{"key":"2022091407545455600_ocac109-B22","first-page":"473","author":"Hochreiter","year":"1996"},{"key":"2022091407545455600_ocac109-B23","author":"Cho"},{"key":"2022091407545455600_ocac109-B24","author":"Bai","year":"2018"},{"key":"2022091407545455600_ocac109-B25","author":"Lea","year":"2017"},{"key":"2022091407545455600_ocac109-B26","first-page":"802","volume-title":"Advances in Neural Information Processing Systems","author":"Shi","year":"2015"},{"key":"2022091407545455600_ocac109-B27","first-page":"1394","author":"Siami-Namini","year":"2018"},{"issue":"21","key":"2022091407545455600_ocac109-B28","doi-asserted-by":"crossref","first-page":"16453","DOI":"10.1007\/s00500-020-04954-0","article-title":"Temporal convolutional neural (TCN) network for an effective weather forecasting using time-series data from the local weather station","volume":"24","author":"Hewage","year":"2020","journal-title":"Soft Comput"},{"issue":"7","key":"2022091407545455600_ocac109-B29","doi-asserted-by":"crossref","first-page":"e0220640","DOI":"10.1371\/journal.pone.0220640","article-title":"Combining patient visual timelines with deep learning to predict mortality","volume":"14","author":"Mayampurath","year":"2019","journal-title":"PLoS One"},{"key":"2022091407545455600_ocac109-B30","first-page":"157","author":"Wang","year":"2019"},{"key":"2022091407545455600_ocac109-B31","first-page":"500","volume-title":"Advances in Visual Computing","author":"Bashiri","year":"2018"},{"key":"2022091407545455600_ocac109-B32","doi-asserted-by":"crossref","first-page":"108288","DOI":"10.1016\/j.measurement.2020.108288","article-title":"A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic","volume":"167","author":"Loey","year":"2021","journal-title":"Measurement (Lond)"},{"issue":"3","key":"2022091407545455600_ocac109-B33","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"issue":"11","key":"2022091407545455600_ocac109-B34","doi-asserted-by":"crossref","first-page":"1389","DOI":"10.1109\/LSP.2014.2337313","article-title":"Fast implementation of DeLong\u2019s algorithm for comparing the areas under correlated receiver operating characteristic curves","volume":"21","author":"Sun","year":"2014","journal-title":"IEEE Signal Process Lett"},{"key":"2022091407545455600_ocac109-B35","doi-asserted-by":"crossref","first-page":"101820","DOI":"10.1016\/j.artmed.2020.101820","article-title":"Early detection of sepsis utilizing deep learning on electronic health record event sequences","volume":"104","author":"Lauritsen","year":"2020","journal-title":"Artif Intell Med"},{"key":"2022091407545455600_ocac109-B36","doi-asserted-by":"crossref","first-page":"103957","DOI":"10.1016\/j.compbiomed.2020.103957","article-title":"Automated prediction of sepsis using temporal convolutional network","volume":"127","author":"Kok","year":"2020","journal-title":"Comput Biol Med"},{"key":"2022091407545455600_ocac109-B37","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.jbi.2018.04.007","article-title":"Benchmarking deep learning models on large healthcare datasets","volume":"83","author":"Purushotham","year":"2018","journal-title":"J Biomed Inform"},{"issue":"11","key":"2022091407545455600_ocac109-B38","doi-asserted-by":"crossref","first-page":"e1020-8","DOI":"10.1097\/CCM.0000000000004556","article-title":"The development and validation of a machine learning model to predict bacteremia and fungemia in hospitalized patients using electronic health record data","volume":"48","author":"Bhavani","year":"2020","journal-title":"Crit Care Med"},{"issue":"1","key":"2022091407545455600_ocac109-B39","doi-asserted-by":"crossref","first-page":"26","DOI":"10.4103\/0974-2727.154785","article-title":"Depressed monocytic activity may be a predictor for sepsis","volume":"7","author":"Bhardwaj","year":"2015","journal-title":"J Lab Physicians"},{"issue":"8","key":"2022091407545455600_ocac109-B40","doi-asserted-by":"crossref","first-page":"1018","DOI":"10.1097\/CCM.0000000000003799","article-title":"Monocyte distribution width: a novel indicator of sepsis-2 and sepsis-3 in high-risk emergency department patients","volume":"47","author":"Crouser","year":"2019","journal-title":"Crit Care Med"},{"issue":"1","key":"2022091407545455600_ocac109-B41","doi-asserted-by":"crossref","first-page":"7193","DOI":"10.1038\/s41598-018-24876-0","article-title":"A cluster-then-label semi-supervised learning approach for pathology image classification","volume":"8","author":"Peikari","year":"2018","journal-title":"Sci Rep"},{"issue":"1","key":"2022091407545455600_ocac109-B42","doi-asserted-by":"crossref","first-page":"e23086","DOI":"10.2196\/23086","article-title":"ALBERT-based self-ensemble model with semisupervised learning and data augmentation for clinical semantic textual similarity calculation: algorithm validation study","volume":"9","author":"Li","year":"2021","journal-title":"JMIR Med Inform"},{"key":"2022091407545455600_ocac109-B43","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1186\/s12911-020-1108-1","article-title":"A semi-supervised approach for extracting TCM clinical terms based on feature words","volume":"20 (Suppl 3)","author":"Liu","year":"2020","journal-title":"BMC Med Inform Decis Mak"},{"key":"2022091407545455600_ocac109-B44","author":"Wang","year":"2020"},{"key":"2022091407545455600_ocac109-B45","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/978-3-030-32692-0_18","volume-title":"Machine Learning in Medical Imaging","author":"Imran","year":"2019"},{"key":"2022091407545455600_ocac109-B46","doi-asserted-by":"crossref","first-page":"103863","DOI":"10.1016\/j.infrared.2021.103863","article-title":"Deep calibration transfer: transferring deep learning models between infrared spectroscopy instruments","volume":"117","author":"Mishra","year":"2021","journal-title":"Infrared Phys Technol"},{"key":"2022091407545455600_ocac109-B47","author":"Masko","year":"2017"},{"key":"2022091407545455600_ocac109-B48","first-page":"182","author":"Guill\u00e9n","year":"2015"},{"key":"2022091407545455600_ocac109-B49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.resuscitation.2016.02.005","article-title":"The value of vital sign trends for detecting clinical deterioration on the wards","volume":"102","author":"Churpek","year":"2016","journal-title":"Resuscitation"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/10\/1696\/45798244\/ocac109.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/29\/10\/1696\/45798244\/ocac109.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,24]],"date-time":"2023-11-24T22:24:45Z","timestamp":1700864685000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/29\/10\/1696\/6649188"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,23]]},"references-count":49,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,7,23]]},"published-print":{"date-parts":[[2022,9,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocac109","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,10,1]]},"published":{"date-parts":[[2022,7,23]]}}}