{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T23:12:06Z","timestamp":1773357126402,"version":"3.50.1"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2020,2,17]],"date-time":"2020-02-17T00:00:00Z","timestamp":1581897600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"intramural research program"},{"name":"US National Library of Medicine"},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"NIH HPC Biowulf cluster"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Reliable longitudinal risk prediction for hospitalized patients is needed to provide quality care. Our goal is to develop a generalizable model capable of leveraging clinical notes to predict healthcare-associated diseases 24\u201396 hours in advance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We developed a reCurrent Additive Network for Temporal RIsk Prediction (CANTRIP) to predict the risk of hospital acquired (occurring \u2265 48 hours after admission) acute kidney injury, pressure injury, or anemia \u2265 24 hours before it is implicated by the patient\u2019s chart, labs, or notes. We rely on the MIMIC III critical care database and extract distinct positive and negative cohorts for each disease. We retrospectively determine the date-of-event using structured and unstructured criteria and use it as a form of indirect supervision to train and evaluate CANTRIP to predict disease risk using clinical notes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our experiments indicate that CANTRIP, operating on text alone, obtains 74%\u201387% area under the curve and 77%\u201385% Specificity. Baseline shallow models showed lower performance on all metrics, while bidirectional long short-term memory obtained the highest Sensitivity at the cost of significantly lower Specificity and Precision.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>Proper model architecture allows clinical text to be successfully harnessed to predict nosocomial disease, outperforming shallow models and obtaining similar performance to disease-specific models reported in the literature.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Clinical text on its own can provide a competitive alternative to traditional structured features (eg, lab values, vital signs). CANTRIP is able to generalize across nosocomial diseases without disease-specific feature extraction and is available at https:\/\/github.com\/h4ste\/cantrip.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocaa004","type":"journal-article","created":{"date-parts":[[2020,1,18]],"date-time":"2020-01-18T12:09:19Z","timestamp":1579349359000},"page":"567-576","source":"Crossref","is-referenced-by-count":29,"title":["A customizable deep learning model for nosocomial risk prediction from critical care notes with indirect supervision"],"prefix":"10.1093","volume":"27","author":[{"given":"Travis R","family":"Goodwin","sequence":"first","affiliation":[{"name":"Lister Hill National Center for Biomedical Communications, US National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA"}]},{"given":"Dina","family":"Demner-Fushman","sequence":"additional","affiliation":[{"name":"Lister Hill National Center for Biomedical Communications, US National Library of Medicine, National Institutes of Health, Bethesda, Maryland, USA"}]}],"member":"286","published-online":{"date-parts":[[2020,2,17]]},"reference":[{"issue":"13","key":"2020110613074310800_ocaa004-B1","doi-asserted-by":"crossref","first-page":"1198","DOI":"10.1056\/NEJMoa1306801","article-title":"Multistate point-prevalence survey of health care\u2013associated infections","volume":"370","author":"Magill","year":"2014","journal-title":"N Engl J Med"},{"key":"2020110613074310800_ocaa004-B2","doi-asserted-by":"crossref","first-page":"197","DOI":"10.2147\/CEOR.S102505","article-title":"Estimated hospital costs associated with preventable health care-associated infections if health care antiseptic products were unavailable","volume":"8","author":"Schmier","year":"2016","journal-title":"Clincoecon Outcomes Res"},{"key":"2020110613074310800_ocaa004-B3","article-title":"Hospital-acquired anemia: prevalence, outcomes, and healthcare implications","volume":"8","author":"Henderson","year":"2013","journal-title":"J Hosp Med"},{"issue":"10","key":"2020110613074310800_ocaa004-B4","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.1007\/s00134-002-1487-z","article-title":"Pressure ulcers in intensive care patients: a review of risks and prevention","volume":"28","author":"Keller","year":"2002","journal-title":"Intensive Care Med"},{"issue":"2","key":"2020110613074310800_ocaa004-B5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.12788\/jhm.2683","article-title":"Cost of acute kidney injury in hospitalized patients","volume":"12","author":"Silver","year":"2017","journal-title":"J Hosp Med"},{"issue":"1","key":"2020110613074310800_ocaa004-B6","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613074310800_ocaa004-B7","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s12911-017-0418-4","article-title":"Early recognition of multiple sclerosis using natural language processing of the electronic health record","volume":"17","author":"Chase","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"issue":"5","key":"2020110613074310800_ocaa004-B8","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1016\/j.jbi.2009.08.007","article-title":"What can natural language processing do for clinical decision support?","volume":"42","author":"Demner-Fushman","year":"2009","journal-title":"J Biomed Inform"},{"key":"2020110613074310800_ocaa004-B9","first-page":"2493","article-title":"Natural language processing (almost) from scratch","volume":"12","author":"Collobert","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2020110613074310800_ocaa004-B10","author":"Goodwin","year":"2015"},{"key":"2020110613074310800_ocaa004-B11","author":"Goodwin"},{"key":"2020110613074310800_ocaa004-B12","first-page":"78","article-title":"Inferring the interactions of risk factors from EHRs","volume":"2016","author":"Goodwin","year":"2016","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"issue":"11","key":"2020110613074310800_ocaa004-B13","doi-asserted-by":"crossref","first-page":"3365","DOI":"10.1681\/ASN.2004090740","article-title":"Acute kidney injury, mortality, length of stay, and costs in hospitalized patients","volume":"16","author":"Chertow","year":"2005","journal-title":"J Am Soc Nephrol"},{"issue":"4","key":"2020110613074310800_ocaa004-B14","doi-asserted-by":"crossref","first-page":"1613","DOI":"10.1111\/j.1523-1755.2004.00927.x","article-title":"Spectrum of acute renal failure in the intensive care unit: the PICARD experience","volume":"66","author":"Mehta","year":"2004","journal-title":"Kidney Int"},{"issue":"11","key":"2020110613074310800_ocaa004-B15","doi-asserted-by":"crossref","first-page":"1935","DOI":"10.2215\/CJN.00280116","article-title":"Development of a multicenter ward-based AKI prediction model","volume":"11","author":"Koyner","year":"2016","journal-title":"Clin J Am Soc Nephrol"},{"issue":"6","key":"2020110613074310800_ocaa004-B16","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1007\/s00134-017-4678-3","article-title":"AKIpredictor, an online prognostic calculator for acute kidney injury in adult critically ill patients: development, validation and comparison to serum neutrophil gelatinase-associated lipocalin","volume":"43","author":"Flechet","year":"2017","journal-title":"Intensive Care Med"},{"issue":"1","key":"2020110613074310800_ocaa004-B17","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1186\/s12911-016-0277-4","article-title":"Prediction and detection models for acute kidney injury in hospitalized older adults","volume":"16","author":"Kate","year":"2016","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613074310800_ocaa004-B18","doi-asserted-by":"crossref","first-page":"1888","DOI":"10.1093\/ndt\/gfu082","article-title":"A real-time electronic alert to improve detection of acute kidney injury in a large teaching hospital","volume":"29","author":"Porter","year":"2014","journal-title":"Nephrol Dial Transplant"},{"issue":"5","key":"2020110613074310800_ocaa004-B19","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann Stat"},{"key":"2020110613074310800_ocaa004-B20","article-title":"Prediction of acute kidney injury with a machine learning algorithm using electronic health record data","volume":"5. doi: 10.1177\/2054358118776326","author":"Mohamadlou","year":"2018","journal-title":"Can J Kidney Health Dis"},{"key":"2020110613074310800_ocaa004-B21","doi-asserted-by":"crossref","first-page":"c179","DOI":"10.1159\/000339789","article-title":"KDIGO clinical practice guidelines for acute kidney injury","volume":"120","author":"Khwaja","year":"2012","journal-title":"Nephron Clin Pract"},{"issue":"7767","key":"2020110613074310800_ocaa004-B22","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1038\/s41586-019-1390-1","article-title":"A clinically applicable approach to continuous prediction of future acute kidney injury","volume":"572","author":"Toma\u0161ev","year":"2019","journal-title":"Nature"},{"issue":"3","key":"2020110613074310800_ocaa004-B23","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1515\/jccm-2016-0017","article-title":"Anemia in intensive care: a review of current concepts","volume":"2","author":"Rawal","year":"2016","journal-title":"J Crit Care Med"},{"issue":"6","key":"2020110613074310800_ocaa004-B24","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1111\/j.1525-1497.2005.0094.x","article-title":"Do blood tests cause anemia in hospitalized patients? The effect of diagnostic phlebotomy on hemoglobin and hematocrit levels","volume":"20","author":"Thavendiranathan","year":"2005","journal-title":"J Gen Intern Med"},{"issue":"6","key":"2020110613074310800_ocaa004-B25","doi-asserted-by":"crossref","first-page":"eS1","DOI":"10.4037\/ajcc2013729","article-title":"Anemia, bleeding, and blood transfusion in the intensive care unit: causes, risks, costs, and new strategies","volume":"22","author":"McEvoy","year":"2013","journal-title":"Am J Crit Care"},{"issue":"4","key":"2020110613074310800_ocaa004-B26","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1177\/0310057X0603400414","article-title":"Highly conservative phlebotomy in adult intensive care: a prospective randomized controlled trial","volume":"34","author":"Harber","year":"2006","journal-title":"Anaesth Intensive Care"},{"issue":"5","key":"2020110613074310800_ocaa004-B27","doi-asserted-by":"crossref","first-page":"R140","DOI":"10.1186\/cc5054","article-title":"Anemia, transfusion, and phlebotomy practices in critically ill patients with prolonged ICU length of stay: a cohort study","volume":"10","author":"Chant","year":"2006","journal-title":"Crit Care"},{"issue":"3","key":"2020110613074310800_ocaa004-B28","doi-asserted-by":"crossref","first-page":"2057","DOI":"10.1007\/s10916-011-9668-3","article-title":"Artificial intelligence models for predicting iron deficiency anemia and iron serum level based on accessible laboratory data","volume":"36","author":"Azarkhish","year":"2012","journal-title":"J Med Syst"},{"issue":"5","key":"2020110613074310800_ocaa004-B29","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1007\/s10620-017-4512-3","article-title":"A novel model for predicting incident moderate to severe anemia and iron deficiency in patients with newly diagnosed ulcerative colitis","volume":"62","author":"Khan","year":"2017","journal-title":"Dig Dis Sci"},{"issue":"4","key":"2020110613074310800_ocaa004-B30","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1016\/j.amjsurg.2009.12.021","article-title":"High cost of stage IV pressure ulcers","volume":"200","author":"Brem","year":"2010","journal-title":"Am J Surg"},{"issue":"12","key":"2020110613074310800_ocaa004-B31","doi-asserted-by":"crossref","first-page":"1435","DOI":"10.1111\/j.1532-5415.1996.tb04067.x","article-title":"Hospital-acquired pressure ulcers and risk of death","volume":"44","author":"Thomas","year":"1996","journal-title":"J Am Geriatr Soc"},{"issue":"2","key":"2020110613074310800_ocaa004-B32","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/S0029-6465(22)01289-0","article-title":"A clinical trial of the Braden Scale for Predicting Pressure Sore Risk","volume":"22","author":"Bergstrom","year":"1987","journal-title":"Nurs Clin North Am"},{"key":"2020110613074310800_ocaa004-B33","doi-asserted-by":"crossref","first-page":"514","DOI":"10.4037\/ajcc2013991","article-title":"Predictive validity of the Braden scale for patients in intensive care units","volume":"22","author":"Hyun","year":"2013","journal-title":"Am J Crit Care"},{"issue":"1","key":"2020110613074310800_ocaa004-B34","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1136\/qshc.2005.015362","article-title":"Prediction of pressure ulcer development in hospitalized patients: a tool for risk assessment","volume":"15","author":"Schoonhoven","year":"2006","journal-title":"Qual Saf Health Care"},{"issue":"1","key":"2020110613074310800_ocaa004-B35","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"issue":"23","key":"2020110613074310800_ocaa004-B36","doi-asserted-by":"crossref","first-page":"E215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"2020110613074310800_ocaa004-B37","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1055\/s-0038-1634945","article-title":"The Unified Medical Language System","volume":"32","author":"Lindberg","year":"1993","journal-title":"Methods Inf Med"},{"issue":"4","key":"2020110613074310800_ocaa004-B38","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1093\/jamia\/ocw177","article-title":"MetaMap Lite: an evaluation of a new Java implementation of MetaMap","volume":"24","author":"Demner-Fushman","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613074310800_ocaa004-B39","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.jbi.2018.08.002","article-title":"Trie-based rule processing for clinical NLP: a use-case study of n-trie, making the ConText algorithm more efficient and scalable","volume":"85","author":"Shi","year":"2018","journal-title":"J Biomed Inform"},{"key":"2020110613074310800_ocaa004-B40","first-page":"677","article-title":"Extending the NegEx lexicon for multiple languages","volume":"192","author":"Chapman","year":"2013","journal-title":"Stud Health Technol Inform"},{"key":"2020110613074310800_ocaa004-B41","first-page":"151","article-title":"A prototype system to support evidence-based practice","volume":"2008","author":"Demner-Fushman","year":"2008","journal-title":"AMIA Annu Symp Proc"},{"key":"2020110613074310800_ocaa004-B42","first-page":"5998","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017;","author":"Vaswani"},{"issue":"7","key":"2020110613074310800_ocaa004-B43","doi-asserted-by":"crossref","first-page":"1711","DOI":"10.1097\/CCM.0b013e31828a24fe","article-title":"A new severity of illness scale using a subset of Acute Physiology and Chronic Health Evaluation data elements shows comparable predictive accuracy","volume":"41","author":"Johnson","year":"2013","journal-title":"Crit Care Med"},{"issue":"8","key":"2020110613074310800_ocaa004-B44","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"issue":"3","key":"2020110613074310800_ocaa004-B45","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support vector machine","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach Learn"},{"key":"2020110613074310800_ocaa004-B46","first-page":"85","author":"Srivastava","year":"2015"},{"key":"2020110613074310800_ocaa004-B47","author":"Hendrycks"},{"key":"2020110613074310800_ocaa004-B48","author":"Devlin"},{"key":"2020110613074310800_ocaa004-B49","author":"Lee"},{"key":"2020110613074310800_ocaa004-B50","author":"Glorot","year":"2011"},{"key":"2020110613074310800_ocaa004-B51","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/978-3-642-23808-6_10","volume-title":"Machine Learning and Knowledge Discovery in Databases","author":"Sechidis","year":"2011"},{"key":"2020110613074310800_ocaa004-B52","first-page":"22","volume-title":"Proceedings of the First International Workshop on Learning with Imbalanced Domains: theory and Applications. ECML-PKDD","author":"Szyma\u0144ski","year":"2017"},{"key":"2020110613074310800_ocaa004-B53","author":"Szyma\u0144ski","year":"2017"},{"key":"2020110613074310800_ocaa004-B54","author":"Guo"},{"issue":"2","key":"2020110613074310800_ocaa004-B55","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","article-title":"Comparison of the predicted and observed secondary structure of T4 phage lysozyme","volume":"405","author":"Matthews","year":"1975","journal-title":"Biochim Biophys Acta BBA - Protein Struct"},{"issue":"6","key":"2020110613074310800_ocaa004-B56","doi-asserted-by":"crossref","first-page":"e0177678","DOI":"10.1371\/journal.pone.0177678","article-title":"Optimal classifier for imbalanced data using Matthews Correlation Coefficient metric","volume":"12","author":"Boughorbel","year":"2017","journal-title":"PLoS One"},{"issue":"2","key":"2020110613074310800_ocaa004-B57","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1111\/j.1365-2648.2009.05153.x","article-title":"Braden Scale: evaluation of clinical usefulness in an intensive care unit","volume":"66","author":"Cho","year":"2010","journal-title":"J Adv Nurs"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/4\/567\/34152273\/ocaa004.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/4\/567\/34152273\/ocaa004.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,10,11]],"date-time":"2022-10-11T22:15:02Z","timestamp":1665526502000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/4\/567\/5739340"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,17]]},"references-count":57,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2,17]]},"published-print":{"date-parts":[[2020,4,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa004","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,4]]},"published":{"date-parts":[[2020,2,17]]}}}