{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T12:17:06Z","timestamp":1771935426030,"version":"3.50.1"},"reference-count":37,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2020,11,5]],"date-time":"2020-11-05T00:00:00Z","timestamp":1604534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["1K23MH121190-01"],"award-info":[{"award-number":["1K23MH121190-01"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"name":"NIH-funded Third Coast Center for AIDS Research","award":["P30 AI117943"],"award-info":[{"award-number":["P30 AI117943"]}]},{"name":"The Center for Research Informatics is funded by the Biological Sciences Division"},{"name":"Institute for Translational Medicine\/CTSA","award":["NIH UL1 TR000430"],"award-info":[{"award-number":["NIH UL1 TR000430"]}]},{"DOI":"10.13039\/100007234","name":"University of Chicago","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007234","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,1,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Adherence to a treatment plan from HIV-positive patients is necessary to decrease their mortality and improve their quality of life, however some patients display poor appointment adherence and become lost to follow-up (LTFU). We applied natural language processing (NLP) to analyze indications towards or against LTFU in HIV-positive patients\u2019 notes.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Unstructured lemmatized notes were labeled with an LTFU or Retained status using a 183-day threshold. An NLP and supervised machine learning system with a linear model and elastic net regularization was trained to predict this status. Prevalence of characteristics domains in the learned model weights were evaluated.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>We analyzed 838 LTFU vs 2964 Retained notes and obtained a weighted F1 mean of 0.912 via nested cross-validation; another experiment with notes from the same patients in both classes showed substantially lower metrics. \u201cComorbidities\u201d were associated with LTFU through, for instance, \u201cHCV\u201d (hepatitis C virus) and likewise \u201cGood adherence\u201d with Retained, represented with \u201cWell on ART\u201d (antiretroviral therapy).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Mentions of mental health disorders and substance use were associated with disparate retention outcomes, however history vs active use was not investigated. There remains further need to model transitions between LTFU and being retained in care over time.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>We provided an important step for the future development of a model that could eventually help to identify patients who are at risk for falling out of care and to analyze which characteristics could be factors for this. Further research is needed to enhance this method with structured electronic medical record fields.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa220","type":"journal-article","created":{"date-parts":[[2020,8,25]],"date-time":"2020-08-25T04:04:56Z","timestamp":1598328296000},"page":"104-112","source":"Crossref","is-referenced-by-count":19,"title":["Development of a predictive model for retention in HIV care using natural language processing of clinical notes"],"prefix":"10.1093","volume":"28","author":[{"given":"Tomasz","family":"Oliwa","sequence":"first","affiliation":[{"name":"Center for Research Informatics, University of Chicago, Chicago, Illinois, USA"}]},{"given":"Brian","family":"Furner","sequence":"additional","affiliation":[{"name":"Center for Research Informatics, University of Chicago, Chicago, Illinois, USA"}]},{"given":"Jessica","family":"Schmitt","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago, Chicago, Illinois, USA"},{"name":"Chicago Center for HIV Elimination, University of Chicago, Chicago, Illinois, USA"}]},{"given":"John","family":"Schneider","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago, Chicago, Illinois, USA"},{"name":"Chicago Center for HIV Elimination, University of Chicago, Chicago, Illinois, USA"}]},{"given":"Jessica P","family":"Ridgway","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago, Chicago, Illinois, USA"},{"name":"Chicago Center for HIV Elimination, University of Chicago, Chicago, Illinois, USA"}]}],"member":"286","published-online":{"date-parts":[[2020,11,5]]},"reference":[{"issue":"1","key":"2021011515545779300_ocaa220-B1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1089\/apc.2008.0132","article-title":"The therapeutic implications of timely linkage and early retention in HIV care","volume":"23","author":"Ulett","year":"2009","journal-title":"AIDS Patient Care STDS"},{"issue":"4","key":"2021011515545779300_ocaa220-B2","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1001\/jamainternmed.2014.8180","article-title":"Human immunodeficiency virus transmission at each step of the care continuum in the United States","volume":"175","author":"Skarbinski","year":"2015","journal-title":"JAMA Intern Med"},{"issue":"11","key":"2021011515545779300_ocaa220-B3","first-page":"e475","article-title":"The Lancet HIV","volume":"4","year":"2017","journal-title":"U=U taking off in 2017. Lancet HIV"},{"key":"2021011515545779300_ocaa220-B4","year":"2019"},{"key":"2021011515545779300_ocaa220-B5"},{"issue":"2","key":"2021011515545779300_ocaa220-B6","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1007\/s10461-018-2215-1","article-title":"Who will show? Predicting missed visits among patients in routine HIV primary care in the United States","volume":"23","author":"Pence","year":"2019","journal-title":"AIDS Behav"},{"issue":"7","key":"2021011515545779300_ocaa220-B7","doi-asserted-by":"crossref","first-page":"e027688","DOI":"10.1136\/bmjopen-2018-027688","article-title":"Using big data analytics to improve HIV medical care utilisation in South Carolina: a study protocol","volume":"9","author":"Olatosi","year":"2019","journal-title":"BMJ Open"},{"issue":"Suppl 1","key":"2021011515545779300_ocaa220-B8","first-page":"S421","article-title":"Electronic data sharing between public health department and clinical providers improves accuracy of HIV retention data","volume":"4","author":"Ridgway","year":"2017","journal-title":"Open Forum Infect Dis"},{"key":"2021011515545779300_ocaa220-B9","article-title":"Percentage of office-based physicians using any electronic health record (EHR)\/electronic medical record (EMR) system and physicians that have a certified EHR\/EMR system, by U","author":"Myrick","year":"2017","journal-title":"S. state: National Electronic Health Records Survey"},{"key":"2021011515545779300_ocaa220-B10","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.jbi.2015.02.003","article-title":"Efficient and sparse feature selection for biomedical text classification via the elastic net: Application to ICU risk stratification from nursing notes","volume":"54","author":"Marafino","year":"2015","journal-title":"J Biomed Inform"},{"issue":"2","key":"2021011515545779300_ocaa220-B11","doi-asserted-by":"crossref","first-page":"e0211116","DOI":"10.1371\/journal.pone.0211116","article-title":"Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records","volume":"14","author":"Carson","year":"2019","journal-title":"PLoS One"},{"issue":"10","key":"2021011515545779300_ocaa220-B12","doi-asserted-by":"crossref","first-page":"e921","DOI":"10.1038\/tp.2015.182","article-title":"Predicting early psychiatric readmission with natural language processing of narrative discharge summaries","volume":"6","author":"Rumshisky","year":"2016","journal-title":"Transl Psychiatry"},{"issue":"3","key":"2021011515545779300_ocaa220-B13","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1097\/MLR.0000000000000651","article-title":"A novel model for predicting rehospitalization risk incorporating physical function, cognitive status, and psychosocial support using natural language processing","volume":"55","author":"Greenwald","year":"2017","journal-title":"Med Care"},{"issue":"2","key":"2021011515545779300_ocaa220-B14","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1097\/QAI.0000000000001580","article-title":"Using clinical notes and natural language processing for automated HIV risk assessment","volume":"77","author":"Feller","year":"2018","journal-title":"J Acquir Immune Defic Syndr"},{"issue":"1","key":"2021011515545779300_ocaa220-B15","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1472-6947-8-32","article-title":"Automated de-identification of free-text medical records","volume":"8","author":"Neamatullah","year":"2008","journal-title":"BMC Med Inform Decis Mak"},{"issue":"23","key":"2021011515545779300_ocaa220-B16","doi-asserted-by":"crossref","first-page":"E215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"issue":"12","key":"2021011515545779300_ocaa220-B17","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1016\/j.ijmedinf.2010.09.007","article-title":"The MITRE Identification Scrubber Toolkit: design, training, and assessment","volume":"79","author":"Aberdeen","year":"2010","journal-title":"Int J Med Inform"},{"key":"2021011515545779300_ocaa220-B18","author":"Finkel","year":"2005"},{"key":"2021011515545779300_ocaa220-B19","doi-asserted-by":"crossref","DOI":"10.1145\/1008992.1009034","article-title":"Feature selection using linear classifier weights","author":"Mladeni\u0107","year":"2004"},{"issue":"5","key":"2021011515545779300_ocaa220-B20","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1097\/QAI.0b013e318273762f","article-title":"Measuring retention in HIV care: the elusive gold standard","volume":"61","author":"Mugavero","year":"2012","journal-title":"J Acquir Immune Defic Syndr"},{"issue":"9","key":"2021011515545779300_ocaa220-B21","doi-asserted-by":"crossref","first-page":"e75761","DOI":"10.1371\/journal.pone.0075761","article-title":"Are they really lost? \u201ctrue\u201d status and reasons for treatment discontinuation among HIV infected patients on antiretroviral therapy considered lost to follow up in Urban Malawi","volume":"8","author":"Tweya","year":"2013","journal-title":"PLoS One"},{"key":"2021011515545779300_ocaa220-B22","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2021011515545779300_ocaa220-B23","author":"M\u00fcller","year":"2016"},{"key":"2021011515545779300_ocaa220-B24","doi-asserted-by":"crossref","DOI":"10.1155\/2019\/8161495","article-title":"HIV care continuum among postpartum women living with HIV in Atlanta","author":"Meade","year":"2019","journal-title":"Infect Dis Obstet Gynecol"},{"key":"2021011515545779300_ocaa220-B25"},{"issue":"5","key":"2021011515545779300_ocaa220-B26","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1097\/COH.0000000000000302","article-title":"Interventions to improve treatment, retention and survival outcomes for adolescents with perinatal HIV-1 transitioning to adult care: moving on up","volume":"11","author":"Judd","year":"2016","journal-title":"Curr Opin HIV AIDS"},{"issue":"13","key":"2021011515545779300_ocaa220-B27","doi-asserted-by":"crossref","first-page":"1945","DOI":"10.1097\/QAD.0000000000000316","article-title":"Adherence to antiretroviral therapy in adolescents living with HIV: systematic review and meta-analysis","volume":"28","author":"Kim","year":"2014","journal-title":"AIDS"},{"issue":"1","key":"2021011515545779300_ocaa220-B28","doi-asserted-by":"crossref","first-page":"18579","DOI":"10.7448\/IAS.16.1.18579","article-title":"Antiretroviral treatment, management challenges and outcomes in perinatally HIV-infected adolescents","volume":"16","author":"Agwu","year":"2013","journal-title":"J Int AIDS Soc"},{"issue":"Suppl 2","key":"2021011515545779300_ocaa220-B29","doi-asserted-by":"crossref","first-page":"S144","DOI":"10.1097\/QAI.0000000000000176","article-title":"Epidemiology of HIV and AIDS among adolescents: current status, inequities, and data gaps","volume":"66","author":"Idele","year":"2014","journal-title":"J Acquir Immune Defic Syndr"},{"issue":"5","key":"2021011515545779300_ocaa220-B30","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1310\/hct1005-299","article-title":"Predictors of retention in HIV care among a national cohort of US veterans","volume":"10","author":"Giordano","year":"2009","journal-title":"HIV Clin Trials"},{"issue":"6","key":"2021011515545779300_ocaa220-B31","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1037\/hea0000606","article-title":"Mental health and retention in HIV care: a systematic review and meta-analysis","volume":"37","author":"Rooks-Peck","year":"2018","journal-title":"Health Psychol"},{"issue":"3","key":"2021011515545779300_ocaa220-B32","doi-asserted-by":"crossref","first-page":"752","DOI":"10.1007\/s10461-016-1644-y","article-title":"Predictors of adult retention in HIV care: a systematic review","volume":"22","author":"Bulsara","year":"2018","journal-title":"AIDS Behav"},{"issue":"3","key":"2021011515545779300_ocaa220-B33","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1007\/s10461-017-1826-2","article-title":"Influence of substance use disorders on 2-year HIV care retention in the United States","volume":"22","author":"Hartzler","year":"2018","journal-title":"AIDS Behav"},{"issue":"5","key":"2021011515545779300_ocaa220-B34","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1080\/09540121.2015.1118431","article-title":"Regular clinic attendance in two large San Francisco HIV primary care settings","volume":"28","author":"Cohen","year":"2016","journal-title":"AIDS Care"},{"issue":"3","key":"2021011515545779300_ocaa220-B35","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1007\/s10461-016-1638-9","article-title":"Predictors of HIV care engagement, antiretroviral medication adherence, and viral suppression among people living with HIV infection in St. Petersburg, Russia","volume":"22","author":"Amirkhanian","year":"2018","journal-title":"AIDS Behav"},{"issue":"15","key":"2021011515545779300_ocaa220-B36","doi-asserted-by":"crossref","first-page":"2217","DOI":"10.1097\/QAD.0000000000001936","article-title":"Beyond binary retention in HIV care: predictors of the dynamic processes of patient engagement, disengagement, and re-entry into care in a US clinical cohort","volume":"32","author":"Lee","year":"2018","journal-title":"AIDS"},{"issue":"5","key":"2021011515545779300_ocaa220-B37","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/1\/104\/35885559\/ocaa220.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/1\/104\/35885559\/ocaa220.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,1,15]],"date-time":"2021-01-15T21:04:31Z","timestamp":1610744671000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/1\/104\/5956344"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,5]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,11,5]]},"published-print":{"date-parts":[[2021,1,15]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa220","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,1]]},"published":{"date-parts":[[2020,11,5]]}}}