{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,25]],"date-time":"2026-10-25T16:27:18Z","timestamp":1792945638033,"version":"4.5.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T00:00:00Z","timestamp":1658793600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,26]],"date-time":"2022-07-26T00:00:00Z","timestamp":1658793600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000200","name":"United States Agency for International Development","doi-asserted-by":"crossref","award":["72067418CA00029"],"award-info":[{"award-number":["72067418CA00029"]}],"id":[{"id":"10.13039\/100000200","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Eunice Kennedy Shriver National Institute of Child Health and Human Development","award":["R01HD103466"],"award-info":[{"award-number":["R01HD103466"]}]},{"name":"Eunice Kennedy Shriver National Institute of Child Health and Human Development","award":["R01HD103466"],"award-info":[{"award-number":["R01HD103466"]}]},{"DOI":"10.13039\/100000060","name":"National Institute of Allergy and Infectious Diseases","doi-asserted-by":"crossref","award":["R01AI152149"],"award-info":[{"award-number":["R01AI152149"]}],"id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000060","name":"National Institute of Allergy and Infectious Diseases","doi-asserted-by":"crossref","award":["R01AI152149"],"award-info":[{"award-number":["R01AI152149"]}],"id":[{"id":"10.13039\/100000060","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>HIV treatment programs face challenges in identifying patients at risk for loss-to-follow-up and uncontrolled viremia. We applied predictive machine learning algorithms to anonymised, patient-level HIV programmatic data from two districts in South Africa, 2016\u20132018. We developed patient risk scores for two outcomes: (1) visit attendance\u2009\u2264\u200928\u00a0days of the next scheduled clinic visit and (2) suppression of the next HIV viral load (VL). Demographic, clinical, behavioral and laboratory data were investigated in multiple models as predictor variables of attending the next scheduled visit and VL results at the next test. Three classification algorithms (logistical regression, random forest and AdaBoost) were evaluated for building predictive models. Data were randomly sampled on a 70\/30 split into a training and test set. The training set included a balanced set of positive and negative examples from which the classification algorithm could learn. The predictor variable data from the unseen test set were given to the model, and each predicted outcome was scored against known outcomes. Finally, we estimated performance metrics for each model in terms of sensitivity, specificity, positive and negative predictive value and area under the curve (AUC). In total, 445,636 patients were included in the retention model and 363,977 in the VL model. The predictive metric (AUC) ranged from 0.69 for attendance at the next scheduled visit to 0.76 for VL suppression, suggesting that the model correctly classified whether a scheduled visit would be attended in 2 of 3 patients and whether the VL result at the next test would be suppressed in approximately 3 of 4 patients. Variables that were important predictors of both outcomes included prior late visits, number of prior VL tests, time since their last visit, number of visits on their current regimen, age, and treatment duration. For retention, the number of visits at the current facility and the details of the next appointment date were also predictors, while for VL suppression, other predictors included the range of the previous VL value. Machine learning can identify HIV patients at risk for disengagement and unsuppressed VL. Predictive modeling can improve the targeting of interventions through differentiated models of care before patients disengage from treatment programmes, increasing cost-effectiveness and improving patient outcomes.<\/jats:p>","DOI":"10.1038\/s41598-022-16062-0","type":"journal-article","created":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T09:18:02Z","timestamp":1658913482000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":62,"title":["Applying machine learning and predictive modeling to retention and viral suppression in South African HIV treatment cohorts"],"prefix":"10.1038","volume":"12","author":[{"given":"Mhairi","family":"Maskew","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kieran","family":"Sharpey-Schafer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lucien","family":"De Voux","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"Crompton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacob","family":"Bor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus","family":"Rennick","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Admire","family":"Chirowodza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacqui","family":"Miot","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seithati","family":"Molefi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chuka","family":"Onaga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pappie","family":"Majuba","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ian","family":"Sanne","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pedro","family":"Pisa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,7,26]]},"reference":[{"key":"16062_CR1","unstructured":"UNAIDS Joint United Nations Programme on HIV\/AIDS. Fact Sheet\u2014World AIDS Day 2020. Geneva, Switzerland. https:\/\/www.unaids.org\/sites\/default\/files\/media_asset\/UNAIDS_FactSheet_en.pdf (2020)."},{"key":"16062_CR2","unstructured":"Joint United Nations Programme on HIV\/AIDS (UNAIDS). South Africa takes bold step to provide HIV treatment for all. 2016."},{"key":"16062_CR3","unstructured":"UNAIDS Joint United Nations Programme on HIV\/AIDS. Country factsheets. Geneva, Switzerland. https:\/\/www.unaids.org\/en\/regionscountries\/countries\/southafrica (2020)."},{"key":"16062_CR4","unstructured":"Johnson, L. & Dorrington, R. Thembisa Version 4.3: A Model for Evaluating the Impact of HIV\/AIDS in South Africa. Cape Town. https:\/\/www.thembisa.org\/publications (2020)."},{"issue":"Suppl 1","key":"16062_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1111\/j.1365-3156.2010.02508.x","volume":"15","author":"MP Fox","year":"2010","unstructured":"Fox, M. P. & Rosen, S. Patient retention in antiretroviral therapy programs up to three years on treatment in sub-Saharan Africa, 2007\u20132009: Systematic review. Trop. Med. Int. Health 15(Suppl 1), 1\u201315 (2010).","journal-title":"Trop. Med. Int. Health"},{"key":"16062_CR6","unstructured":"Johnson, L. F. & Dorrington, R. E. Modelling the impact of HIV in South Africa\u2019s provinces: 2018 update (2018)."},{"key":"16062_CR7","unstructured":"Johnson, L. F. & Dorrington, R. E. Thembisa version 4.2: A model for evaluating the impact of HIV\/AIDS in South Africa (2019)."},{"key":"16062_CR8","doi-asserted-by":"publisher","DOI":"10.1002\/jia2.25640","author":"L Long","year":"2020","unstructured":"Long, L. et al. Retention in care and viral suppression in differentiated service delivery models for HIV treatment delivery in sub-Saharan Africa: A rapid systematic review. J. Int. AIDS Soc. https:\/\/doi.org\/10.1002\/jia2.25640 (2020).","journal-title":"J. Int. AIDS Soc."},{"key":"16062_CR9","unstructured":"South African National Department of Health. Minimum Package of Interventions to Support Linkage to care Adherence and Retention in Care. Pretoria, South Africa (2020)."},{"key":"16062_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.12688\/gatesopenres.13159.1","volume":"4","author":"S Rosen","year":"2020","unstructured":"Rosen, S., Grimsrud, A., Ehrenkranz, P. & Katz, I. Models of service delivery for optimizing a patient\u2019s first six months on antiretroviral therapy for HIV: An applied research agenda. Gates Open Res. 4, 1\u201315 (2020).","journal-title":"Gates Open Res."},{"key":"16062_CR11","unstructured":"Fahner G. Developing Transparent Credit Risk Scorecards More Effectively: An Explainable Artificial Intelligence Approach (2018)."},{"key":"16062_CR12","unstructured":"PEPFAR (U.S. President\u2019s Emergency Plan for AIDS Relief). Monitoring, Evaluation, and Reporting Indicator Reference Guide (2019)."},{"key":"16062_CR13","unstructured":"Republic of South Africa - National Department of Health. National Consolidated Guidelines: For the prevention of mother-to-child transmission of HIV (PMTCT) and the management of HIV in children, adolescents and adults (2015)."},{"key":"16062_CR14","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1007\/978-1-4899-7687-1_110","volume-title":"Encyclopedia of Machine Learning and Data Mining","author":"CX Ling","year":"2017","unstructured":"Ling, C. X. & Sheng, V. S. Class imbalance problem. In Encyclopedia of Machine Learning and Data Mining (eds Sammut, C. & Webb, G. I.) 204\u2013205 (Springer US, 2017). https:\/\/doi.org\/10.1007\/978-1-4899-7687-1_110."},{"issue":"1","key":"16062_CR15","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1006\/jcss.1997.1504","volume":"55","author":"Y Freund","year":"1997","unstructured":"Freund, Y. & Schapire, R. E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55(1), 119\u2013139 (1997).","journal-title":"J. Comput. Syst. Sci."},{"key":"16062_CR16","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."},{"issue":"1","key":"16062_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4102\/sajhivmed.v20i1.984","volume":"20","author":"S Kaplan","year":"2019","unstructured":"Kaplan, S., Nteso, K. S., Ford, N., Boulle, A. & Meintjes, G. Loss to follow-up from antiretroviral therapy clinics: A systematic review and meta-analysis of published studies in South Africa from 2011 to 2015. S. Afr. J. HIV Med. 20(1), 1\u201310 (2019).","journal-title":"S. Afr. J. HIV Med."},{"issue":"8","key":"16062_CR18","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T Fawcett","year":"2006","unstructured":"Fawcett, T. An introduction to ROC analysis. Pattern Recognit. Lett. 27(8), 861\u2013874 (2006).","journal-title":"Pattern Recognit. Lett."},{"issue":"5","key":"16062_CR19","doi-asserted-by":"publisher","first-page":"1113","DOI":"10.1007\/s10796-020-10022-7","volume":"22","author":"JM Johnson","year":"2020","unstructured":"Johnson, J. M. & Khoshgoftaar, T. M. The effects of data sampling with deep learning and highly imbalanced big data. Inf. Syst. Front. 22(5), 1113\u20131131. https:\/\/doi.org\/10.1007\/s10796-020-10022-7 (2020).","journal-title":"Inf. Syst. Front."},{"key":"16062_CR20","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0067863","author":"Q Wei","year":"2013","unstructured":"Wei, Q. & Dunbrack, R. L. The role of balanced training and testing data sets for binary classifiers in bioinformatics. PLoS ONE https:\/\/doi.org\/10.1371\/journal.pone.0067863 (2013).","journal-title":"PLoS ONE"},{"issue":"6122","key":"16062_CR21","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1126\/science.1230413","volume":"339","author":"J Bor","year":"2013","unstructured":"Bor, J., Herbst, A. J., Newell, M.-L. & B\u00e4rnighausen, T. Increases in adult life expectancy in rural South Africa: Valuing the scale-up of HIV treatment. Science (80-) 339(6122), 961\u2013965 (2013).","journal-title":"Science (80-)"},{"issue":"4","key":"16062_CR22","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1093\/cid\/ciw335","volume":"63","author":"CE Oldenburg","year":"2016","unstructured":"Oldenburg, C. E. et al. Antiretroviral therapy to prevent HIV acquisition in serodiscordant couples in a hyperendemic community in rural South Africa. Clin. Infect. Dis. 63(4), 548\u2013554 (2016).","journal-title":"Clin. Infect. Dis."},{"key":"16062_CR23","volume-title":"Anna Karenina","author":"L Tolstoy","year":"1961","unstructured":"Tolstoy, L. Anna Karenina 6th edn. (New American Library, 1961).","edition":"6"},{"key":"16062_CR24","doi-asserted-by":"publisher","first-page":"e1002643","DOI":"10.1371\/journal.pmed.1002643","volume":"15","author":"M Fox","year":"2018","unstructured":"Fox, M. et al. How much is retention in HIV care underestimated due to patient transfers? Estimating retention using a national laboratory database in South Africa. PLoS Med. 15, e1002643 (2018).","journal-title":"PLoS Med."},{"issue":"1","key":"16062_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12889-016-3025-x","volume":"16","author":"M Evangeli","year":"2016","unstructured":"Evangeli, M., Newell, M. L. & McGrath, N. Factors associated with pre-ART loss-to-follow up in adults in rural KwaZulu-Natal, South Africa: A prospective cohort study. BMC Public Health 16(1), 1\u201313. https:\/\/doi.org\/10.1186\/s12889-016-3025-x (2016).","journal-title":"BMC Public Health"},{"issue":"10","key":"16062_CR26","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.1016\/S1473-3099(19)30474-8","volume":"19","author":"S Blower","year":"2019","unstructured":"Blower, S. & Okano, J. T. Precision public health and HIV in Africa. Lancet Infect. Dis. 19(10), 1050\u20131052 (2019).","journal-title":"Lancet Infect. Dis."},{"issue":"6","key":"16062_CR27","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1056\/NEJMoa1105243","volume":"365","author":"MS Cohen","year":"2011","unstructured":"Cohen, M. S. et al. Prevention of HIV-1 infection with early antiretroviral therapy. N. Engl. J. Med. 365(6), 493\u2013505 (2011).","journal-title":"N. Engl. J. Med."},{"issue":"10189","key":"16062_CR28","doi-asserted-by":"publisher","first-page":"2428","DOI":"10.1016\/S0140-6736(19)30418-0","volume":"393","author":"AJ Rodger","year":"2019","unstructured":"Rodger, A. J. et al. Risk of HIV transmission through condomless sex in serodifferent gay couples with the HIV-positive partner taking suppressive antiretroviral therapy (PARTNER): Final results of a multicentre, prospective, observational study. Lancet 393(10189), 2428\u20132438 (2019).","journal-title":"Lancet"},{"issue":"13","key":"16062_CR29","doi-asserted-by":"publisher","first-page":"921","DOI":"10.1056\/NEJM200003303421303","volume":"342","author":"T Quinn","year":"2000","unstructured":"Quinn, T. et al. Viral load and heterosexual transmission of human immunodeficiency virus type 1. N. Engl. J. Med. 342(13), 921\u2013929 (2000).","journal-title":"N. Engl. J. Med."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-16062-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-16062-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-16062-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T23:42:01Z","timestamp":1669333321000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-16062-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,26]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["16062"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-16062-0","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.02.03.21251100","asserted-by":"object"}]},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,26]]},"assertion":[{"value":"21 December 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 July 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 July 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"12715"}}