{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T08:50:54Z","timestamp":1775724654295,"version":"3.50.1"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,9,9]],"date-time":"2020-09-09T00:00:00Z","timestamp":1599609600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,9,9]],"date-time":"2020-09-09T00:00:00Z","timestamp":1599609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Tuberculosis (TB) is the leading cause of preventable death in HIV-positive patients, and yet often remains undiagnosed and untreated. Chest x-ray is often used to assist in diagnosis, yet this presents additional challenges due to atypical radiographic presentation and radiologist shortages in regions where co-infection is most common. We developed a deep learning algorithm to diagnose TB using clinical information and chest x-ray images from 677 HIV-positive patients with suspected TB from two hospitals in South Africa. We then sought to determine whether the algorithm could assist clinicians in the diagnosis of TB in HIV-positive patients as a web-based diagnostic assistant. Use of the algorithm resulted in a modest but statistically significant improvement in clinician accuracy (<jats:italic>p<\/jats:italic>\u2009=\u20090.002), increasing the mean clinician accuracy from 0.60 (95% CI 0.57, 0.63) without assistance to 0.65 (95% CI 0.60, 0.70) with assistance. However, the accuracy of assisted clinicians was significantly lower (<jats:italic>p<\/jats:italic>\u2009&lt;\u20090.001) than that of the stand-alone algorithm, which had an accuracy of 0.79 (95% CI 0.77, 0.82) on the same unseen test cases. These results suggest that deep learning assistance may improve clinician accuracy in TB diagnosis using chest x-rays, which would be valuable in settings with a high burden of HIV\/TB co-infection. Moreover, the high accuracy of the stand-alone algorithm suggests a potential value particularly in settings with a scarcity of radiological expertise.<\/jats:p>","DOI":"10.1038\/s41746-020-00322-2","type":"journal-article","created":{"date-parts":[[2020,9,10]],"date-time":"2020-09-10T11:32:35Z","timestamp":1599737555000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":107,"title":["CheXaid: deep learning assistance for physician diagnosis of tuberculosis using chest x-rays in patients with HIV"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8030-3727","authenticated-orcid":false,"given":"Pranav","family":"Rajpurkar","sequence":"first","affiliation":[]},{"given":"Chloe","family":"O\u2019Connell","sequence":"additional","affiliation":[]},{"given":"Amit","family":"Schechter","sequence":"additional","affiliation":[]},{"given":"Nishit","family":"Asnani","sequence":"additional","affiliation":[]},{"given":"Jason","family":"Li","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4471-477X","authenticated-orcid":false,"given":"Amirhossein","family":"Kiani","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7335-3339","authenticated-orcid":false,"given":"Robyn L.","family":"Ball","sequence":"additional","affiliation":[]},{"given":"Marc","family":"Mendelson","sequence":"additional","affiliation":[]},{"given":"Gary","family":"Maartens","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4300-0372","authenticated-orcid":false,"given":"Dani\u00ebl J.","family":"van Hoving","sequence":"additional","affiliation":[]},{"given":"Rulan","family":"Griesel","sequence":"additional","affiliation":[]},{"given":"Andrew Y.","family":"Ng","sequence":"additional","affiliation":[]},{"given":"Tom H.","family":"Boyles","sequence":"additional","affiliation":[]},{"given":"Matthew P.","family":"Lungren","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,9,9]]},"reference":[{"key":"322_CR1","unstructured":"WHO | Global tuberculosis report 2018. WHO. http:\/\/www.who.int\/tb\/publications\/global_report\/en\/ (2019)."},{"key":"322_CR2","doi-asserted-by":"publisher","first-page":"1987","DOI":"10.1097\/QAD.0000000000000802","volume":"29","author":"RK Gupta","year":"2015","unstructured":"Gupta, R. K., Lucas, S. B., Fielding, K. L. & Lawn, S. D. Prevalence of tuberculosis in post-mortem studies of HIV-infected adults and children in resource-limited settings: a systematic review and meta-analysis. AIDS Lond. Engl. 29, 1987\u20132002 (2015).","journal-title":"AIDS Lond. Engl."},{"key":"322_CR3","doi-asserted-by":"publisher","first-page":"S201","DOI":"10.1086\/651492","volume":"50 Suppl 3","author":"H Getahun","year":"2010","unstructured":"Getahun, H., Gunneberg, C., Granich, R. & Nunn, P. HIV infection-associated tuberculosis: the epidemiology and the response. Clin. Infect. Dis. Publ. Infect. Dis. Soc. Am. 50 Suppl 3, S201\u2013207 (2010).","journal-title":"Clin. Infect. Dis. Publ. Infect. Dis. Soc. Am."},{"key":"322_CR4","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0020755","volume":"6","author":"M Straetemans","year":"2011","unstructured":"Straetemans, M., Glaziou, P., Bierrenbach, A. L., Sismanidis, C. & van der Werf, M. J. Assessing tuberculosis case fatality ratio: a meta-analysis. PLoS ONE 6, e20755 (2011).","journal-title":"PLoS ONE"},{"key":"322_CR5","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1590\/S0037-86822001000400010","volume":"34","author":"MdeFMde de Albuquerque","year":"2001","unstructured":"de Albuquerque, M de F. M. de et al. Radiographic features of pulmonary tuberculosis in patients infected by HIV: is there an objective indicator of co-infection?. Rev. Soc. Bras. Med. Trop. 34, 369\u2013372 (2001).","journal-title":"Rev. Soc. Bras. Med. Trop."},{"key":"322_CR6","doi-asserted-by":"publisher","first-page":"518","DOI":"10.1016\/0962-8479(95)90527-8","volume":"76","author":"FA Post","year":"1995","unstructured":"Post, F. A., Wood, R. & Pillay, G. P. Pulmonary tuberculosis in HIV infection: radiographic appearance is related to CD4+ T-lymphocyte count. Tuber. Lung Dis. J. Int. Union Tuberc. Lung Dis. 76, 518\u2013521 (1995).","journal-title":"Tuber. Lung Dis. J. Int. Union Tuberc. Lung Dis."},{"key":"322_CR7","doi-asserted-by":"publisher","first-page":"1211","DOI":"10.1183\/09031936.00201711","volume":"40","author":"JG Peter","year":"2012","unstructured":"Peter, J. G. et al. Diagnostic accuracy of a urine lipoarabinomannan strip-test for TB detection in HIV-infected hospitalised patients. Eur. Respir. J. 40, 1211\u20131220 (2012).","journal-title":"Eur. Respir. J."},{"key":"322_CR8","first-page":"847","volume":"5","author":"NJ Hargreaves","year":"2001","unstructured":"Hargreaves, N. J. et al. \u2018Smear-negative\u2019 pulmonary tuberculosis in a DOTS programme: poor outcomes in an area of high HIV seroprevalence. Int. J. Tuberc. Lung Dis. J. Int. Union Tuberc. Lung Dis. 5, 847\u2013854 (2001).","journal-title":"Int. J. Tuberc. Lung Dis. J. Int. Union Tuberc. Lung Dis."},{"key":"322_CR9","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0167685","volume":"11","author":"S Modi","year":"2016","unstructured":"Modi, S. et al. Performance of clinical screening algorithms for tuberculosis intensified case finding among people living with HIV in western Kenya. PLoS ONE 11, e0167685 (2016).","journal-title":"PLoS ONE"},{"key":"322_CR10","unstructured":"WHO. TB\/HIV facts 2015. WHO. http:\/\/www.who.int\/hiv\/topics\/tb\/tbhiv_facts_2015\/en\/ (2019)."},{"key":"322_CR11","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1148\/radiol.2017162326","volume":"284","author":"P Lakhani","year":"2017","unstructured":"Lakhani, P. & Sundaram, B. Deep learning at chest radiography: automated classification of pulmonary tuberculosis by using convolutional neural networks. Radiology 284, 574\u2013582 (2017).","journal-title":"Radiology"},{"key":"322_CR12","doi-asserted-by":"publisher","unstructured":"Hwang, E. J. et al. Development and validation of a deep learning-based automatic detection algorithm for active pulmonary tuberculosis on chest radiographs. Clin. Infect. Dis. (2019). https:\/\/doi.org\/10.1093\/cid\/ciy967.","DOI":"10.1093\/cid\/ciy967"},{"key":"322_CR13","doi-asserted-by":"publisher","first-page":"1602159","DOI":"10.1183\/13993003.02159-2016","volume":"49","author":"M Rahman","year":"2017","unstructured":"Rahman, M. et al. An evaluation of automated chest radiography reading software for tuberculosis screening among public- and private-sector patients. Eur. Respir. J. 49, 1602159 (2017).","journal-title":"Eur. Respir. J."},{"key":"322_CR14","doi-asserted-by":"crossref","unstructured":"Philipsen, R. H. H. M. et al. Automated chest-radiography as a triage for Xpert testing in resource-constrained settings: a prospective study of diagnostic accuracy and costs. Sci. Rep. 5 (2015).","DOI":"10.1038\/srep12215"},{"key":"322_CR15","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1109\/TMI.2013.2284099","volume":"33","author":"S Jaeger","year":"2014","unstructured":"Jaeger, S. et al. Automatic tuberculosis screening using chest radiographs. IEEE Trans. Med. Imaging 33, 233\u2013245 (2014).","journal-title":"IEEE Trans. Med. Imaging"},{"key":"322_CR16","first-page":"89-99\u201399","volume":"3","author":"S Jaeger","year":"2013","unstructured":"Jaeger, S. et al. Automatic screening for tuberculosis in chest radiographs: a survey. Quant. Imaging Med. Surg. 3, 89-99\u201399 (2013).","journal-title":"Quant. Imaging Med. Surg."},{"key":"322_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-30810-1","volume":"8","author":"SMA Zaidi","year":"2018","unstructured":"Zaidi, S. M. A. et al. Evaluation of the diagnostic accuracy of computer-aided detection of tuberculosis on Chest radiography among private sector patients in Pakistan. Sci. Rep. 8, 12339 (2018).","journal-title":"Sci. Rep."},{"key":"322_CR18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0093757","volume":"9","author":"M Muyoyeta","year":"2014","unstructured":"Muyoyeta, M. et al. The sensitivity and specificity of using a computer aided diagnosis program for automatically scoring chest X-rays of presumptive TB patients compared with Xpert MTB\/RIF in Lusaka Zambia. PLoS ONE 9, e93757 (2014).","journal-title":"PLoS ONE"},{"key":"322_CR19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-018-37186-2","volume":"9","author":"ZZ Qin","year":"2019","unstructured":"Qin, Z. Z. et al. Using artificial intelligence to read chest radiographs for tuberculosis detection: a multi-site evaluation of the diagnostic accuracy of three deep learning systems. Sci. Rep. 9, 1\u201310 (2019).","journal-title":"Sci. Rep."},{"key":"322_CR20","doi-asserted-by":"publisher","DOI":"10.1038\/srep25265","volume":"6","author":"J Melendez","year":"2016","unstructured":"Melendez, J. et al. An automated tuberculosis screening strategy combining X-ray-based computer-aided detection and clinical information. Sci. Rep. 6, 25265 (2016).","journal-title":"Sci. Rep."},{"key":"322_CR21","unstructured":"Rajpurkar, P. et al. CheXpedition: investigating generalization challenges for translation of chest x-ray algorithms to the clinical setting. In: Workshop at ACM Conference on Health, Inference, and Learning (CHIL). (2020)."},{"key":"322_CR22","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.2019180084","volume":"1","author":"K Liu","year":"2019","unstructured":"Liu, K. et al. Evaluating a fully automated pulmonary nodule detection approach and its impact on radiologist performance. Radiol. Artif. Intell. 1, e180084 (2019).","journal-title":"Radiol. Artif. Intell."},{"key":"322_CR23","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1148\/radiol.2019182716","volume":"292","author":"A Yala","year":"2019","unstructured":"Yala, A., Lehman, C., Schuster, T., Portnoi, T. & Barzilay, R. A deep learning mammography-based model for improved breast cancer risk prediction. Radiology 292, 60\u201366 (2019).","journal-title":"Radiology"},{"key":"322_CR24","doi-asserted-by":"publisher","first-page":"e195600","DOI":"10.1001\/jamanetworkopen.2019.5600","volume":"2","author":"A Park","year":"2019","unstructured":"Park, A. et al. Deep learning\u2013assisted diagnosis of cerebral aneurysms using the HeadXNet model. JAMA Netw. Open 2, e195600\u2013e195600 (2019).","journal-title":"JAMA Netw. Open"},{"key":"322_CR25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1002699","volume":"15","author":"N Bien","year":"2018","unstructured":"Bien, N. et al. Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet. PLOS Med. 15, e1002699 (2018).","journal-title":"PLOS Med."},{"key":"322_CR26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-020-0232-8","volume":"3","author":"A Kiani","year":"2020","unstructured":"Kiani, A. et al. Impact of a deep learning assistant on the histopathologic classification of liver cancer. Npj Digit. Med. 3, 1\u20138 (2020).","journal-title":"Npj Digit. Med."},{"key":"322_CR27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-018-0076-7","volume":"2","author":"BN Patel","year":"2019","unstructured":"Patel, B. N. et al. Human\u2013machine partnership with artificial intelligence for chest radiograph diagnosis. Npj Digit. Med. 2, 1\u201310 (2019).","journal-title":"Npj Digit. Med."},{"key":"322_CR28","unstructured":"Consolidated guidelines on the use of antiretroviral drugs for treating and preventing hiv infection: recommendations for a public health approach. 2nd edn. (World Health Organization, 2016)."},{"key":"322_CR29","doi-asserted-by":"publisher","first-page":"e0171384","DOI":"10.1371\/journal.pone.0171384","volume":"12","author":"K Clouse","year":"2017","unstructured":"Clouse, K. et al. Low implementation of Xpert MTB\/RIF among HIV\/TB co-infected adults in the International epidemiologic databases to evaluate AIDS (IeDEA) program. PLoS ONE 12, e0171384 (2017).","journal-title":"PLoS ONE"},{"key":"322_CR30","doi-asserted-by":"publisher","first-page":"35","DOI":"10.12688\/gatesopenres.12842.1","volume":"2","author":"D Cazabon","year":"2018","unstructured":"Cazabon, D. et al. Market penetration of Xpert MTB\/RIF in high tuberculosis burden countries: A trend analysis from 2014\u20142016. Gates Open Res 2, 35 (2018).","journal-title":"Gates Open Res"},{"key":"322_CR31","unstructured":"Dawson, R. et al. Chest radiograph reading and recording system: evaluation for tuberculosis screening in patients with advanced HIV. https:\/\/www.ingentaconnect.com\/content\/iuatld\/ijtld\/2010\/00000014\/00000001\/art00009 (2010)."},{"key":"322_CR32","doi-asserted-by":"publisher","first-page":"221","DOI":"10.4103\/1947-2714.95904","volume":"4","author":"M Padyana","year":"2012","unstructured":"Padyana, M., Bhat, R. V., Dinesha, M. & Nawaz, A. HIV-tuberculosis: a study of chest x-ray patterns in relation to CD4 count. North Am. J. Med. Sci. 4, 221\u2013225 (2012).","journal-title":"North Am. J. Med. Sci."},{"key":"322_CR33","doi-asserted-by":"publisher","first-page":"263","DOI":"10.4103\/0970-2113.85687","volume":"28","author":"A Jaryal","year":"2011","unstructured":"Jaryal, A., Raina, R., Sarkar, M. & Sharma, A. Manifestations of tuberculosis in HIV\/AIDS patients and its relationship with CD4 count. Lung India. Organ Indian Chest Soc. 28, 263\u2013266 (2011).","journal-title":"Lung India. Organ Indian Chest Soc."},{"key":"322_CR34","doi-asserted-by":"publisher","first-page":"2540","DOI":"10.1128\/JCM.02319-10","volume":"49","author":"V Vadwai","year":"2011","unstructured":"Vadwai, V. et al. Xpert MTB\/RIF: a new pillar in diagnosis of extrapulmonary tuberculosis? J. Clin. Microbiol. 49, 2540\u20132545 (2011).","journal-title":"J. Clin. Microbiol."},{"key":"322_CR35","doi-asserted-by":"publisher","DOI":"10.1002\/14651858.CD009593.pub3","author":"KR Steingart","year":"2014","unstructured":"Steingart, K. R. et al. Xpert\u00ae MTB\/RIF assay for pulmonary tuberculosis and rifampicin resistance in adults. Cochrane Database Syst. Rev. https:\/\/doi.org\/10.1002\/14651858.CD009593.pub3 (2014).","journal-title":"Cochrane Database Syst. Rev."},{"key":"322_CR36","unstructured":"WHO. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. WHO. http:\/\/www.who.int\/hiv\/pub\/arv\/arv-2016\/en\/ (2019)."},{"key":"322_CR37","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1093\/cid\/cix988","volume":"66","author":"R Griesel","year":"2018","unstructured":"Griesel, R. et al. Optimizing tuberculosis diagnosis in human immunodeficiency virus\u2013infected inpatients meeting the criteria of seriously Ill in the World Health Organization algorithm. Clin. Infect. Dis. 66, 1419\u20131426 (2018).","journal-title":"Clin. Infect. Dis."},{"key":"322_CR38","doi-asserted-by":"publisher","first-page":"e10","DOI":"10.1097\/QAI.0000000000002002","volume":"81","author":"DJ Van Hoving","year":"2019","unstructured":"Van Hoving, D. J. et al. Brief report: real-world performance and interobserver agreement of urine lipoarabinomannan in diagnosing HIV-Associated tuberculosis in an emergency center. J. Acquir. Immune Defic. Syndr. 1999 81, e10\u2013e14 (2019).","journal-title":"J. Acquir. Immune Defic. Syndr. 1999"},{"key":"322_CR39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v067.i01","volume":"67","author":"D Bates","year":"2015","unstructured":"Bates, D., M\u00e4chler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Software 67, 1\u201348 (2015).","journal-title":"J. Stat. Software"},{"key":"322_CR40","unstructured":"(6) R: A Language and Environment for Statistical Computing. ResearchGate. https:\/\/www.researchgate.net\/publication\/312455832_R_A_Language_and_Environment_for_Statistical_Computing (2019)."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00322-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00322-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00322-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:32:50Z","timestamp":1670380370000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-00322-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,9]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["322"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-00322-2","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,9,9]]},"assertion":[{"value":"8 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"115"}}