{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T09:11:33Z","timestamp":1792746693389,"version":"4.5.0"},"reference-count":57,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,2,18]],"date-time":"2022-02-18T00:00:00Z","timestamp":1645142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>The manual observation of sputum smears by fluorescence microscopy for the diagnosis and treatment monitoring of patients with tuberculosis (TB) is a laborious and subjective task. In this work, we introduce an automatic pipeline which employs a novel deep learning-based approach to rapidly detect Mycobacterium tuberculosis (Mtb) organisms in sputum samples and thus quantify the burden of the disease. Fluorescence microscopy images are used as input in a series of networks, which ultimately produces a final count of present bacteria more quickly and consistently than manual analysis by healthcare workers. The pipeline consists of four stages: annotation by cycle-consistent generative adversarial networks (GANs), extraction of salient image patches, classification of the extracted patches, and finally, regression to yield the final bacteria count. We empirically evaluate the individual stages of the pipeline as well as perform a unified evaluation on previously unseen data that were given ground-truth labels by an experienced microscopist. We show that with no human intervention, the pipeline can provide the bacterial count for a sample of images with an error of less than 5%.<\/jats:p>","DOI":"10.3390\/info13020096","type":"journal-article","created":{"date-parts":[[2022,2,21]],"date-time":"2022-02-21T08:18:05Z","timestamp":1645431485000},"page":"96","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Tuberculosis Bacteria Detection and Counting in Fluorescence Microscopy Images Using a Multi-Stage Deep Learning Pipeline"],"prefix":"10.3390","volume":"13","author":[{"given":"Marios","family":"Zachariou","sequence":"first","affiliation":[{"name":"School of Computer Science, University of St Andrews, St Andrews KY16 9AJ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9314-194X","authenticated-orcid":false,"given":"Ognjen","family":"Arandjelovi\u0107","sequence":"additional","affiliation":[{"name":"School of Computer Science, University of St Andrews, St Andrews KY16 9AJ, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wilber","family":"Sabiiti","sequence":"additional","affiliation":[{"name":"School of Medicine, University of St Andrews, St Andrews KY16 9TF, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bariki","family":"Mtafya","sequence":"additional","affiliation":[{"name":"Mbeya Medical Research Center, Mbeya 2410, Tanzania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derek","family":"Sloan","sequence":"additional","affiliation":[{"name":"School of Medicine, University of St Andrews, St Andrews KY16 9TF, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.thorsurg.2018.09.010","article-title":"The global fight against tuberculosis","volume":"29","author":"Daley","year":"2019","journal-title":"Thorac. Surg. Clin."},{"key":"ref_2","unstructured":"World Health Organization (2018). Global Tuberculosis Report, World Health Organization. Technical Report."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/S1473-3099(18)30073-2","article-title":"Genetic sequencing for surveillance of drug resistance in tuberculosis in highly endemic countries: A multi-country population-based surveillance study","volume":"18","author":"Zignol","year":"2018","journal-title":"Lancet Infect. Dis."},{"key":"ref_4","first-page":"1","article-title":"Pastoralism and delay in diagnosis of TB in Ethiopia","volume":"9","author":"Gele","year":"2009","journal-title":"Lancet Infect. Dis."},{"key":"ref_5","first-page":"123","article-title":"Diagnosis of TB: State of the art","volume":"58","author":"Peter","year":"2012","journal-title":"Eur. Respir. Monogr."},{"key":"ref_6","first-page":"12","article-title":"Evaluation of Laboratory Professionals on AFB Smear Reading at Hawassa District Health Institutions, Southern Ethiopia","volume":"4","author":"Hailemariam","year":"2018","journal-title":"Int. J. Res. Stud. Microbiol. Biotechnol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2541","DOI":"10.4049\/jimmunol.1800993","article-title":"The end of the binary era: Revisiting the spectrum of tuberculosis","volume":"201","author":"Lin","year":"2018","journal-title":"J. Immunol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1111\/j.1574-695X.2012.00987.x","article-title":"Diagnosis of extrapulmonary tuberculosis by PCR","volume":"66","author":"Mehta","year":"2012","journal-title":"FEMS Immunol. Med. Microbiol."},{"key":"ref_9","unstructured":"Toman, K. (2004). Toman\u2019s Tuberculosis: Case Detection, Treatment and Monitoring. Questions and Answers, World Health Organization."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1126\/science.1216166","article-title":"Asymmetry and Aging of Mycobacterial Cells Lead to Variable Growth and Antibiotic Susceptibility","volume":"335","author":"Aldridge","year":"2012","journal-title":"Science"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.tube.2016.05.006","article-title":"Defining dormancy in mycobacterial disease","volume":"99","author":"Lipworth","year":"2016","journal-title":"Tuberculosis"},{"key":"ref_12","first-page":"57","article-title":"Using Machine Learning for Automatic Estimation of M. Smegmatis Cell Count from Fluorescence Microscopy Images","volume":"843","author":"Vente","year":"2019","journal-title":"Int. Workshop Health Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1093\/cid\/civ195","article-title":"Pharmacodynamic modeling of bacillary elimination rates and detection of bacterial lipid bodies in sputum to predict and understand outcomes in treatment of pulmonary tuberculosis","volume":"61","author":"Sloan","year":"2015","journal-title":"Clin. Infect. Dis."},{"key":"ref_14","first-page":"13","article-title":"Autofocus functions for tuberculosis diagnosis with conventional sputum smear microscopy","volume":"1","author":"Costa","year":"2012","journal-title":"Curr. Microsc. Contrib. Adv. Sci. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"570","DOI":"10.1016\/S1473-3099(06)70578-3","article-title":"Fluorescence versus conventional sputum smear microscopy for tuberculosis: A systematic review","volume":"9","author":"Steingart","year":"2006","journal-title":"Lancet Infect. Dis."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3934","DOI":"10.3892\/etm.2016.3859","article-title":"Staining with two observational methods for the diagnosis of tuberculous meningitis","volume":"12","author":"Zou","year":"2016","journal-title":"Exp. Ther. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1128\/JCM.00348-09","article-title":"High sensitivity and specificity of acid-fast microscopy for diagnosis of pulmonary tuberculosis in an African population with a high prevalence of human immunodeficiency virus","volume":"47","author":"Shea","year":"2009","journal-title":"J. Clin. Microbiol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J. Big Data"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"12878","DOI":"10.1038\/s41598-018-31284-x","article-title":"A deep convolutional neural network approach for astrocyte detection","volume":"8","author":"Suleymanova","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1016\/j.bbe.2018.05.007","article-title":"Automatic detection of tuberculosis bacilli from microscopic sputum smear images using deep learning methods","volume":"38","author":"Panicker","year":"2018","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Sotaquira, M., Rueda, L., and Narvaez, R. (2009, January 7\u20139). Detection and quantification of bacilli and clusters present in sputum smear samples: A novel algorithm for pulmonary tuberculosis diagnosis. Proceedings of the International Conference on Digital Image Processing, Bangkok, Thailand.","DOI":"10.1109\/ICDIP.2009.59"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12046-018-0878-y","article-title":"FHDT: Fuzzy and Hyco-entropy-based decision tree classifier for tuberculosis diagnosis from sputum images","volume":"43","author":"Mithra","year":"2018","journal-title":"S\u0101dhan\u0101"},{"key":"ref_23","unstructured":"Daniel Chaves Viquez, K., Arandjelovic, O., Blaikie, A., and Ae Hwang, I. (2017, January 22\u201329). Synthesising wider field images from narrow-field retinal video acquired using a low-cost direct ophthalmoscope (Arclight) attached to a smartphone. Proceedings of the IEEE International Conference on Computer Vision Workshops, Venice, Italy."},{"key":"ref_24","unstructured":"Rudzki, M. (2009, January 17\u201320). Vessel detection method based on eigenvalues of the hessian matrix and its applicability to airway tree segmentation. Proceedings of the 11th International PhD Workshop OWD, Wisla, Poland."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"921","DOI":"10.1587\/transinf.E94.D.921","article-title":"Improving Hessian matrix detector for SURF","volume":"94","author":"Chi","year":"2011","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_26","first-page":"920401","article-title":"Automation of hessian-based tubularity measure response function in 3D biomedical images","volume":"2011","author":"Dzyubak","year":"2011","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"203","DOI":"10.35841\/biomedicalresearch.30-19-068","article-title":"Optimized maximum principal curvatures based segmentation of blood vessels from retinal images","volume":"30","author":"Kumar","year":"2019","journal-title":"Biomed. Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Ghiass, R.S., Arandjelovic, O., Bendada, H., and Maldague, X. (2013, January 14\u201318). Vesselness features and the inverse compositional AAM for robust face recognition using thermal IR. Proceedings of the Twenty-Seventh AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v27i1.8628"},{"key":"ref_29","first-page":"2672","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_30","unstructured":"Wang, S. (2017). Generative Adversarial Networks (GAN): A Gentle Introduction. Tutorial on GAN in LIN395C: Research in Computational Linguistics, University of Texas at Austin."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Magister, L.C., and Arandjelovi\u0107, O. (2021, January 1\u20135). Generative Image Inpainting for Retinal Images using Generative Adversarial Networks. Proceedings of the 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), Mexico City, Mexico.","DOI":"10.1109\/EMBC46164.2021.9630619"},{"key":"ref_32","unstructured":"Hjelm, R.D., Jacob, A.P., Che, T., Trischler, A., Cho, K., and Bengio, Y. (2017). Boundary-seeking generative adversarial networks. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2662","DOI":"10.1016\/j.patcog.2014.02.006","article-title":"Hallucinating optimal high-dimensional subspaces","volume":"47","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ledig, C., Theis, L., Husz\u00e1r, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., and Wang, Z. (2017, January 21\u201326). Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.19"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, C., and Wand, M. (2016, January 8\u201316). Precomputed real-time texture synthesis with markovian generative adversarial networks. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_43"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zachariou, M., Dimitriou, N., and Arandjelovi\u0107, O. (2020). Visual reconstruction of ancient coins using cycle-consistent generative adversarial networks. Science, 2.","DOI":"10.3390\/sci2030052"},{"key":"ref_39","unstructured":"Gadermayr, M., Gupta, L., Klinkhammer, B.M., Boor, P., and Merhof, D. (2018). Unsupervisedly training GANs for segmenting digital pathology with automatically generated annotations. arXiv."},{"key":"ref_40","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1214\/aoms\/1177729586","article-title":"A stochastic approximation method","volume":"22","author":"Robbins","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_42","first-page":"18795","article-title":"Adabelief optimizer: Adapting stepsizes by the belief in observed gradients","volume":"33","author":"Zhuang","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_43","unstructured":"Yue, X., Dimitriou, N., and Arandjelovic, O. (2019). Colorectal cancer outcome prediction from H&E whole slide images using machine learning and automatically inferred phenotype profiles. arXiv."},{"key":"ref_44","first-page":"112","article-title":"Algorithms for the reduction of the number of points required to represent a digitized line or its caricature","volume":"10","author":"Douglas","year":"1973","journal-title":"Cartogr. Int. J. Geogr. Inf. Geovisualization"},{"key":"ref_45","first-page":"820","article-title":"Dual learning for machine translation","volume":"29","author":"He","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_47","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., and Keutzer, K. (2016). SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Smith, L.N. (2017, January 24\u201331). Cyclical learning rates for training neural networks. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.58"},{"key":"ref_49","unstructured":"Cooper, J., Um, I.H., Arandjelovi\u0107, O., and Harrison, D.J. (2021). Hoechst Is All You Need: Lymphocyte Classification with Deep Learning. arXiv."},{"key":"ref_50","first-page":"4","article-title":"A Tutorial on Speech Understanding Systems","volume":"29","author":"Newell","year":"1975","journal-title":"Speech Recognit."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Beykikhoshk, A., Arandjelovi\u0107, O., Phung, D., and Venkatesh, S. (2015, January 25\u201328). Overcoming data scarcity of Twitter: Using tweets as bootstrap with application to autism-related topic content analysis. Proceedings of the 2015 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining, Paris, France.","DOI":"10.1145\/2808797.2808908"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1080\/07038992.2017.1259557","article-title":"Application of the dice coefficient to accuracy assessment of object-based image classification","volume":"43","author":"Guindon","year":"2017","journal-title":"Can. J. Remote. Sens."},{"key":"ref_53","first-page":"277","article-title":"Automated identification of tubercle bacilli in sputum: A preliminary investigation","volume":"21","author":"Veropoulos","year":"1999","journal-title":"Can. J. Remote Sens."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1016\/j.rti.2004.05.007","article-title":"Identification of tuberculosis bacteria based on shape and color","volume":"10","author":"Forero","year":"2004","journal-title":"Real-Time Imaging"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Kant, S., and Srivastava, M.M. (2018, January 18\u201321). Towards automated tuberculosis detection using deep learning. Proceedings of the IEEE Symposium Series on Computational Intelligence, Bangalore, India.","DOI":"10.1109\/SSCI.2018.8628800"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Zhai, Y., Liu, Y., Zhou, D., and Liu, S. (2010, January 14\u201318). Automatic identification of mycobacterium tuberculosis from ZN-stained sputum smear: Algorithm and system design. Proceedings of the IEEE International Conference on Robotics and Biomimetics, Tianjin, China.","DOI":"10.1109\/ROBIO.2010.5723300"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Ghosh, P., Bhattacharjee, D., and Nasipuri, M. (2016, January 3\u20135). A hybrid approach to diagnosis of tuberculosis from sputum. Proceedings of the International Conference on Electrical, Electronics, and Optimization Techniques, Chennai, India.","DOI":"10.1109\/ICEEOT.2016.7754790"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/96\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:22:02Z","timestamp":1760134922000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/96"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,18]]},"references-count":57,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020096"],"URL":"https:\/\/doi.org\/10.3390\/info13020096","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,18]]}}}