{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T10:30:52Z","timestamp":1776940252696,"version":"3.51.4"},"update-to":[{"DOI":"10.1371\/journal.pntd.0010943","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,12,19]],"date-time":"2022-12-19T00:00:00Z","timestamp":1671408000000}}],"reference-count":47,"publisher":"Public Library of Science (PLoS)","issue":"12","license":[{"start":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T00:00:00Z","timestamp":1670371200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NIH","award":["P20GM125498"],"award-info":[{"award-number":["P20GM125498"]}]}],"content-domain":{"domain":["www.plosntds.org"],"crossmark-restriction":false},"short-container-title":["PLoS Negl Trop Dis"],"abstract":"<jats:sec id=\"sec001\"><jats:title>Background<\/jats:title><jats:p>Though significant progress in disease elimination has been made over the past decades, trachoma is the leading infectious cause of blindness globally. Further efforts in trachoma elimination are paradoxically being limited by the relative rarity of the disease, which makes clinical training for monitoring surveys difficult. In this work, we evaluate the plausibility of an Artificial Intelligence model to augment or replace human image graders in the evaluation\/diagnosis of trachomatous inflammation\u2014follicular (TF).<\/jats:p><\/jats:sec><jats:sec id=\"sec002\"><jats:title>Methods<\/jats:title><jats:p>We utilized a dataset consisting of 2300 images with a 5% positivity rate for TF. We developed classifiers by implementing two state-of-the-art Convolutional Neural Network architectures, ResNet101 and VGG16, and applying a suite of data augmentation\/oversampling techniques to the positive images. We then augmented our data set with additional images from independent research groups and evaluated performance.<\/jats:p><\/jats:sec><jats:sec id=\"sec003\"><jats:title>Results<\/jats:title><jats:p>Models performed well in minimizing the number of false negatives, given the constraint of the low numbers of images in which TF was present. The best performing models achieved a sensitivity of 95% and positive predictive value of 50\u201370% while reducing the number images requiring skilled grading by 66\u201375%. Basic oversampling and data augmentation techniques were most successful at improving model performance, while techniques that are grounded in clinical experience, such as highlighting follicles, were less successful.<\/jats:p><\/jats:sec><jats:sec id=\"sec004\"><jats:title>Discussion<\/jats:title><jats:p>The developed models perform well and significantly reduce the burden on graders by minimizing the number of false negative identifications. Further improvements in model skill will benefit from data sets with more TF as well as a range in image quality and image capture techniques used. While these models approach\/meet the community-accepted standard for skilled field graders (i.e., Cohen\u2019s Kappa &gt;0.7), they are insufficient to be deployed independently\/clinically at this time; rather, they can be utilized to significantly reduce the burden on skilled image graders.<\/jats:p><\/jats:sec>","DOI":"10.1371\/journal.pntd.0010943","type":"journal-article","created":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T18:45:53Z","timestamp":1670438753000},"page":"e0010943","update-policy":"https:\/\/doi.org\/10.1371\/journal.pntd.corrections_policy","source":"Crossref","is-referenced-by-count":10,"title":["Detection of trachoma using machine learning approaches"],"prefix":"10.1371","volume":"16","author":[{"given":"Damien","family":"Socia","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher J.","family":"Brady","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheila K.","family":"West","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3224-7617","authenticated-orcid":true,"given":"R. Chase","family":"Cockrell","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2022,12,7]]},"reference":[{"issue":"6","key":"pntd.0010943.ref001","doi-asserted-by":"crossref","first-page":"e339","DOI":"10.1016\/S2214-109X(13)70113-X","article-title":"Causes of vision loss worldwide, 1990\u20132010: a systematic analysis","volume":"1","author":"RR Bourne","year":"2013","journal-title":"Lancet Glob Health"},{"issue":"1","key":"pntd.0010943.ref002","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.ophtha.2019.11.001","article-title":"Trachoma: Time to Talk Eradication.","volume":"127","author":"TM Lietman","year":"2020","journal-title":"Ophthalmology"},{"key":"pntd.0010943.ref003","first-page":"1","article-title":"Impact and Trends in Global Ophthalmology.","volume":"22","author":"LB Williams","year":"2020","journal-title":"Curr Ophthalmol Rep"},{"issue":"31","key":"pntd.0010943.ref004","first-page":"353","article-title":"WHO Alliance for the Global Elimination of Trachoma: progress report on elimination of trachoma, 2021\u2013Alliance de l\u2019OMS pour l\u2019\u00e9limination mondiale du trachome: rapport de situation sur l\u2019\u00e9limination du trachome, 2021.","volume":"97","author":"World Health Organization","year":"2022","journal-title":"Weekly Epidemiological Record = Relev\u00e9 \u00e9pid\u00e9miologique hebdomadaire."},{"key":"pntd.0010943.ref005","unstructured":"W. H. O. Alliance for the Global Elimination of Trachoma. Meeting, W. H. O. Programme for the Prevention of Blindness and Deafness. Report of the third meeting of the WHO Alliance for the Global Elimination of Trachoma, Ouarzazate, Morocco, 19\u201320 October 1998 Geneva: World Health Organization; 1999 [2\/14\/2021]. Available from: https:\/\/apps.who.int\/iris\/handle\/10665\/65933."},{"issue":"11","key":"pntd.0010943.ref006","doi-asserted-by":"crossref","first-page":"e0009928","DOI":"10.1371\/journal.pntd.0009928","article-title":"Evaluation of photography using head-mounted display technology (ICAPS) for district Trachoma surveys.","volume":"15","author":"F Naufal","year":"2021","journal-title":"PLoS Negl Trop Dis"},{"key":"pntd.0010943.ref007","unstructured":"World Health Organization. Reaching the Last Mile Forum: Keynote Address 2019 [2\/14\/2021]. Available from: https:\/\/www.who.int\/director-general\/speeches\/detail\/reaching-the-last-mile-forum."},{"key":"pntd.0010943.ref008","volume-title":"Report of the 23rd Meeting of the WHO Alliance for the Global Elimination of Trachoma by 2020, Virtual meeting, 30 November\u20131 December 2020 [in press].","author":"World Health Organization","year":"2021"},{"key":"pntd.0010943.ref009","volume-title":"Network of WHO collaborating centres for trachoma: second meeting report","author":"World Health Organization","year":"2017"},{"issue":"5\u20136","key":"pntd.0010943.ref010","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1080\/09286586.2018.1500616","article-title":"A diagnostic instrument to help field graders evaluate active trachoma.","volume":"25","author":"AW Solomon","year":"2018","journal-title":"Ophthalmic Epidemiol."},{"issue":"6","key":"pntd.0010943.ref011","doi-asserted-by":"crossref","first-page":"2488","DOI":"10.4269\/ajtmh.20-0386","article-title":"Comparison of Smartphone Photography, Single-Lens Reflex Photography, and Field-Grading for Trachoma","volume":"103","author":"JM Nesemann","year":"2020","journal-title":"The American journal of tropical medicine and hygiene"},{"issue":"6","key":"pntd.0010943.ref012","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1093\/inthealth\/ihz035","article-title":"Smartphone photography as a possible method of post-validation trachoma surveillance in resource-limited settings.","volume":"11","author":"BM Snyder","year":"2019","journal-title":"Int Health"},{"issue":"1","key":"pntd.0010943.ref013","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1136\/bjo.74.1.12","article-title":"Reliability of photographs for grading trachoma in field studies","volume":"74","author":"SK West","year":"1990","journal-title":"Br J Ophthalmol"},{"key":"pntd.0010943.ref014","unstructured":"Center for Medicare & Medicaid Services. Search the Physician Fee Schedule. Available from: https:\/\/www.cms.gov\/medicare\/physician-fee-schedule\/search."},{"key":"pntd.0010943.ref015","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.2147\/OPTH.S357766","article-title":"The Cost-Effectiveness of a Telemedicine Screening Program for Diabetic Retinopathy in New York City.","volume":"16","author":"H Muqri","year":"2022","journal-title":"Clinical Ophthalmology (Auckland, NZ)."},{"issue":"12","key":"pntd.0010943.ref016","doi-asserted-by":"crossref","first-page":"2571","DOI":"10.1016\/j.ophtha.2016.08.021","article-title":"Cost-effectiveness of a national telemedicine diabetic retinopathy screening program in Singapore","volume":"123","author":"HV Nguyen","year":"2016","journal-title":"Ophthalmology"},{"issue":"1","key":"pntd.0010943.ref017","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12962-020-00211-1","article-title":"Cost-effectiveness of diabetic retinopathy screening programs using telemedicine: a systematic review.","volume":"18","author":"D Avidor","year":"2020","journal-title":"Cost Effectiveness and Resource Allocation"},{"issue":"25","key":"pntd.0010943.ref018","doi-asserted-by":"crossref","DOI":"10.1097\/MD.0000000000020306","article-title":"Cost-effectiveness and diagnostic accuracy of telemedicine in macular disease and diabetic retinopathy: A systematic review and meta-analysis.","volume":"99","author":"W Ullah","year":"2020","journal-title":"Medicine"},{"issue":"6","key":"pntd.0010943.ref019","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1097\/IJG.0000000000000660","article-title":"Crowdsourcing to Evaluate Fundus Photographs for the Presence of Glaucoma","volume":"26","author":"X Wang","year":"2017","journal-title":"Journal of glaucoma"},{"issue":"6","key":"pntd.0010943.ref020","doi-asserted-by":"crossref","first-page":"e222","DOI":"10.2196\/jmir.7984","article-title":"Improving Consensus Scoring of Crowdsourced Data Using the Rasch Model: Development and Refinement of a Diagnostic Instrument.","volume":"19","author":"CJ Brady","year":"2017","journal-title":"Journal of medical Internet research"},{"issue":"10","key":"pntd.0010943.ref021","doi-asserted-by":"crossref","first-page":"e233","DOI":"10.2196\/jmir.3807","article-title":"Rapid grading of fundus photographs for diabetic retinopathy using crowdsourcing","volume":"16","author":"CJ Brady","year":"2014","journal-title":"Journal of medical Internet research"},{"issue":"2","key":"pntd.0010943.ref022","doi-asserted-by":"crossref","first-page":"e0210463","DOI":"10.1371\/journal.pone.0210463","article-title":"Sensitivity and specificity of computer vision classification of eyelid photographs for programmatic trachoma assessment","volume":"14","author":"MC Kim","year":"2019","journal-title":"PLoS One"},{"issue":"8","key":"pntd.0010943.ref023","first-page":"1788","article-title":"Crowdsourcing Can Match Field Grading Validity for Follicular Trachoma","volume":"62","author":"CJ Brady","year":"2021","journal-title":"Invest Ophthalmol Vis Sci"},{"key":"pntd.0010943.ref024","article-title":"An introduction to convolutional neural networks.","author":"K O\u2019Shea","year":"2015","journal-title":"arXiv preprint arXiv:151108458."},{"key":"pntd.0010943.ref025","article-title":"A survey of convolutional neural networks: analysis, applications, and prospects","author":"Z Li","year":"2021","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"issue":"4","key":"pntd.0010943.ref026","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/s13244-018-0639-9","article-title":"Convolutional neural networks: an overview and application in radiology.","volume":"9","author":"R Yamashita","year":"2018","journal-title":"Insights into imaging."},{"key":"pntd.0010943.ref027","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1016\/j.jss.2021.10.017","article-title":"Optical Biopsy Using a Neural Network to Predict Gene Expression From Photos of Wounds.","volume":"270","author":"G Schumaker","year":"2022","journal-title":"Journal of Surgical Research"},{"key":"pntd.0010943.ref028","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016.","DOI":"10.1109\/CVPR.2016.90"},{"key":"pntd.0010943.ref029","article-title":"Very deep convolutional networks for large-scale image recognition.","author":"K Simonyan","year":"2014","journal-title":"arXiv preprint arXiv:14091556."},{"key":"pntd.0010943.ref030","article-title":"Adabatch: Adaptive batch sizes for training deep neural networks.","author":"A Devarakonda","year":"2017","journal-title":"arXiv preprint arXiv:171202029."},{"key":"pntd.0010943.ref031","doi-asserted-by":"crossref","unstructured":"Liu L, Qi H, editors. Learning effective binary descriptors via cross entropy. 2017 IEEE winter conference on applications of computer vision (WACV); 2017: IEEE.","DOI":"10.1109\/WACV.2017.144"},{"key":"pntd.0010943.ref032","doi-asserted-by":"crossref","DOI":"10.1109\/CAC.2017.8243510","volume-title":"Research on data augmentation for image classification based on convolution neural networks","author":"J Shijie","year":"2017"},{"key":"pntd.0010943.ref033","doi-asserted-by":"crossref","unstructured":"Quiroga F, Ronchetti F, Lanzarini L, Bariviera AF, editors. Revisiting data augmentation for rotational invariance in convolutional neural networks. International Conference on Modelling and Simulation in Management Sciences; 2018: Springer.","DOI":"10.1007\/978-3-030-15413-4_10"},{"key":"pntd.0010943.ref034","doi-asserted-by":"crossref","first-page":"4935","DOI":"10.1109\/ACCESS.2019.2962572","article-title":"Perspective transformation data augmentation for object detection.","volume":"8","author":"K Wang","year":"2019","journal-title":"IEEE Access"},{"key":"pntd.0010943.ref035","doi-asserted-by":"crossref","unstructured":"Miko\u0142ajczyk A, Grochowski M, editors. Data augmentation for improving deep learning in image classification problem. 2018 international interdisciplinary PhD workshop (IIPhDW); 2018: IEEE.","DOI":"10.1109\/IIPHDW.2018.8388338"},{"key":"pntd.0010943.ref036","unstructured":"Lasseck M, editor Acoustic bird detection with deep convolutional neural networks. Proceedings of the Detection and Classification of Acoustic Scenes and Events 2018 Workshop (DCASE2018); 2018."},{"key":"pntd.0010943.ref037","doi-asserted-by":"crossref","unstructured":"Qiao Y, Su D, Kong H, Sukkarieh S, Lomax S, Clark C, editors. Data augmentation for deep learning based cattle segmentation in precision livestock farming. 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE); 2020: IEEE.","DOI":"10.1109\/CASE48305.2020.9216758"},{"issue":"2","key":"pntd.0010943.ref038","doi-asserted-by":"crossref","first-page":"237","DOI":"10.3390\/diagnostics12020237","article-title":"Applications of Explainable Artificial Intelligence in Diagnosis and Surgery.","volume":"12","author":"Y Zhang","year":"2022","journal-title":"Diagnostics"},{"key":"pntd.0010943.ref039","article-title":"Hierarchical Text-Conditional Image Generation with CLIP Latents.","author":"A Ramesh","year":"2022","journal-title":"arXiv preprint arXiv:220406125."},{"issue":"1","key":"pntd.0010943.ref040","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-022-00592-y","article-title":"Machine learning for medical imaging: methodological failures and recommendations for the future.","volume":"5","author":"G Varoquaux","year":"2022","journal-title":"npj Digital Medicine."},{"key":"pntd.0010943.ref041","doi-asserted-by":"crossref","unstructured":"S\u00fcsstrunk S, Buckley R, Swen S, editors. Standard RGB color spaces. Color and imaging conference; 1999: Society for Imaging Science and Technology.","DOI":"10.2352\/CIC.1999.7.1.art00024"},{"key":"pntd.0010943.ref042","doi-asserted-by":"crossref","unstructured":"Shuhua L, Gaizhi G, editors. The application of improved HSV color space model in image processing. 2010 2nd International Conference on Future Computer and Communication; 2010: IEEE.","DOI":"10.1109\/ICFCC.2010.5497299"},{"key":"pntd.0010943.ref043","doi-asserted-by":"crossref","first-page":"474","DOI":"10.1016\/B978-0-12-336156-1.50061-6","article-title":"Contrast limited adaptive histogram equalization","author":"K. Zuiderveld","year":"1994","journal-title":"Graphics gems."},{"issue":"3","key":"pntd.0010943.ref044","doi-asserted-by":"crossref","first-page":"214","DOI":"10.3109\/09286586.2015.1037401","article-title":"The Global Trachoma Mapping Project: methodology of a 34-country population-based study.","volume":"22","author":"AW Solomon","year":"2015","journal-title":"Ophthalmic Epidemiol."},{"key":"pntd.0010943.ref045","volume-title":"Tropical data: training system for trachoma prevalence surveys","author":"P Courtright","year":"2016"},{"issue":"7","key":"pntd.0010943.ref046","doi-asserted-by":"crossref","first-page":"e23863","DOI":"10.2196\/23863","article-title":"Performance and limitation of machine learning algorithms for diabetic retinopathy screening: meta-analysis","volume":"23","author":"J-H Wu","year":"2021","journal-title":"J Med Internet Res"},{"key":"pntd.0010943.ref047","article-title":"A Novel Hands-Free Augmented-Reality System to Document the Prevalence of Active Trachoma.","author":"F Naufal","year":"2021","journal-title":"CUG?H"}],"updated-by":[{"DOI":"10.1371\/journal.pntd.0010943","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,12,19]],"date-time":"2022-12-19T00:00:00Z","timestamp":1671408000000}}],"container-title":["PLOS Neglected Tropical Diseases"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pntd.0010943","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,2]],"date-time":"2023-12-02T16:48:11Z","timestamp":1701535691000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pntd.0010943"}},"subtitle":[],"editor":[{"given":"Joseph M.","family":"Vinetz","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2022,12,7]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12,7]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pntd.0010943","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pntd.0010943","asserted-by":"object"}]},"ISSN":["1935-2735"],"issn-type":[{"value":"1935-2735","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,7]]}}}