{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T19:17:56Z","timestamp":1774379876849,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2023,2,3]],"date-time":"2023-02-03T00:00:00Z","timestamp":1675382400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Research to Prevent Blindness"},{"name":"Payden Glaucoma Research Fund"},{"name":"Simms\/Mann Family Foundation"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>This study describes the development of a convolutional neural network (CNN) for automated assessment of optic disc photograph quality. Using a code-free deep learning platform, a total of 2377 optic disc photographs were used to develop a deep CNN capable of determining optic disc photograph quality. Of these, 1002 were good-quality images, 609 were acceptable-quality, and 766 were poor-quality images. The dataset was split 80\/10\/10 into training, validation, and test sets and balanced for quality. A ternary classification model (good, acceptable, and poor quality) and a binary model (usable, unusable) were developed. In the ternary classification system, the model had an overall accuracy of 91% and an AUC of 0.98. The model had higher predictive accuracy for images of good (93%) and poor quality (96%) than for images of acceptable quality (91%). The binary model performed with an overall accuracy of 98% and an AUC of 0.99. When validated on 292 images not included in the original training\/validation\/test dataset, the model\u2019s accuracy was 85% on the three-class classification task and 97% on the binary classification task. The proposed system for automated image-quality assessment for optic disc photographs achieves high accuracy in both ternary and binary classification systems, and highlights the success achievable with a code-free platform. There is wide clinical and research potential for such a model, with potential applications ranging from integration into fundus camera software to provide immediate feedback to ophthalmic photographers, to prescreening large databases before their use in research.<\/jats:p>","DOI":"10.3390\/jcm12031217","type":"journal-article","created":{"date-parts":[[2023,2,3]],"date-time":"2023-02-03T02:05:28Z","timestamp":1675389928000},"page":"1217","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["A Neural Network for Automated Image Quality Assessment of Optic Disc Photographs"],"prefix":"10.3390","volume":"12","author":[{"given":"Ella","family":"Bouris","sequence":"first","affiliation":[{"name":"Department of Ophthalmology, Jules Stein Eye Institute, University of California-Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tyler","family":"Davis","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of California-Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Esteban","family":"Morales","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Jules Stein Eye Institute, University of California-Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lourdes","family":"Grassi","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Jules Stein Eye Institute, University of California-Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Diana","family":"Salazar Vega","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Vision Consultants and Surgeons, Falls Church, VA 22046, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph","family":"Caprioli","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Jules Stein Eye Institute, University of California-Los Angeles, Los Angeles, CA 90095, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/S0002-9394(02)01930-X","article-title":"Structure and Function Evaluation (SAFE): II. Comparison of Optic Disk and Visual Field Characteristics","volume":"135","author":"Johnson","year":"2003","journal-title":"Am. J. Ophthalmol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"485","DOI":"10.1016\/S0002-9394(14)72385-2","article-title":"Initial Glaucomatous Optic Disk and Retinal Nerve Fiber Layer Abnormalities and Their Progression","volume":"111","author":"Tuulonen","year":"1991","journal-title":"Am. J. Ophthalmol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1016\/j.ajo.2005.01.003","article-title":"Use of Progressive Glaucomatous Optic Disk Change as the Reference Standard for Evaluation of Diagnostic Tests in Glaucoma","volume":"139","author":"Medeiros","year":"2005","journal-title":"Am. J. Ophthalmol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1167\/iovs.03-1087","article-title":"Confocal Scanning Laser Ophthalmoscopy Classifiers and Stereophotograph Evaluation for Prediction of Visual Field Abnormalities in Glaucoma-Suspect Eyes","volume":"45","author":"Bowd","year":"2004","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1016\/S0002-9394(99)00445-6","article-title":"Relationship between Structural Abnormalities and Short-Wavelength Perimetric Defects in Eyes at Risk of Glaucoma","volume":"129","author":"Ugurlu","year":"2000","journal-title":"Am. J. Ophthalmol."},{"key":"ref_6","first-page":"532","article-title":"Variability of Expert Observers in Evaluating the Optic Disc","volume":"74","author":"Lichter","year":"1976","journal-title":"Trans. Am. Ophthalmol. Soc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"OCT80","DOI":"10.1167\/iovs.15-18931","article-title":"Technology and the Glaucoma Suspect","volume":"57","author":"Blumberg","year":"2016","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"350","DOI":"10.1016\/S0161-6420(88)33177-5","article-title":"Intraobserver and Interobserver Agreement in Measurement of Optic Disc Characteristics","volume":"95","author":"Tielsch","year":"1988","journal-title":"Ophthalmology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ajo.2008.07.023","article-title":"Agreement among Glaucoma Specialists in Assessing Progressive Disc Changes from Photographs in Open-Angle Glaucoma Patients","volume":"147","author":"Jampel","year":"2009","journal-title":"Am. J. Ophthalmol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1016\/S0008-4182(03)80109-6","article-title":"Effectiveness and Safety of Screening for Diabetic Retinopathy with Two NonmYdriatic Digital Images Compared with the Seven Standard Stereoscopic Photographic Fields","volume":"38","author":"Boucher","year":"2003","journal-title":"Can. J. Ophthalmol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1038\/sj.eye.6700409","article-title":"Is Screening with Digital Imaging Using One Retinal View Adequate?","volume":"17","author":"Herbert","year":"2003","journal-title":"Eye"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1046\/j.1464-5491.2003.00954.x","article-title":"The Effectiveness of Screening for Diabetic Retinopathy by Digital Imaging Photography and Technician Ophthalmoscopy","volume":"20","author":"Scanlon","year":"2003","journal-title":"Diabet. Med. J. Br. Diabet. Assoc."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1097\/APO.0000000000000404","article-title":"Deep Learning for Retinal Image Quality Assessment of Optic Nerve Head Disorders","volume":"10","author":"Chan","year":"2021","journal-title":"Asia-Pac. J. Ophthalmol."},{"key":"ref_14","unstructured":"(2023, January 23). Artificial Intelligence in Glaucoma: Current Opinion in Ophthalmology. Available online: https:\/\/journals.lww.com\/co-ophthalmology\/Fulltext\/2019\/03000\/Artificial_intelligence_in_glaucoma.5.aspx."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"(2023, January 23). Glaucoma Management in the Era of Artificial Intelligence. British Journal of Ophthalmology. Available online: https:\/\/bjo.bmj.com\/content\/104\/3\/301.long.","DOI":"10.1136\/bjophthalmol-2019-315016"},{"key":"ref_16","unstructured":"Campilho, A., and Kamel, M. Detection of Peri-Papillary Atrophy and RNFL Defect from Retinal Images. Proceedings of the Image Analysis and Recognition."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1016\/j.compmedimag.2014.05.005","article-title":"Thickness Related Textural Properties of Retinal Nerve Fiber Layer in Color Fundus Images","volume":"38","author":"Odstrcilik","year":"2014","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1109\/TMI.2013.2247770","article-title":"Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening","volume":"32","author":"Cheng","year":"2013","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Al-Bander, B., Williams, B.M., Al-Nuaimy, W., Al-Taee, M.A., Pratt, H., and Zheng, Y. (2018). Dense Fully Convolutional Segmentation of the Optic Disc and Cup in Colour Fundus for Glaucoma Diagnosis. Symmetry, 10.","DOI":"10.3390\/sym10040087"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.compmedimag.2016.07.012","article-title":"Glaucoma Detection Using Entropy Sampling and Ensemble Learning for Automatic Optic Cup and Disc Segmentation","volume":"55","author":"Zilly","year":"2017","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1001\/jamaophthalmol.2019.3501","article-title":"Development and Validation of a Deep Learning System to Detect Glaucomatous Optic Neuropathy Using Fundus Photographs","volume":"137","author":"Liu","year":"2019","journal-title":"JAMA Ophthalmol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"14665","DOI":"10.1038\/s41598-018-33013-w","article-title":"Development of a Deep Residual Learning Algorithm to Screen for Glaucoma from Fundus Photography","volume":"8","author":"Shibata","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1001\/jama.2017.18152","article-title":"Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes","volume":"318","author":"Ting","year":"2017","journal-title":"JAMA"},{"key":"ref_24","unstructured":"Trucco, E., MacGillivray, T., and Xu, Y. (2019). Computational Retinal Image Analysis, Academic Press."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Fu, H., Wang, B., Shen, J., Cui, S., Xu, Y., Liu, J., and Shao, L. (2019, January 13\u201317). Evaluation of Retinal Image Quality Assessment Networks in Different Color-Spaces. Proceedings of the 22nd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2019, Shenzhen, China.","DOI":"10.1007\/978-3-030-32239-7_6"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1005","DOI":"10.1364\/BOE.8.001005","article-title":"Fully Automated Diagnosis of Papilledema through Robust Extraction of Vascular Patterns and Ocular Pathology from Fundus Photographs","volume":"8","author":"Fatima","year":"2017","journal-title":"Biomed. Opt. Express"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1007\/s10278-018-0084-9","article-title":"Automated Quality Assessment of Colour Fundus Images for Diabetic Retinopathy Screening in Telemedicine","volume":"31","author":"Saha","year":"2018","journal-title":"J. Digit. Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.compbiomed.2018.10.004","article-title":"Retinal Image Quality Assessment Using Deep Learning","volume":"103","author":"Zago","year":"2018","journal-title":"Comput. Biol. Med."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.compbiomed.2019.03.019","article-title":"Quality and Content Analysis of Fundus Images Using Deep Learning","volume":"108","author":"Chalakkal","year":"2019","journal-title":"Comput. Biol. Med."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1016\/j.compmedimag.2015.10.003","article-title":"An Approach to Locate Optic Disc in Retinal Images with Pathological Changes","volume":"47","author":"Xiong","year":"2016","journal-title":"Comput. Med. Imaging Graph."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1016\/j.ophtha.2018.01.023","article-title":"Efficacy of a Deep Learning System for Detecting Glaucomatous Optic Neuropathy Based on Color Fundus Photographs","volume":"125","author":"Li","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Campolo, D. (2010). New Developments in Biomedical Engineering, InTech.","DOI":"10.5772\/154"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1120","DOI":"10.1167\/iovs.05-1155","article-title":"Automated Assessment of Diabetic Retinal Image Quality Based on Clarity and Field Definition","volume":"47","author":"Fleming","year":"2006","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e51","DOI":"10.1016\/S2589-7500(20)30240-5","article-title":"A Global Review of Publicly Available Datasets for Ophthalmological Imaging: Barriers to Access, Usability, and Generalisability","volume":"3","author":"Khan","year":"2021","journal-title":"Lancet Digit. Health"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mahapatra, D., Roy, P.K., Sedai, S., and Garnavi, R. (2016, January 16\u201320). A CNN Based Neurobiology Inspired Approach for Retinal Image Quality Assessment. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7590946"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"419","DOI":"10.2147\/OPTH.S235751","article-title":"Artificial Intelligence to Identify Retinal Fundus Images, Quality Validation, Laterality Evaluation, Macular Degeneration, and Suspected Glaucoma","volume":"14","author":"Zapata","year":"2020","journal-title":"Clin. Ophthalmol. Auckl. NZ"},{"key":"ref_37","first-page":"816","article-title":"Automated Fundus Image Quality Assessment and Segmentation of Optic Disc Using Convolutional Neural Networks","volume":"10","author":"Bhatkalkar","year":"2020","journal-title":"Int. J. Electr. Comput. Eng. IJECE"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1038\/s42256-021-00305-2","article-title":"Code-Free Deep Learning for Multi-Modality Medical Image Classification","volume":"3","author":"Korot","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1950030","DOI":"10.1142\/S0219519419500301","article-title":"A Cnn-Based Retinal Image Quality Assessment System for Teleophthalmology","volume":"19","author":"Wang","year":"2019","journal-title":"J. Mech. Med. Biol."}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/12\/3\/1217\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:23:04Z","timestamp":1760120584000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/12\/3\/1217"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,3]]},"references-count":39,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["jcm12031217"],"URL":"https:\/\/doi.org\/10.3390\/jcm12031217","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,3]]}}}