{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T05:17:14Z","timestamp":1787807834656,"version":"build-2784847793"},"reference-count":38,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T00:00:00Z","timestamp":1787616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"VUNO Inc.","award":["SNUH 0620242810"],"award-info":[{"award-number":["SNUH 0620242810"]}]},{"DOI":"10.13039\/100031738","name":"Korea Medical Institute","doi-asserted-by":"crossref","award":["KMI20260230"],"award-info":[{"award-number":["KMI20260230"]}],"id":[{"id":"10.13039\/100031738","id-type":"DOI","asserted-by":"crossref"}]},{"award":["KMI20260230"],"award-info":[{"award-number":["KMI20260230"]}],"id":[{"id":"https:\/\/ror.org\/03c6j9p64","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Double reading with arbitration improves diagnostic reliability in fundus screening but requires repeated human interpretation. Whether artificial intelligence (AI) can serve as a first-pass decision source within this workflow, and whether this applies consistently across retinal diseases with differing inter-reader agreement, remains unclear. Methods: In this retrospective study, 6904 color fundus photographs from 2593 patients at a tertiary screening center were analyzed for age-related macular degeneration (AMD), diabetic retinopathy (DR), and retinal vein occlusion (RVO). Three retina specialists independently labeled each image, and an AI system provided binary classifications at a prespecified operating threshold targeting 0.99-sensitivity. In AI\u2013human double reading, the AI and one reader independently interpreted each image, and a second reader arbitrated discordant cases; this was compared with conventional human\u2013human double reading. Three reader combinations were evaluated per disease. Results: AI\u2013human reading required 1.02\u20131.11 human reads per image versus 2.00\u20132.10 for human\u2013human reading. For DR and RVO, human\u2013human reading yielded sensitivities of 0.974 and 0.982 and specificities of 0.999 and 1.000, respectively. Across AI\u2013human combinations, sensitivity and specificity did not differ significantly from human\u2013human reading (all p \u2265 0.05; specificity differences \u22640.001). For AMD (human\u2013human sensitivity 0.845, specificity 1.000), AI\u2013human sensitivity varied: two combinations were higher (0.916 and 0.950; both p &lt; 0.001) and one comparable (0.842; p = 0.742). AMD specificity remained \u22650.978. Conclusions: AI\u2013human reading halved human reading volume without significant loss for DR and RVO; AMD varied by configuration. AI use within double-reading workflows should account for disease-specific inter-reader agreement, reader composition, and operating threshold. This was a single-center, retrospective study, prospective external validation in a multicenter setting is warranted before clinical implementation.<\/jats:p>","DOI":"10.3390\/diagnostics16172707","type":"journal-article","created":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T08:00:48Z","timestamp":1787644848000},"page":"2707","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating Artificial Intelligence as a First-Pass Reader in Fundus Photograph Screening: A Multireader Workflow Validation Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-9988-9317","authenticated-orcid":false,"given":"Jihyeon","family":"Baek","sequence":"first","affiliation":[{"name":"VUNO Inc., Seoul 06541, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3221-5121","authenticated-orcid":false,"given":"Richul","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 03080, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6237-468X","authenticated-orcid":false,"given":"Doohyun","family":"Park","sequence":"additional","affiliation":[{"name":"VUNO Inc., Seoul 06541, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7387-1315","authenticated-orcid":false,"given":"Kunho","family":"Bae","sequence":"additional","affiliation":[{"name":"Department of Ophthalmology, Seoul National University Hospital, 101 Daehak-ro, Jongno-gu, Seoul 03080, Republic of Korea"},{"name":"Mass Eye and Ear and the Department of Ophthalmology, Harvard Medical School, Boston, MA 02114, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,25]]},"reference":[{"key":"ref_1","unstructured":"Taylor, D., and Widdowson, S. (2026, August 19). The Management of Grading Quality: Good Practice in the Quality Assurance of Grading, Available online: https:\/\/assets.publishing.service.gov.uk\/government\/uploads\/system\/uploads\/attachment_data\/file\/512832\/The_Management_of_Grading.pdf."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3310\/hta20920","article-title":"An Observational Study to Assess If Automated Diabetic Retinopathy Image Assessment Software Can Replace One or More Steps of Manual Imaging Grading and to Determine Their Cost-Effectiveness","volume":"20","author":"Tufail","year":"2016","journal-title":"Health Technol. Assess."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1136\/bjophthalmol-2020-316594","article-title":"Prospective Evaluation of an Artificial Intelligence-Enabled Algorithm for Automated Diabetic Retinopathy Screening of 30 000 Patients","volume":"105","author":"Heydon","year":"2021","journal-title":"Br. J. Ophthalmol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100914","DOI":"10.1016\/j.landig.2025.100914","article-title":"Automated Retinal Image Analysis Systems to Triage for Grading of Diabetic Retinopathy: A Large-Scale, Open-Label, National Screening Programme in England","volume":"7","author":"Rudnicka","year":"2025","journal-title":"Lancet Digit. Health"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1016\/j.ophtha.2018.02.037","article-title":"A Deep Learning Algorithm for Prediction of Age-Related Eye Disease Study Severity Scale for Age-Related Macular Degeneration from Color Fundus Photography","volume":"125","author":"Grassmann","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2026","DOI":"10.1038\/s41433-022-02239-4","article-title":"Artificial Intelligence to Distinguish Retinal Vein Occlusion Patients Using Color Fundus Photographs","volume":"37","author":"Ren","year":"2023","journal-title":"Eye"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.ophtha.2019.05.029","article-title":"Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images","volume":"127","author":"Son","year":"2020","journal-title":"Ophthalmology"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1038\/s41746-023-00991-9","article-title":"The Performance of a Deep Learning System in Assisting Junior Ophthalmologists in Diagnosing 13 Major Fundus Diseases: A Prospective Multi-Center Clinical Trial","volume":"7","author":"Li","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e070904","DOI":"10.1136\/bmj-2022-070904","article-title":"Reporting Guideline for the Early Stage Clinical Evaluation of Decision Support Systems Driven by Artificial Intelligence: DECIDE-AI","volume":"377","author":"Vasey","year":"2022","journal-title":"BMJ"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1264","DOI":"10.1016\/j.ophtha.2018.01.034","article-title":"Grader Variability and the Importance of Reference Standards for Evaluating Machine Learning Models for Diabetic Retinopathy","volume":"125","author":"Krause","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1001\/jama.2017.18152","article-title":"Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images from Multiethnic Populations with Diabetes","volume":"318","author":"Ting","year":"2017","journal-title":"JAMA"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"Gulshan","year":"2016","journal-title":"JAMA"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2125","DOI":"10.1007\/s40123-024-00981-4","article-title":"Retina Fundus Photograph-Based Artificial Intelligence Algorithms in Medicine: A Systematic Review","volume":"13","author":"Grzybowski","year":"2024","journal-title":"Ophthalmol. Ther."},{"key":"ref_14","unstructured":"Lee, S., Kang, S., Park, I., Kwon, G., Baek, J., and Park, D. (2026, January 23\u201327). Frequency-Balanced Retinal Representation Learning with Mutual Information Regularization. Proceedings of the International Conference on Learning Representations, Rio de Janeiro, Brazil."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A Coefficient of Agreement for Nominal Scales","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ. Psychol. Meas."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring Nominal Scale Agreement among Many Raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol. Bull."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/BF02295996","article-title":"Note on the Sampling Error of the Difference Between Correlated Proportions or Percentages","volume":"12","author":"McNemar","year":"1947","journal-title":"Psychometrika"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0272989X06295361","article-title":"Decision Curve Analysis: A Novel Method for Evaluating Prediction Models","volume":"26","author":"Vickers","year":"2006","journal-title":"Med. Decis. Mak."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"i6","DOI":"10.1136\/bmj.i6","article-title":"Net Benefit Approaches to the Evaluation of Prediction Models, Molecular Markers, and Diagnostic Tests","volume":"352","author":"Vickers","year":"2016","journal-title":"BMJ"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1093\/biomet\/26.4.404","article-title":"The Use of Confidence or Fiducial Limits Illustrated in the Case of the Binomial","volume":"26","author":"Clopper","year":"1934","journal-title":"Biometrika"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1038\/s41746-024-01389-x","article-title":"Mitigation of AI Adoption Bias through an Improved Autonomous AI System for Diabetic Retinal Disease","volume":"7","author":"Lavin","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The Measurement of Observer Agreement for Categorical Data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e537","DOI":"10.1016\/S2589-7500(20)30218-1","article-title":"Reporting Guidelines for Clinical Trial Reports for Interventions Involving Artificial Intelligence: The CONSORT-AI Extension","volume":"2","author":"Liu","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.ophtha.2016.11.014","article-title":"Automated Diabetic Retinopathy Image Assessment Software","volume":"124","author":"Tufail","year":"2017","journal-title":"Ophthalmology"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4548","DOI":"10.1167\/iovs.13-11804","article-title":"Methods and Reproducibility of Grading Optimized Digital Color Fundus Photographs in the Age-Related Eye Disease Study 2 (AREDS2 Report Number 2)","volume":"54","author":"Danis","year":"2013","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1007\/s00417-002-0602-8","article-title":"Inter- and Intra-Observer Variability in Grading Lesions of Age-Related Maculopathy and Macular Degeneration","volume":"241","author":"Scholl","year":"2003","journal-title":"Graefe\u2019s Arch. Clin. Exp. Ophthalmol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal Trial of an Autonomous AI-Based Diagnostic System for Detection of Diabetic Retinopathy in Primary Care Offices","volume":"1","author":"Lavin","year":"2018","journal-title":"npj Digit. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e235","DOI":"10.1016\/S2589-7500(22)00017-6","article-title":"Real-Time Diabetic Retinopathy Screening by Deep Learning in a Multisite National Screening Programme: A Prospective Interventional Cohort Study","volume":"4","author":"Ruamviboonsuk","year":"2022","journal-title":"Lancet Digit. Health"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e549","DOI":"10.1016\/S2589-7500(20)30219-3","article-title":"Guidelines for Clinical Trial Protocols for Interventions Involving Artificial Intelligence: The SPIRIT-AI Extension","volume":"2","author":"Liu","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1001\/jama.2023.22557","article-title":"Automation Bias and Assistive AI: Risk of Harm from AI-Driven Clinical Decision Support","volume":"330","author":"Khera","year":"2023","journal-title":"JAMA"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e58666","DOI":"10.2196\/58666","article-title":"Facilitating Trust Calibration in Artificial Intelligence\u2013Driven Diagnostic Decision Support Systems for Determining Physicians\u2019 Diagnostic Accuracy: Quasi-Experimental Study","volume":"8","author":"Sakamoto","year":"2024","journal-title":"JMIR Form. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2275","DOI":"10.1001\/jama.2023.22295","article-title":"Measuring the Impact of AI in the Diagnosis of Hospitalized Patients","volume":"330","author":"Jabbour","year":"2023","journal-title":"JAMA"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1167\/tvst.13.10.34","article-title":"Clinical Utility of Deep Learning Assistance for Detecting Various Abnormal Findings in Color Retinal Fundus Images: A Reader Study","volume":"13","author":"Shin","year":"2024","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1038\/s41746-024-01248-9","article-title":"Effects of Artificial Intelligence Implementation on Efficiency in Medical Imaging\u2014A Systematic Literature Review and Meta-Analysis","volume":"7","author":"Wenderott","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"844","DOI":"10.1016\/j.ophtha.2012.10.036","article-title":"Clinical Classification of Age-Related Macular Degeneration","volume":"120","author":"Ferris","year":"2013","journal-title":"Ophthalmology"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1677","DOI":"10.1016\/S0161-6420(03)00475-5","article-title":"Proposed International Clinical Diabetic Retinopathy and Diabetic Macular Edema Disease Severity Scales","volume":"110","author":"Wilkinson","year":"2003","journal-title":"Ophthalmology"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1016\/j.ophtha.2010.01.058","article-title":"Natural History of Branch Retinal Vein Occlusion: An Evidence-Based Systematic Review","volume":"117","author":"Rogers","year":"2010","journal-title":"Ophthalmology"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1016\/j.ophtha.2010.01.060","article-title":"Natural History of Central Retinal Vein Occlusion: An Evidence-Based Systematic Review","volume":"117","author":"McIntosh","year":"2010","journal-title":"Ophthalmology"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/17\/2707\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T04:17:08Z","timestamp":1787804228000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/17\/2707"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,25]]},"references-count":38,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2026,9]]}},"alternative-id":["diagnostics16172707"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16172707","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,25]]}}}