{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T15:28:49Z","timestamp":1789658929028,"version":"4.0.1"},"reference-count":31,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T00:00:00Z","timestamp":1789430400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"VUNO Inc.","award":["NA"],"award-info":[{"award-number":["NA"]}]},{"name":"Korea Medical Institute","award":["KMI20260230"],"award-info":[{"award-number":["KMI20260230"]}],"id":[{"id":"https:\/\/ror.org\/03c6j9p64","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>\n                    The clinical value of artificial intelligence (AI) in fundus photography depends on workflow efficiency as well as diagnostic performance. We evaluated an AI-assisted negative-screening workflow for diabetic retinopathy (DR), retinal vein occlusion (RVO), and age-related macular degeneration (AMD) using 6,904 color fundus photographs independently graded by three retina specialists. In this workflow, AI-negative images were classified as negative without ophthalmologist review, whereas AI-positive images were referred for human interpretation. For each reader\u2013disease pair, the reference standard was agreement between the other two readers; discordant cases were excluded (DR, 1.2%\u20132.3%; RVO, 0.3%\u20130.6%; AMD, 6.1%\u201311.6%). The workflow reduced direct ophthalmologist review to 7.3%\u20138.0% of images for DR, 11.7%\u201311.9% for RVO, and 13.5%\u201316.8% for AMD. For DR, sensitivity decreased significantly (0.9168 to 0.8838; difference, -0.0330; 95% confidence interval [CI], -0.0495 to -0.0165;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001), whereas specificity did not change significantly (0.9961 to 0.9983;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.183). For RVO, sensitivity was unchanged (0.9807) and specificity did not change significantly (0.9985 to 0.9988;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.320). For AMD, sensitivity decreased significantly (0.8570 to 0.8445; difference, -0.0124; 95% CI, -0.0215 to -0.0033;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.008), whereas specificity did not change significantly (0.9685 to 0.9843;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.324). In an exploratory image-level analysis for at least one target disease, review decreased to 27.7%\u201330.1%; sensitivity changed from 0.9122 to 0.9039 (difference, -0.0083; 95% CI, -0.0131 to -0.0035;\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.001) and specificity from 0.9659 to 0.9804 (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.334). AI-assisted negative-screening can reduce ophthalmologist workload across multiple retinal diseases, but with disease-specific safety trade-offs: because AI-negative images are not reviewed, reduced sensitivity for DR and AMD means a small proportion of true-positive cases would go undetected, potentially delaying diagnosis and treatment for sight-threatening disease. These results support disease-specific implementation strategies that balance workload reduction against missed positive cases.\n                  <\/jats:p>","DOI":"10.1371\/journal.pdig.0001700","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T17:53:12Z","timestamp":1789494792000},"page":"e0001700","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating an AI-assisted triage workflow for retinal diseases"],"prefix":"10.1371","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6237-468X","authenticated-orcid":true,"given":"Doohyun","family":"Park","sequence":"first","affiliation":[{"name":"VUNO Inc.","place":["Seoul, Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richul","family":"Oh","sequence":"additional","affiliation":[{"name":"Seoul National University Hospital","place":["Seoul, Korea"],"department":["Department of Ophthalmology"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9988-9317","authenticated-orcid":true,"given":"Jihyeon","family":"Baek","sequence":"additional","affiliation":[{"name":"VUNO Inc.","place":["Seoul, Korea"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7387-1315","authenticated-orcid":true,"given":"Kunho","family":"Bae","sequence":"additional","affiliation":[{"name":"Seoul National University Hospital","place":["Seoul, Korea"],"department":["Department of Ophthalmology"]},{"name":"Mass Eye and Ear and the Department of Ophthalmology at Harvard Medical School","place":["Boston, Massachusetts, United States of America"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,15]]},"reference":[{"key":"pdig.0001700.ref001","doi-asserted-by":"crossref","first-page":"101971","DOI":"10.1016\/j.media.2021.101971","article-title":"Applications of deep learning in fundus images: A review","volume":"69","author":"T Li","year":"2021","journal-title":"Med Image Anal"},{"issue":"22","key":"pdig.0001700.ref002","doi-asserted-by":"crossref","first-page":"2402","DOI":"10.1001\/jama.2016.17216","article-title":"Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs","volume":"316","author":"V Gulshan","year":"2016","journal-title":"JAMA"},{"key":"pdig.0001700.ref003","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1038\/s41746-018-0040-6","article-title":"Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices","volume":"1","author":"MD Abr\u00e0moff","year":"2018","journal-title":"NPJ Digit Med"},{"issue":"3","key":"pdig.0001700.ref004","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1177\/1932296820909900","article-title":"Autonomous Artificial Intelligence in Diabetic Retinopathy: From Algorithm to Clinical Application","volume":"15","author":"R Channa","year":"2021","journal-title":"J Diabetes Sci Technol"},{"issue":"1","key":"pdig.0001700.ref005","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1038\/s41746-023-00929-1","article-title":"Autonomous AI systems in the face of liability, regulations and costs","volume":"6","author":"AD Saenz","year":"2023","journal-title":"NPJ Digit Med"},{"issue":"1","key":"pdig.0001700.ref006","doi-asserted-by":"crossref","first-page":"142","DOI":"10.2337\/cd23-0019","article-title":"Clinical Implementation of Autonomous Artificial Intelligence Systems for Diabetic Eye Exams: Considerations for Success","volume":"42","author":"RM Wolf","year":"2024","journal-title":"Clinical Diabetes"},{"issue":"1","key":"pdig.0001700.ref007","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1038\/s41746-024-01248-9","article-title":"Effects of artificial intelligence implementation on efficiency in medical imaging-a systematic literature review and meta-analysis","volume":"7","author":"K Wenderott","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"4","key":"pdig.0001700.ref008","doi-asserted-by":"crossref","first-page":"100330","DOI":"10.1016\/j.xops.2023.100330","article-title":"AI-Human Hybrid Workflow Enhances Teleophthalmology for the Detection of Diabetic Retinopathy","volume":"3","author":"ER Dow","year":"2023","journal-title":"Ophthalmol Sci"},{"issue":"5","key":"pdig.0001700.ref009","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2022.9960","article-title":"Artificial Intelligence for Screening of Multiple Retinal and Optic Nerve Diseases","volume":"5","author":"L Dong","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"1","key":"pdig.0001700.ref010","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1038\/s41746-023-00991-9","article-title":"The performance of a deep learning system in assisting junior ophthalmologists in diagnosing 13 major fundus diseases: a prospective multi-center clinical trial","volume":"7","author":"B Li","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"1","key":"pdig.0001700.ref011","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.ophtha.2019.05.029","article-title":"Development and Validation of Deep Learning Models for Screening Multiple Abnormal Findings in Retinal Fundus Images","volume":"127","author":"J Son","year":"2020","journal-title":"Ophthalmology"},{"issue":"1","key":"pdig.0001700.ref012","doi-asserted-by":"crossref","first-page":"5934","DOI":"10.1038\/s41598-023-32518-3","article-title":"An interpretable and interactive deep learning algorithm for a clinically applicable retinal fundus diagnosis system by modelling finding-disease relationship","volume":"13","author":"J Son","year":"2023","journal-title":"Sci Rep"},{"issue":"9","key":"pdig.0001700.ref013","doi-asserted-by":"crossref","first-page":"1677","DOI":"10.1016\/S0161-6420(03)00475-5","article-title":"Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales","volume":"110","author":"CP Wilkinson","year":"2003","journal-title":"Ophthalmology"},{"issue":"5","key":"pdig.0001700.ref014","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1038\/s41433-022-02007-4","article-title":"Retinal vein occlusion (RVO) guideline: executive summary","volume":"36","author":"L Nicholson","year":"2022","journal-title":"Eye"},{"issue":"4","key":"pdig.0001700.ref015","doi-asserted-by":"crossref","first-page":"844","DOI":"10.1016\/j.ophtha.2012.10.036","article-title":"Clinical classification of age-related macular degeneration","volume":"120","author":"FL Ferris","year":"2013","journal-title":"Ophthalmology"},{"issue":"421","key":"pdig.0001700.ref016","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1080\/01621459.1993.10594284","article-title":"Approximate Inference in Generalized Linear Mixed Models","volume":"88","author":"NE Breslow","year":"1993","journal-title":"J Am Stat Assoc"},{"issue":"2","key":"pdig.0001700.ref017","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1080\/03610919508813243","article-title":"Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests an anova approach with dependent observations","volume":"24","author":"NA Obuchowski","year":"1995","journal-title":"Commun Stat Simul Comput"},{"issue":"2","key":"pdig.0001700.ref018","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/BF02295996","article-title":"Note on the sampling error of the difference between correlated proportions or percentages","volume":"12","author":"Q McNemar","year":"1947","journal-title":"Psychometrika"},{"issue":"9","key":"pdig.0001700.ref019","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1016\/j.ophtha.2018.02.037","article-title":"A Deep Learning Algorithm for Prediction of Age-Related Eye Disease Study Severity Scale for Age-Related Macular Degeneration from Color Fundus Photography","volume":"125","author":"F Grassmann","year":"2018","journal-title":"Ophthalmology"},{"issue":"8","key":"pdig.0001700.ref020","first-page":"1079","article-title":"Development and evaluation of a deep learning model for the detection of multiple fundus diseases based on colour fundus photography","volume":"106","author":"B Li","year":"2022","journal-title":"Br J Ophthalmol"},{"issue":"8","key":"pdig.0001700.ref021","doi-asserted-by":"crossref","first-page":"2125","DOI":"10.1007\/s40123-024-00981-4","article-title":"Retina fundus photograph-based artificial intelligence algorithms in medicine: a systematic review","volume":"13","author":"A Grzybowski","year":"2024","journal-title":"Ophthalmol Ther"},{"issue":"10","key":"pdig.0001700.ref022","doi-asserted-by":"crossref","first-page":"2026","DOI":"10.1038\/s41433-022-02239-4","article-title":"Artificial intelligence to distinguish retinal vein occlusion patients using color fundus photographs","volume":"37","author":"X Ren","year":"2023","journal-title":"Eye"},{"issue":"3","key":"pdig.0001700.ref023","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1136\/bjo-2022-322940","article-title":"Application of artificial intelligence system for screening multiple fundus diseases in Chinese primary healthcare settings: a real-world, multicentre and cross-sectional study of 4795 cases","volume":"108","author":"C Gu","year":"2024","journal-title":"Br J Ophthalmol"},{"issue":"10","key":"pdig.0001700.ref024","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1167\/tvst.13.10.34","article-title":"Clinical Utility of Deep Learning Assistance for Detecting Various Abnormal Findings in Color Retinal Fundus Images: A Reader Study","volume":"13","author":"JY Shin","year":"2024","journal-title":"Transl Vis Sci Technol"},{"issue":"2","key":"pdig.0001700.ref025","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1001\/jama.2023.26074","article-title":"Age-Related Macular Degeneration","volume":"331","author":"M Fleckenstein","year":"2024","journal-title":"JAMA"},{"issue":"1","key":"pdig.0001700.ref026","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1007\/s00417-002-0602-8","article-title":"Inter- and intra-observer variability in grading lesions of age-related maculopathy and macular degeneration","volume":"241","author":"HPN Scholl","year":"2003","journal-title":"Graefes Arch Clin Exp Ophthalmol"},{"issue":"23","key":"pdig.0001700.ref027","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.1001\/jama.2023.22557","article-title":"Automation Bias and Assistive AI","volume":"330","author":"R Khera","year":"2023","journal-title":"JAMA"},{"key":"pdig.0001700.ref028","doi-asserted-by":"crossref","DOI":"10.2196\/58666","article-title":"Facilitating trust calibration in artificial intelligence\u2013driven diagnostic decision support systems for determining physicians\u2019 diagnostic accuracy: Quasi-experimental study","volume":"8","author":"T Sakamoto","year":"2024","journal-title":"JMIR Form Res"},{"issue":"4","key":"pdig.0001700.ref029","doi-asserted-by":"crossref","first-page":"2618","DOI":"10.3390\/ai5040126","article-title":"ChatGPT: Transforming Healthcare with AI","volume":"5","author":"F Neha","year":"2024","journal-title":"AI"},{"issue":"4","key":"pdig.0001700.ref030","doi-asserted-by":"crossref","first-page":"142","DOI":"10.3390\/ai7040142","article-title":"Mapping the LLM Landscape: A Cross-Family Survey of Architectures, Alignment Methods, and Benchmark Performance","volume":"7","author":"D Bhati","year":"2026","journal-title":"AI"},{"issue":"9","key":"pdig.0001700.ref031","doi-asserted-by":"crossref","first-page":"226","DOI":"10.3390\/ai6090226","article-title":"Retrieval-Augmented Generation (RAG) in Healthcare: A Comprehensive Review","volume":"6","author":"F Neha","year":"2025","journal-title":"AI"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001700","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T17:53:18Z","timestamp":1789494798000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001700"}},"subtitle":[],"editor":[{"given":"Arun J.","family":"Thirunavukarasu","sequence":"first","affiliation":[{"name":"London School of Hygiene & Tropical Medicine","place":["UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,15]]},"references-count":31,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,15]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001700","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,15]]}}}