{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:55:00Z","timestamp":1790297700415,"version":"4.1.0"},"reference-count":33,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T00:00:00Z","timestamp":1753056000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100008349","name":"Boehringer Ingelheim International GmbH","doi-asserted-by":"publisher","award":["749092"],"award-info":[{"award-number":["749092"]}],"id":[{"id":"10.13039\/100008349","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008349","name":"Boehringer Ingelheim International GmbH","doi-asserted-by":"publisher","award":["1436-0008"],"award-info":[{"award-number":["1436-0008"]}],"id":[{"id":"10.13039\/100008349","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008349","name":"Boehringer Ingelheim International GmbH","doi-asserted-by":"publisher","award":["SIVS1068"],"award-info":[{"award-number":["SIVS1068"]}],"id":[{"id":"10.13039\/100008349","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Discrepancies in diabetic retinopathy (DR) grading are well-documented, with retinal non-perfusion (RNP) quantification posing greater challenges. This study assessed intergrader agreement in DR evaluation, focusing on qualitative severity grading and quantitative RNP measurement. We aimed to improve agreement through structured consensus meetings. Methods: A retrospective analysis of 100 comparisons from 50 eyes (36 patients) was conducted. Two paired medical retina fellows graded ultra-widefield color fundus photographs (CFP) and fundus fluorescein angiography (FFA) images. CFP assessments included DR severity using the International Clinical Diabetic Retinopathy (ICDR) grading system, DR Severity Scale (DRSS), and predominantly peripheral lesions (PPL). FFA-based RNP was defined as capillary loss with grayscale matching the foveal avascular zone. Weekly adjudication by a senior specialist resolved discrepancies. Intergrader agreement was evaluated using Cohen\u2019s kappa (qualitative DRSS) and intraclass correlation coefficients (ICC) (quantitative RNP). Bland\u2013Altman analysis assessed bias and variability. Results: After eight consensus meetings, CFP grading agreement improved to excellent: kappa = 91% (ICDR DR severity), 89% (DRSS), and 89% (PPL). FFA-based PPL agreement reached 100%. For RNP, the non-perfusion index (NPI) showed moderate overall ICC (0.49), with regional ICCs ranging from 0.40 to 0.57 (highest in the nasal region, ICC = 0.57). Bland\u2013Altman analysis revealed a mean NPI difference of 0.12 (limits: \u22120.11 to 0.35), indicating acceptable variability despite outliers. Conclusions: Structured consensus training achieved excellent intergrader agreement for DR severity and PPL grading, supporting the clinical reliability of ultra-widefield imaging. However, RNP measurement variability underscores the need for standardized protocols and automated tools to enhance reproducibility. This process is critical for developing robust AI-based screening systems.<\/jats:p>","DOI":"10.3390\/diagnostics15141831","type":"journal-article","created":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T10:35:31Z","timestamp":1753094131000},"page":"1831","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Intergrader Agreement on Qualitative and Quantitative Assessment of Diabetic Retinopathy Severity Using Ultra-Widefield Imaging: INSPIRED Study Report 1"],"prefix":"10.3390","volume":"15","author":[{"given":"Eleonora","family":"Riotto","sequence":"first","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, 162 City Road, London EC1V 2PD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei-Shan","family":"Tsai","sequence":"additional","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, 162 City Road, London EC1V 2PD, UK"},{"name":"Hampshire Hospitals NHS Foundation Trust, Aldermaston Road, Basingstoke RG24 9NA, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hagar","family":"Khalid","sequence":"additional","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, 162 City Road, London EC1V 2PD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7956-6056","authenticated-orcid":false,"given":"Francesca","family":"Lamanna","sequence":"additional","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, 162 City Road, London EC1V 2PD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Louise","family":"Roch","sequence":"additional","affiliation":[{"name":"Boehringer Ingelheim International GmbH, Birkendorfer Str. 65, 88397 Biberach, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Medha","family":"Manoj","sequence":"additional","affiliation":[{"name":"Leicester Medical School, George Davies Centre, Lancaster Rd, Leicester LE1 7HA, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8952-0659","authenticated-orcid":false,"given":"Sobha","family":"Sivaprasad","sequence":"additional","affiliation":[{"name":"Moorfields Eye Hospital NHS Foundation Trust, 162 City Road, London EC1V 2PD, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1608","DOI":"10.1016\/j.ophtha.2018.04.007","article-title":"Guidelines on Diabetic Eye Care: The International Council of Ophthalmology Recommendations for Screening, Follow-up, Referral, and Treatment Based on Resource Settings","volume":"125","author":"Wong","year":"2018","journal-title":"Ophthalmology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1007\/s00592-021-01687-w","article-title":"The contribution of the English NHS Diabetic Eye Screening Programme to reductions in diabetes-related blindness, comparisons within Europe, and future challenges","volume":"58","author":"Scanlon","year":"2021","journal-title":"Acta Diabetol."},{"key":"ref_3","unstructured":"(2025, March 11). Grading Diabetic Retinopathy from Stereoscopic Color Fundus Photographs\u2014An Extension of the Modified Airlie House Classification. ETDRS Report Number 10. Early Treatment Diabetic Retinopathy Study Research Group, Available online: https:\/\/pubmed.ncbi.nlm.nih.gov\/2062513\/."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1360","DOI":"10.1016\/j.ophtha.2016.01.043","article-title":"Identification of Diabetic Retinopathy and Ungradable Image Rate with Ultrawide Field Imaging in a National Teleophthalmology Program","volume":"123","author":"Silva","year":"2016","journal-title":"Ophthalmology"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"(1995). A Randomized Clinical Trial of Early Panretinal Photocoagulation for Ischermic Central Vein Occlusion: The Central Vein Occlusion Study Group N Report. Ophthalmology, 102, 1434\u20131444.","DOI":"10.1016\/S0161-6420(95)30848-2"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1016\/j.oret.2019.05.007","article-title":"Classification and Guidelines for Widefield Imaging: Recommendations from the International Widefield Imaging Study Group","volume":"3","author":"Choudhry","year":"2019","journal-title":"Ophthalmol Retin."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1302","DOI":"10.1097\/IAE.0000000000003479","article-title":"Assessment of Fluorescein Angiography Nonperfusion in Eyes with Diabetic Retinopathy using Ultrawide Field Retinal Imaging","volume":"42","author":"Silva","year":"2022","journal-title":"Retina"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2465","DOI":"10.1016\/j.ophtha.2015.07.034","article-title":"Diabetic Retinopathy Severity and Peripheral Lesions Are Associated with Nonperfusion on Ultrawide Field Angiography","volume":"122","author":"Silva","year":"2015","journal-title":"Ophthalmology"},{"key":"ref_9","unstructured":"(2025, March 11). Association of Predominantly Peripheral Lesions on Ultra-Widefield Imaging and the Risk of Diabetic Retinopathy Worsening Over Time, Available online: https:\/\/pubmed.ncbi.nlm.nih.gov\/35980608\/."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1431","DOI":"10.1016\/S0161-6420(84)34126-4","article-title":"Distribution of capillary nonperfusion in early-stage diabetic retinopathy","volume":"91","author":"Niki","year":"1984","journal-title":"Ophthalmology"},{"key":"ref_11","unstructured":"(2025, March 11). Peripheral Lesions Identified on Ultrawide Field Imaging Predict Increased Risk of Diabetic Retinopathy Progression over 4 Years, Available online: https:\/\/pubmed.ncbi.nlm.nih.gov\/25704318\/."},{"key":"ref_12","first-page":"1140","article-title":"Standard Care vs. Corticosteroid for Retinal Vein Occlusion (SCORE) Study system for evaluation of stereoscopic color fundus photographs and fluorescein angiograms: SCORE Study Report 9","volume":"128","author":"Blodi","year":"2010","journal-title":"Arch. Ophthalmol. Chic. Ill. 1960."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.preteyeres.2018.10.001","article-title":"The clinical relevance of visualising the peripheral retina","volume":"68","author":"Quinn","year":"2019","journal-title":"Prog. Retin. Eye Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1097\/IAE.0b013e3182278b64","article-title":"Ultra-wide-field angiography improves the detection and classification of diabetic retinopathy","volume":"32","author":"Wessel","year":"2012","journal-title":"Retina"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1136\/bjophthalmol-2015-306652","article-title":"Measuring the precise area of peripheral retinal non-perfusion using ultra-widefield imaging and its correlation with the ischaemic index","volume":"100","author":"Tan","year":"2016","journal-title":"Br. J. Ophthalmol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1111\/aos.15667","article-title":"Inter-grader reliability in the Danish screening programme for diabetic retinopathy","volume":"101","author":"Thykjaer","year":"2023","journal-title":"Acta Ophthalmol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Midena, E., Marchione, G., Di Giorgio, S., Rotondi, G., Longhin, E., Frizziero, L., Pilotto, E., Parrozzani, R., and Midena, G. (2022). Ultra-wide-field fundus photography compared to ophthalmoscopy in diagnosing and classifying major retinal diseases. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-23170-4"},{"key":"ref_18","unstructured":"(2025, April 17). Greater Houston Retina Research. Faricimab for Retinal Non-Perfusion Associated With Non-Proliferative Diabetic Retinopathy: The MAGIC Phase 2, Multi-Center, Open-Label, Randomized Controlled Trial. Report No.: NCT05681884, Available online: https:\/\/clinicaltrials.gov\/study\/NCT05681884."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"864","DOI":"10.1016\/j.ophtha.2014.11.016","article-title":"Assessment of Accuracy and Precision of Quantification of Ultra-Widefield Images","volume":"122","author":"Sagong","year":"2015","journal-title":"Ophthalmology"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1677","DOI":"10.1016\/S0161-6420(03)00475-5","article-title":"Proposed international clinical diabetic retinopathy and diabetic macular edema disease severity scales","volume":"110","author":"Wilkinson","year":"2003","journal-title":"Ophthalmology"},{"key":"ref_21","unstructured":"(2015). Wide Field Ophthalmic Photography Image Storage SOP Classes (Standard No. Supplement 173)."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1511","DOI":"10.1002\/sim.4780111109","article-title":"A goodness-of-fit approach to inference procedures for the kappa statistic: Confidence interval construction, significance-testing and sample size estimation","volume":"11","author":"Donner","year":"1992","journal-title":"Stat. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1167\/tvst.12.7.13","article-title":"Inter- Rater Reliability of Proliferative Diabetic Retinopathy Assessment on Wide-Field OCT-Angiography and Fluorescein Angiography","volume":"12","author":"Shah","year":"2023","journal-title":"Transl. Vis. Sci. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"221","DOI":"10.3109\/08820538.2012.708812","article-title":"Ultra-wide field retinal imaging in detection, classification, and management of diabetic retinopathy","volume":"27","author":"Soliman","year":"2012","journal-title":"Semin. Ophthalmol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1007\/s00417-020-04607-w","article-title":"Distribution of peripheral lesions identified by mydriatic ultra-wide field fundus imaging in diabetic retinopathy","volume":"258","author":"Verma","year":"2020","journal-title":"Graefes Arch. Clin. Exp. Ophthalmol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1097\/IAE.0b013e3181e36c6d","article-title":"Ischemic index and neovascularization in central retinal vein occlusion","volume":"31","author":"Tsui","year":"2011","journal-title":"Retina"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.ajo.2014.03.009","article-title":"Patterns of peripheral retinal and central macula ischemia in diabetic retinopathy as evaluated by ultra-widefield fluorescein angiography","volume":"158","author":"Sim","year":"2014","journal-title":"Am. J. Ophthalmol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1038","DOI":"10.1016\/j.ajo.2013.01.007","article-title":"Characterization of ischemic index using ultra-widefield fluorescein angiography in patients with focal and diffuse recalcitrant diabetic macular edema","volume":"155","author":"Patel","year":"2013","journal-title":"Am. J. Ophthalmol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1016\/j.ajo.2015.02.009","article-title":"Quantitative analysis of peripheral vasculitis, ischemia, and vascular leakage in uveitis using ultra-widefield fluorescein angiography","volume":"159","author":"Karampelas","year":"2015","journal-title":"Am. J. Ophthalmol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1097\/IAE.0000000000000148","article-title":"Area of peripheral retinal nonperfusion and treatment response in branch and central retinal vein occlusion","volume":"34","author":"Singer","year":"2014","journal-title":"Retina"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Miao, J., Yu, J., Zou, W., Su, N., Peng, Z., Wu, X., Huang, J., Fang, Y., Yuan, S., and Xie, P. (2022). Deep Learning Models for Segmenting Non-perfusion Area of Color Fundus Photographs in Patients With Branch Retinal Vein Occlusion. Front. Med., 9.","DOI":"10.3389\/fmed.2022.794045"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"101197","DOI":"10.1016\/j.xcrm.2023.101197","article-title":"An artificial intelligence system for the whole process from diagnosis to treatment suggestion of ischemic retinal diseases","volume":"4","author":"Zhao","year":"2023","journal-title":"Cell Rep. Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.ajo.2017.05.024","article-title":"Distribution of Nonperfusion Area on Ultra-widefield Fluorescein Angiography in Eyes With Diabetic Macular Edema: DAVE Study","volume":"180","author":"Fan","year":"2017","journal-title":"Am. J. Ophthalmol."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/14\/1831\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:13:14Z","timestamp":1760033594000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/14\/1831"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,21]]},"references-count":33,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["diagnostics15141831"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15141831","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,21]]}}}