{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T05:02:46Z","timestamp":1790744566478,"version":"4.1.0"},"reference-count":28,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T00:00:00Z","timestamp":1790726400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Plus disease is a treatment-relevant sign of retinopathy of prematurity, but fundus images from the same infant may differ in image quality, posterior-pole coverage, and visible vascular severity. We evaluated deep-learning classification using a patient-disjoint design, within-infant variability, and an ophthalmologist reader study.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>The primary analysis included 732 Phoenix ICON fundus images from 38 infants, assigned exclusively to a 634-image development pool or a 98-image hold-out set from eight previously unseen infants. Five pretrained convolutional neural networks were evaluated at the image level. Hold-out predictions were grouped by infant and laterality to characterize descriptive within-infant variation. A historical secondary analysis used a random image-level partition without enforced patient separation and was considered exploratory. Two ophthalmologists independently graded 30 unique isolated images for inter-observer analysis; 10 were presented twice, resulting in 40 presentations for intra-observer repeatability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>DenseNet121 achieved an image-level ROC-AUC of 0.929; however, infant-equal-weighted image-level ROC-AUC was 0.905 and leave-one-infant-out ROC-AUC ranged from 0.838 to 0.989, demonstrating sensitivity to the small clustered hold-out cohort. At a development-defined threshold of 0.0184, sensitivity increased to 98.0%, while specificity decreased to 46.8%. Within-infant probability variability was observed across all five CNN architectures. Ophthalmologists 1 and 2 assessed 14\/18 and 17\/18 plus-reference images, respectively, as Healthy. Exact three-category inter-observer agreement was 26\/30 (86.7%), and both reproduced the same severity grade for all 10 repeated images.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Patient-disjoint evaluation revealed substantial uncertainty in image-level performance and variability among images from the same infant. Cluster-aware analyses showed dependence on the small hold-out cohort, while the development-defined operating point demonstrated a marked sensitivity\u2013specificity trade-off. The reader study showed that an examination-derived reference label and the diagnostic evidence visible in an isolated image are not equivalent, despite high inter-observer agreement and complete categorical repeatability. Because examination identifiers were unavailable, within-infant analyses were interpreted descriptively and not as infant or examination-level diagnostic performance. Clinically useful systems should assess image quality, select informative views, and integrate evidence across multiple images and both eyes. External prospective multicentre validation is required before clinical implementation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1960317","type":"journal-article","created":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T04:54:00Z","timestamp":1790744040000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Explainability-informed patient-disjoint evaluation of deep learning for plus disease in retinopathy of prematurity with an ophthalmologist reader study"],"prefix":"10.3389","volume":"13","author":[{"given":"Rouhollah","family":"Kian Ara","sequence":"first","affiliation":[{"name":"Department of Cybernetics and Biomedical Engineering, VSB\u2013Technical University of Ostrava","place":["Ostrava, Czech Republic"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan","family":"Kub\u00ed\u010dek","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Biomedical Engineering, VSB\u2013Technical University of Ostrava","place":["Ostrava, Czech Republic"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sumit","family":"Kaushik","sequence":"additional","affiliation":[{"name":"Department of Cybernetics and Biomedical Engineering, VSB\u2013Technical University of Ostrava","place":["Ostrava, Czech Republic"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krist\u00fdna","family":"Mar\u0161olkov\u00e1","sequence":"additional","affiliation":[{"name":"Clinic of Ophthalmology, University Hospital Ostrava","place":["Ostrava, Czech Republic"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juraj","family":"Timkovi\u010d","sequence":"additional","affiliation":[{"name":"Clinic of Ophthalmology, University Hospital Ostrava","place":["Ostrava, Czech Republic"]},{"name":"Department of Ophthalmology, Faculty of Medicine, University of Ostrava","place":["Ostrava, Czech Republic"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,30]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e2021053924","DOI":"10.1542\/peds.2021-053924","article-title":"Retinopathy of prematurity: A global perspective and recent developments","volume":"150","author":"Sabri","year":"2022","journal-title":"Pediatrics"},{"key":"B2","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1016\/j.preteyeres.2017.09.002","article-title":"Pathophysiology, screening and treatment of ROP: A multi-disciplinary perspective","volume":"62","author":"Chan-Ling","year":"2018","journal-title":"Prog Retin Eye Res"},{"key":"B3","doi-asserted-by":"publisher","first-page":"e51","DOI":"10.1016\/j.ophtha.2021.05.031","article-title":"International classification of retinopathy of prematurity, third edition","volume":"128","author":"Chiang","year":"2021","journal-title":"Ophthalmology"},{"key":"B4","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1016\/j.survophthal.2009.02.021","article-title":"Plus disease","volume":"54","author":"Davitt","year":"2009","journal-title":"Surv Ophthalmol"},{"key":"B5","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1016\/j.jaapos.2013.01.007","article-title":"Plus disease in retinopathy of prematurity: Qualitative analysis of diagnostic process by experts","volume":"17","author":"Hewing","year":"2013","journal-title":"J AAPOS"},{"key":"B6","doi-asserted-by":"publisher","first-page":"651","DOI":"10.1001\/jamaophthalmol.2016.0611","article-title":"Expert diagnosis of plus disease in retinopathy of prematurity from computer-based image analysis","volume":"134","author":"Campbell","year":"2016","journal-title":"JAMA Ophthalmol"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.1016\/j.oret.2022.05.024","article-title":"Variability in plus disease diagnosis using single and serial images","volume":"6","author":"Cole","year":"2022","journal-title":"Ophthalmol Retina"},{"key":"B8","first-page":"47","article-title":"Evolution of plus disease in retinopathy of prematurity: Quantification by ROPtool","volume":"107","author":"Wallace","year":"2009","journal-title":"Trans Am Ophthalmol Soc"},{"key":"B9","doi-asserted-by":"publisher","first-page":"6375","DOI":"10.1038\/s41598-024-57072-4","article-title":"Automated diagnosis of plus disease in retinopathy of prematurity using quantification of vessels characteristics","volume":"14","author":"Sharafi","year":"2024","journal-title":"Sci Rep"},{"key":"B10","doi-asserted-by":"publisher","first-page":"710329","DOI":"10.3389\/fmed.2021.710329","article-title":"The impact of artificial intelligence and deep learning in eye diseases: A review","volume":"8","author":"Nuzzi","year":"2021","journal-title":"Front Med"},{"key":"B11","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1001\/jamaophthalmol.2018.1934","article-title":"Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks","volume":"136","author":"Brown","year":"2018","journal-title":"JAMA Ophthalmol"},{"key":"B12","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1109\/TMI.2018.2863562","article-title":"Automated analysis for retinopathy of prematurity by deep neural networks","volume":"38","author":"Hu","year":"2019","journal-title":"IEEE Trans Med Imaging"},{"key":"B13","doi-asserted-by":"publisher","first-page":"e218758","DOI":"10.1001\/jamanetworkopen.2021.8758","article-title":"Automated explainable multidimensional deep learning platform of retinal images for retinopathy of prematurity screening","volume":"4","author":"Wang","year":"2021","journal-title":"JAMA Network Open"},{"key":"B14","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1167\/tvst.12.8.13","article-title":"Multicenter validation of deep learning algorithm ROP.AI for the automated diagnosis of plus disease in retinopathy of prematurity","volume":"12","author":"Bai","year":"2023","journal-title":"Transl Vis Sci Technol"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1771083","DOI":"10.3389\/fmed.2026.1771083","article-title":"Robust benchmarking of deeplabv3 hybrid models for multiclass fundus screening and referable-disease triage on the fives dataset","volume":"13","author":"Garcia-Atutxa","year":"2026","journal-title":"Front Med"},{"key":"B16","doi-asserted-by":"publisher","first-page":"e240300","DOI":"10.1148\/ryai.240300","article-title":"Checklist for artificial intelligence in medical imaging (CLAIM): 2024 update","volume":"6","author":"Tejani","year":"2024","journal-title":"Radiol Artif Intell"},{"key":"B17","author":"Goji\u0107","year":""},{"key":"B18","author":"Selvaraju","year":""},{"key":"B19","author":"Tan","year":""},{"key":"B20","author":"Huang","year":""},{"key":"B21","author":"He","year":""},{"key":"B22","author":"Howard","year":""},{"key":"B23","author":"Ronneberger","year":""},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1175\/1520-0493","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon Weath Rev"},{"key":"B25","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: The achilles heel of predictive analytics","volume":"17","author":"Van Calster","year":"2019","journal-title":"BMC Med"},{"key":"B26","doi-asserted-by":"publisher","first-page":"3283","DOI":"10.1038\/s41591-025-03953-8","article-title":"The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence","volume":"31","author":"Sounderajah","year":"2025","journal-title":"Nat Med"},{"key":"B27","author":"Ribeiro","year":""},{"key":"B28","doi-asserted-by":"publisher","first-page":"2274","DOI":"10.1109\/TPAMI.2012.120","article-title":"SLIC superpixels compared to state-of-the-art superpixel methods","volume":"34","author":"Achanta","year":"2012","journal-title":"IEEE Trans Pattern Anal Mach Intell"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1960317\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T04:54:06Z","timestamp":1790744046000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1960317\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,30]]},"references-count":28,"alternative-id":["10.3389\/fmed.2026.1960317"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1960317","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,30]]},"article-number":"1960317"}}