{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,6]],"date-time":"2026-11-06T23:52:12Z","timestamp":1794009132616,"version":"4.8.0"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T00:00:00Z","timestamp":1644969600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T00:00:00Z","timestamp":1644969600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Within the next 1.5 decades, 1 in 7 U.S. adults is anticipated to suffer from age-related macular degeneration (AMD), a degenerative retinal disease which leads to blindness if untreated. Optical coherence tomography angiography (OCTA) has become a prime technique for AMD diagnosis, specifically for late-stage neovascular (NV) AMD. Such technologies generate massive amounts of data, challenging to parse by experts alone, transforming artificial intelligence into a valuable partner. We describe a deep learning (DL) approach which achieves multi-class detection of non-AMD vs. non-neovascular (NNV) AMD vs. NV AMD from a combination of OCTA, OCT structure, 2D b-scan flow images, and high definition (HD) 5-line b-scan cubes; DL also detects ocular biomarkers indicative of AMD risk. Multimodal data were used as input to 2D-3D Convolutional Neural Networks (CNNs). Both for CNNs and experts, choroidal neovascularization and geographic atrophy were found to be important biomarkers for AMD. CNNs predict biomarkers with accuracy up to 90.2% (positive-predictive-value up to 75.8%). Just as experts rely on multimodal data to diagnose AMD, CNNs also performed best when trained on multiple inputs combined. Detection of AMD and its biomarkers from OCTA data via CNNs has tremendous potential to expedite screening of early and late-stage AMD patients.<\/jats:p>","DOI":"10.1038\/s41598-022-06273-w","type":"journal-article","created":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T06:03:04Z","timestamp":1644991384000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["A multimodal deep learning system to distinguish late stages of AMD and to compare expert vs. AI ocular biomarkers"],"prefix":"10.1038","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5589-8151","authenticated-orcid":false,"given":"Kaveri A.","family":"Thakoor","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7062-2508","authenticated-orcid":false,"given":"Jiaang","family":"Yao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darius","family":"Bordbar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Omar","family":"Moussa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weijie","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Sajda","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Royce W. S.","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,2,16]]},"reference":[{"issue":"1","key":"6273_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-019-47390-3","volume":"9","author":"S Saha","year":"2019","unstructured":"Saha, S. et al. Automated detection and classification of early AMD biomarkers using deep learning. Sci. Rep. 9(1), 1\u20139 (2019).","journal-title":"Sci. Rep."},{"key":"6273_CR2","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1016\/j.ophtha.2006.10.040","volume":"114","author":"R Klein","year":"2007","unstructured":"Klein, R. et al. Fifteen-year cumulative incidence of age-related macular degeneration: The beaver dam eye study. Ophthalmology 114, 253\u2013262 (2007).","journal-title":"Ophthalmology"},{"key":"6273_CR3","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1016\/j.ophtha.2007.03.008","volume":"115","author":"T Wong","year":"2008","unstructured":"Wong, T. et al. The natural history and prognosis of neovascular age-related macular degeneration: A systematic review of the literature and meta-analysis. Ophthalmology 115, 116\u2013126 (2008).","journal-title":"Ophthalmology"},{"key":"6273_CR4","doi-asserted-by":"publisher","first-page":"1419","DOI":"10.1056\/NEJMoa054481","volume":"355","author":"PJ Rosenfeld","year":"2006","unstructured":"Rosenfeld, P. J. et al. Ranibizumab for neovascular age-related macular degeneration. N. Engl. J. Med. 355, 1419\u20131431 (2006).","journal-title":"N. Engl. J. Med."},{"key":"6273_CR5","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1097\/ICU.0b013e32835f8bf8","volume":"24","author":"A Mehreen","year":"2013","unstructured":"Mehreen, A. & Duker, J. S. Optical coherence tomography\u2013current and future applications. Curr. Opin. Ophthalmol. 24, 213\u2013224 (2013).","journal-title":"Curr. Opin. Ophthalmol."},{"key":"6273_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.preteyeres.2018.07.004","volume":"67","author":"U Schmidt-Erfurth","year":"2018","unstructured":"Schmidt-Erfurth, U., Sadeghipour, A., Gerendas, B. S., Waldstein, S. M. & Bogunovic, H. Artificial intelligence in retina. Prog. Retinal Eye Res. 67, 1\u201329 (2018).","journal-title":"Prog. Retinal Eye Res."},{"key":"6273_CR7","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez-Velasco, A. & Lourdes, M.-V. A survey of machine learning approaches for age related macular degeneration diagnosis and prediction. in\u00a0Mexican International Conference on Artificial Intelligence, 257\u2013266. (Springer, 2017).","DOI":"10.1007\/978-3-030-02837-4_21"},{"key":"6273_CR8","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1016\/j.survophthal.2019.02.003","volume":"64","author":"E Pead","year":"2019","unstructured":"Pead, E. et al. Automated detection of age-related macular degeneration in color fundus photography: A systematic review. Surv. Ophthalmol. 64, 498\u2013511 (2019).","journal-title":"Surv. Ophthalmol."},{"key":"6273_CR9","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1016\/j.oret.2016.12.009","volume":"1","author":"CS Lee","year":"2017","unstructured":"Lee, C. S., Baughman, D. M. & Lee, A. Y. Deep learning is effective for classifying normal versus age-related macular degeneration oct images. Ophthalmol. Retina 1, 322\u2013327 (2017).","journal-title":"Ophthalmol. Retina"},{"key":"6273_CR10","doi-asserted-by":"crossref","unstructured":"An, G. et al. Deep learning classification models built with two-step transfer learning for age related macular degeneration diagnosis. in IEEE International Conference on Engineering in Medicine and Biology Society (EMBC), 2049\u20132052 (2019).","DOI":"10.1109\/EMBC.2019.8857468"},{"key":"6273_CR11","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1007\/s00417-017-3839-y","volume":"256","author":"P Prahs","year":"2018","unstructured":"Prahs, P. et al. Oct-based deep learning algorithm for the evaluation of treatment indication with anti-vascular endothelial growth factor medications. Graefes Arch. Clin. Exp. Ophthalmol. 256, 91\u201398 (2018).","journal-title":"Graefes Arch. Clin. Exp. Ophthalmol."},{"key":"6273_CR12","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.1016\/j.cell.2018.02.010","volume":"172","author":"DS Kermany","year":"2018","unstructured":"Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122\u20131131 (2018).","journal-title":"Cell"},{"key":"6273_CR13","doi-asserted-by":"publisher","first-page":"927","DOI":"10.1364\/BOE.379977","volume":"11","author":"J Wang","year":"2020","unstructured":"Wang, J. et al. Automated diagnosis and segmentation of choroidal neovascularization in oct angiography using deep learning. Biomed. Opt. Express 11, 927\u2013944 (2020).","journal-title":"Biomed. Opt. Express"},{"key":"6273_CR14","doi-asserted-by":"crossref","first-page":"7493419","DOI":"10.1155\/2020\/7493419","volume":"2020","author":"E Vaghefi","year":"2020","unstructured":"Vaghefi, E., Hill, S., Kersten, H. M. & Squirrell, D. Multimodal retinal image analysis via deep learning for the diagnosis of intermediate dry age-related macular degeneration: A feasibility study. J. Ophthalmol. 2020, 7493419 (2020).","journal-title":"J. Ophthalmol."},{"key":"6273_CR15","doi-asserted-by":"crossref","unstructured":"Jin, K. et al. Multimodal deep learning with feature level fusion for identification of choroidal neovascularization activity in age\u2010related macular degeneration.\u00a0Acta Ophthalmol. (2021).","DOI":"10.1111\/aos.14928"},{"issue":"2","key":"6273_CR16","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1016\/j.ajo.2012.07.030","volume":"155","author":"Y Zheng","year":"2013","unstructured":"Zheng, Y. et al. Computerized assessment of intraretinal and subretinal fluid regions in spectral-domain optical coherence tomography images of the retina. Am. J. Ophthalmol. 155(2), 277\u2013286 (2013).","journal-title":"Am. J. Ophthalmol."},{"issue":"11","key":"6273_CR17","doi-asserted-by":"publisher","first-page":"7093","DOI":"10.1167\/iovs.14-14918","volume":"55","author":"L de Sisternes","year":"2014","unstructured":"de Sisternes, L., Simon, N., Tibshirani, R., Leng, T. & Rubin, D. L. Quantitative SD-OCT imaging biomarkers as indicators of age-related macular degeneration progression. Invest. Ophthalmol. Vis. Sci. 55(11), 7093\u20137103 (2014).","journal-title":"Invest. Ophthalmol. Vis. Sci."},{"key":"6273_CR18","doi-asserted-by":"crossref","unstructured":"Thakoor, K. et al. Hybrid 3D-2D deep learning for detection of neovascular age-related macular degeneration using optical coherence tomography b-scans and angiography volumes. in IEEE International Symposium on Biomedical Imaging (ISBI), 1600\u20131604 (2021).","DOI":"10.1109\/ISBI48211.2021.9434111"},{"key":"6273_CR19","doi-asserted-by":"crossref","unstructured":"Thakoor, K. et al. A hybrid deep learning system to distinguish late stages of AMD and to compare expert vs. machine AMD risk features, In Association for Research in Vision and Ophthalmology Annual Meeting (2021).","DOI":"10.1038\/s41598-022-06273-w"},{"issue":"12","key":"6273_CR20","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1016\/j.ophtha.2019.07.024","volume":"126","author":"S Phene","year":"2019","unstructured":"Phene, S. et al. Deep learning and glaucoma specialists: The relative importance of optic disc features to predict glaucoma referral in fundus photographs. Ophthalmology 126(12), 1627\u20131639 (2019).","journal-title":"Ophthalmology"},{"issue":"1","key":"6273_CR21","doi-asserted-by":"publisher","first-page":"P1","DOI":"10.1016\/j.ophtha.2019.09.024","volume":"127","author":"CJ Flaxel","year":"2020","unstructured":"Flaxel, C. J. et al. Age-related macular degeneration preferred practice pattern\u00ae. Ophthalmology 127(1), P1\u2013P65 (2020).","journal-title":"Ophthalmology"},{"issue":"5","key":"6273_CR22","doi-asserted-by":"publisher","first-page":"261","DOI":"10.3390\/diagnostics10050261","volume":"10","author":"TY Heo","year":"2020","unstructured":"Heo, T. Y. et al. Development of a deep-learning-based artificial intelligence tool for differential diagnosis between dry and neovascular age-related macular degeneration. Diagnostics 10(5), 261 (2020).","journal-title":"Diagnostics"},{"issue":"9","key":"6273_CR23","doi-asserted-by":"publisher","first-page":"1664","DOI":"10.1097\/IAE.0000000000002220","volume":"39","author":"V Gualino","year":"2019","unstructured":"Gualino, V. et al. Optical coherence tomography, fluorescein angiography, and diagnosis of choroidal neovascularization in age-related macular degeneration. Retina 39(9), 1664 (2019).","journal-title":"Retina"},{"key":"6273_CR24","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R. et al. Grad-cam: Visual explanations from deep networks via gradient-based localization. in\u00a0Proceedings of the IEEE International Conference on Computer Vision, 618\u2013626 (2017).","DOI":"10.1109\/ICCV.2017.74"},{"key":"6273_CR25","unstructured":"Ben-Baruch, E. et al. Asymmetric loss for multi-label classification. http:\/\/arxiv.org\/abs\/2009.14119\u00a0(2020)."},{"key":"6273_CR26","unstructured":"Cheung, C.Y. et al. A multi-task deep-learning system to classify diabetic macular edema for different optical coherence tomography devices: A multi-center analysis. in Association for Research in Vision and Ophthalmology Annual Meeting (2021)."},{"key":"6273_CR27","doi-asserted-by":"crossref","unstructured":"Mooga, A., Sethuraman, R., & Sheet, D. Zero-shot adaptation to simulate 3D ultrasound volume by learning a multilinear separable 2D convolutional neural network. In IEEE International Symposium on Biomedical Imaging (ISBI), 1780\u20131783 (2020).","DOI":"10.1109\/ISBI45749.2020.9098479"},{"key":"6273_CR28","doi-asserted-by":"publisher","first-page":"104129","DOI":"10.1016\/j.compbiomed.2020.104129","volume":"128","author":"M Salvi","year":"2021","unstructured":"Salvi, M., Acharya, U. R., Molinari, F. & Meiburger, K. M. The impact of pre-and post-image processing techniques on deep learning frameworks: A comprehensive review for digital pathology image analysis. Comput. Biol. Med. 128, 104129 (2021).","journal-title":"Comput. Biol. Med."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-06273-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-06273-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-06273-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T18:13:52Z","timestamp":1669313632000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-022-06273-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,16]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["6273"],"URL":"https:\/\/doi.org\/10.1038\/s41598-022-06273-w","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,16]]},"assertion":[{"value":"25 June 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 January 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 February 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Royce Chen consults for Carl Zeiss Meditec, Inc. No other conflicts of interest exist for the remaining authors.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"2585"}}