{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T23:10:26Z","timestamp":1790723426345,"version":"4.1.0"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T00:00:00Z","timestamp":1648080000000},"content-version":"vor","delay-in-days":23,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["EY030488"],"award-info":[{"award-number":["EY030488"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1845666"],"award-info":[{"award-number":["1845666"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1852606"],"award-info":[{"award-number":["1852606"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1838627"],"award-info":[{"award-number":["1838627"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1837956"],"award-info":[{"award-number":["1837956"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1956002"],"award-info":[{"award-number":["1956002"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["040588"],"award-info":[{"award-number":["040588"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100016300","name":"University of Pittsburgh Center for Research Computing","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100016300","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,3,24]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Age-related macular degeneration (AMD) is the principal cause of blindness in developed countries, and its prevalence will increase to 288 million people in 2040. Therefore, automated grading and prediction methods can be highly beneficial for recognizing susceptible subjects to late-AMD and enabling clinicians to start preventive actions for them. Clinically, AMD severity is quantified by Color Fundus Photographs (CFP) of the retina, and many machine-learning-based methods are proposed for grading AMD severity. However, few models were developed to predict the longitudinal progression status, i.e. predicting future late-AMD risk based on the current CFP, which is more clinically interesting. In this paper, we propose a new deep-learning-based classification model (LONGL-Net) that can simultaneously grade the current CFP and predict the longitudinal outcome, i.e. whether the subject will be in late-AMD in the future time-point. We design a new temporal-correlation-structure-guided Generative Adversarial Network model that learns the interrelations of temporal changes in CFPs in consecutive time-points and provides interpretability for the classifier's decisions by forecasting AMD symptoms in the future CFPs. We used about 30,000 CFP images from 4,628 participants in the Age-Related Eye Disease Study. Our classifier showed average 0.905 (95% CI: 0.886\u20130.922) AUC and 0.762 (95% CI: 0.733\u20130.792) accuracy on the 3-class classification problem of simultaneously grading current time-point's AMD condition and predicting late AMD progression of subjects in the future time-point. We further validated our model on the UK Biobank dataset, where our model showed average 0.905 accuracy and 0.797 sensitivity in grading 300 CFP images.<\/jats:p>","DOI":"10.1093\/pnasnexus\/pgab003","type":"journal-article","created":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T18:24:26Z","timestamp":1648232666000},"source":"Crossref","is-referenced-by-count":16,"title":["LONGL-Net: temporal correlation structure guided deep learning model to predict longitudinal age-related macular degeneration severity"],"prefix":"10.1093","volume":"1","author":[{"given":"Alireza","family":"Ganjdanesh","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh , Pittsburgh, PA 15261 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jipeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh , Pittsburgh, PA 15213 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0999-9802","authenticated-orcid":false,"given":"Emily Y","family":"Chew","sequence":"additional","affiliation":[{"name":"Division of Epidemiology and Clinical Applications, National Eye Institute, National Institutes of Health , Bethesda, MD 20892 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1352-1000","authenticated-orcid":false,"given":"Ying","family":"Ding","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh , Pittsburgh, PA 15213 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heng","family":"Huang","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Swanson School of Engineering, University of Pittsburgh , Pittsburgh, PA 15261 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7196-8703","authenticated-orcid":false,"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, Graduate School of Public Health, University of Pittsburgh , Pittsburgh, PA 15213 , USA"},{"name":"Division of Pulmonary Medicine, Department of Pediatrics, UPMC Children's Hospital of Pittsburgh, University of Pittsburgh , Pittsburgh, PA 15219 , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,3,19]]},"reference":[{"issue":"5","key":"2022111622065697100_bib1","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0039-6257(05)80092-X","article-title":"An international classification and grading system for age-related maculopathy and age-related macular degeneration","volume":"39","author":"Bird","year":"1995","journal-title":"Surv Ophthalmol"},{"key":"2022111622065697100_bib2","volume-title":"Computational retinal image analysis: tools, applications and perspectives","author":"Trucco","year":"2019"},{"issue":"4","key":"2022111622065697100_bib3","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1001\/archopht.122.4.477","article-title":"Causes and prevalence of visual impairment among adults in the United States","volume":"122","author":"Congdon","year":"2004","journal-title":"Arch Ophthalmol"},{"issue":"4","key":"2022111622065697100_bib4","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1038\/eye.2015.288","article-title":"Leading causes of certifiable visual loss in England and Wales during the year ending 31 March 2013","volume":"30","author":"Quartilho","year":"2016","journal-title":"Eye"},{"key":"2022111622065697100_bib5","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1146\/annurev-genom-090413-025610","article-title":"Age-related macular degeneration: genetics and biology coming together","volume":"15","author":"Fritsche","year":"2014","journal-title":"Annu Rev Genomics Hum Genet"},{"key":"2022111622065697100_bib6","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1016\/j.ejpb.2015.04.039","article-title":"The genetics of age-related macular degeneration (AMD)\u2013novel targets for designing treatment options?","volume":"95","author":"Grassmann","year":"2015","journal-title":"Eur J Pharm Biopharm"},{"issue":"2","key":"2022111622065697100_bib7","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1111\/cge.12206","article-title":"Age-related macular degeneration\u2014clinical review and genetics update","volume":"84","author":"Ratnapriya","year":"2013","journal-title":"Clin Genet"},{"issue":"15","key":"2022111622065697100_bib8","doi-asserted-by":"crossref","first-page":"2257","DOI":"10.1093\/hmg\/ddi230","article-title":"Meta-analysis of genome scans of age-related macular degeneration","volume":"14","author":"Fisher","year":"2005","journal-title":"Hum Mol Genet"},{"issue":"2","key":"2022111622065697100_bib9","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1038\/ng.3448","article-title":"A large genome-wide association study of age-related macular degeneration highlights contributions of rare and common variants","volume":"48","author":"Fritsche","year":"2016","journal-title":"Nat Genet"},{"issue":"2","key":"2022111622065697100_bib10","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.ophtha.2010.06.040","article-title":"Complement factor H Y402H variant and risk of age-related macular degeneration in Asians: a systematic review and meta-analysis","volume":"118","author":"Kondo","year":"2011","journal-title":"Ophthalmology"},{"issue":"10","key":"2022111622065697100_bib11","doi-asserted-by":"crossref","first-page":"6839","DOI":"10.1167\/iovs.14-14246","article-title":"Genetic determinants of age-related macular degeneration in diverse populations from the PAGE study","volume":"55","author":"Restrepo","year":"2014","journal-title":"Invest Ophthalmol Vis Sci"},{"issue":"1","key":"2022111622065697100_bib12","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1001\/archopthalmol.2011.1370","article-title":"Population differences in genetic risk for age-related macular degeneration and implications for genetic testing","volume":"130","author":"Spencer","year":"2012","journal-title":"Arch Ophthalmol"},{"issue":"2","key":"2022111622065697100_bib13","doi-asserted-by":"crossref","first-page":"e106","DOI":"10.1016\/S2214-109X(13)70145-1","article-title":"Global prevalence of age-related macular degeneration and disease burden projection for 2020 and 2040: a systematic review and meta-analysis","volume":"2","author":"Wong","year":"2014","journal-title":"Lancet Glob Health"},{"issue":"3","key":"2022111622065697100_bib14","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1016\/j.ajo.2006.11.041","article-title":"Progression of geographic atrophy and impact of fundus autofluorescence patterns in age-related macular degeneration","volume":"143","author":"Holz","year":"2007","journal-title":"Am J Ophthalmol"},{"issue":"2","key":"2022111622065697100_bib15","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1038\/s42256-020-0154-9","article-title":"Deep-learning-based prediction of late age-related macular degeneration progression","volume":"2","author":"Yan","year":"2020","journal-title":"Nat Mach Intell"},{"issue":"10","key":"2022111622065697100_bib16","doi-asserted-by":"crossref","first-page":"1417","DOI":"10.1001\/archopht.119.10.1417","article-title":"A randomized, placebo-controlled, clinical trial of high-dose supplementation with vitamins C and E, beta carotene, and zinc for age-related macular degeneration and vision loss: AREDS report no 8","volume":"119","author":"Group","year":"2001","journal-title":"Arch Ophthalmol"},{"issue":"11","key":"2022111622065697100_bib17","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1016\/j.ophtha.2019.06.005","article-title":"A deep learning approach for automated detection of geographic atrophy from color fundus photographs","volume":"126","author":"Keenan","year":"2019","journal-title":"Ophthalmology"},{"issue":"11","key":"2022111622065697100_bib18","doi-asserted-by":"crossref","first-page":"1170","DOI":"10.1001\/jamaophthalmol.2017.3782","article-title":"Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks","volume":"135","author":"Burlina","year":"2017","journal-title":"JAMA Ophthalmol"},{"key":"2022111622065697100_bib19","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-59710-8_62","article-title":"Predicting potential propensity of adolescents to drugs via new semi-supervised deep ordinal regression model","volume-title":"Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Ganjdanesh","year":"2020"},{"issue":"9","key":"2022111622065697100_bib20","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1016\/j.ophtha.2018.02.037","article-title":"A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography","volume":"125","author":"Grassmann","year":"2018","journal-title":"Ophthalmology"},{"issue":"31","key":"2022111622065697100_bib21","doi-asserted-by":"crossref","first-page":"18412","DOI":"10.1073\/pnas.2001906117","article-title":"A single-cell Raman-based platform to identify developmental stages of human pluripotent stem cell-derived neurons","volume":"117","author":"Hsu","year":"2020","journal-title":"Proc Natl Acad Sci"},{"issue":"39","key":"2022111622065697100_bib22","doi-asserted-by":"crossref","first-page":"e2105070118","DOI":"10.1073\/pnas.2105070118","article-title":"Deep learning identifies synergistic drug combinations for treating COVID-19","volume":"118","author":"Jin","year":"2021","journal-title":"Proc Natl Acad Sci"},{"issue":"13","key":"2022111622065697100_bib23","doi-asserted-by":"crossref","first-page":"E2970","DOI":"10.1073\/pnas.1717139115","article-title":"Predicting cancer outcomes from histology and genomics using convolutional networks","volume":"115","author":"Mobadersany","year":"2018","journal-title":"Proc Natl Acad Sci"},{"issue":"4","key":"2022111622065697100_bib24","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1016\/j.ophtha.2018.11.015","article-title":"DeepSeeNet: a deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs","volume":"126","author":"Peng","year":"2019","journal-title":"Ophthalmology"},{"issue":"1","key":"2022111622065697100_bib25","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1073\/pnas.1805256116","article-title":"Data-driven supervised learning of a viral protease specificity landscape from deep sequencing and molecular simulations","volume":"116","author":"Pethe","year":"2019","journal-title":"Proc Natl Acad Sci"},{"key":"2022111622065697100_bib26","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-00931-1_51","article-title":"Temporal correlation structure learning for MCI conversion prediction","volume-title":"Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Wang","year":"2018"},{"issue":"6","key":"2022111622065697100_bib27","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1016\/S0197-2456(99)00031-8","article-title":"The age-related eye disease study (AREDS): design implications AREDS report no. 1","volume":"20","author":"Group","year":"1999","journal-title":"Control Clin Trials"},{"issue":"5","key":"2022111622065697100_bib28","doi-asserted-by":"crossref","first-page":"668","DOI":"10.1016\/S0002-9394(01)01218-1","article-title":"The age-related eye disease study system for classifying age-related macular degeneration from stereoscopic color fundus photographs: the age-related eye disease study report number 6","volume":"132","author":"Group","year":"2001","journal-title":"Am J Ophthalmol"},{"issue":"3","key":"2022111622065697100_bib29","doi-asserted-by":"crossref","first-page":"e1001779","DOI":"10.1371\/journal.pmed.1001779","article-title":"UK biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age","volume":"12","author":"Sudlow","year":"2015","journal-title":"PLoS Med"},{"key":"2022111622065697100_bib30","doi-asserted-by":"crossref","DOI":"10.1109\/ISBI.2016.7493240","article-title":"Detection of age-related macular degeneration via deep learning","volume-title":"Proceedings of the 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)","author":"Burlina","year":"2016"},{"issue":"12","key":"2022111622065697100_bib31","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1001\/jamaophthalmol.2018.4118","article-title":"Use of deep learning for detailed severity characterization and estimation of 5-year risk among patients with age-related macular degeneration","volume":"136","author":"Burlina","year":"2018","journal-title":"JAMA Ophthalmol"},{"issue":"9","key":"2022111622065697100_bib32","doi-asserted-by":"crossref","first-page":"1410","DOI":"10.1016\/j.ophtha.2018.02.037","article-title":"A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography","volume":"125","author":"Grassmann","year":"2018","journal-title":"Ophthalmology"},{"issue":"4","key":"2022111622065697100_bib33","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1016\/j.ophtha.2018.11.015","article-title":"DeepSeeNet: a deep learning model for automated classification of patient-based age-related macular degeneration severity from color fundus photographs","volume":"126","author":"Peng","year":"2019","journal-title":"Ophthalmology"},{"key":"2022111622065697100_bib34","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1038\/s41746-020-00317-z","article-title":"Predicting risk of late age-related macular degeneration using deep learning","volume":"3","author":"Peng","year":"2020","journal-title":"NPJ Digit Med"},{"issue":"1","key":"2022111622065697100_bib35","doi-asserted-by":"crossref","first-page":"e000569","DOI":"10.1136\/bmjophth-2020-000569","article-title":"Development and validation of a novel prognostic model for predicting AMD progression using longitudinal fundus images","volume":"5","author":"Bridge","year":"2020","journal-title":"BMJ Open Ophthalmol"},{"key":"2022111622065697100_bib36","doi-asserted-by":"crossref","DOI":"10.3115\/v1\/D14-1179","article-title":"Learning phrase representations using RNN encoder-decoder for statistical machine translation","author":"Cho","year":"2014","journal-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMN)"},{"issue":"8","key":"2022111622065697100_bib37","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Schmidhuber","year":"1997","journal-title":"Neural Comput"},{"key":"2022111622065697100_bib38","article-title":"Generative adversarial networks","author":"Goodfellow","year":"2014","journal-title":"Proceedings of the Advances in Neural Information Processing Systems 27 (NIPS 2014)"},{"issue":"1","key":"2022111622065697100_bib39","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J Mach Learn Res"},{"key":"2022111622065697100_bib40","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2017.632","article-title":"Image-to-image translation with conditional adversarial networks","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Isola","year":"2017"},{"key":"2022111622065697100_bib41","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-46487-9_43","article-title":"Precomputed real-time texture synthesis with markovian generative adversarial networks","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Li","year":"2016"},{"key":"2022111622065697100_bib42","article-title":"Pytorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019","journal-title":"Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019)"},{"key":"2022111622065697100_bib43","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"Proceedings of the 3rd International Conference for Learning Representations"},{"key":"2022111622065697100_bib44","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.308","article-title":"Rethinking the inception architecture for computer vision","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Szegedy","year":"2016"},{"key":"2022111622065697100_bib45","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"issue":"3","key":"2022111622065697100_bib46","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int J Comput Vis"},{"key":"2022111622065697100_bib47","first-page":"2825","article-title":"Scikit-learn: machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"issue":"2","key":"2022111622065697100_bib48","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1023\/A:1010920819831","article-title":"A simple generalisation of the area under the ROC curve for multiple class classification problems","volume":"45","author":"Hand","year":"2001","journal-title":"Mach Learn"},{"key":"2022111622065697100_bib49","volume-title":"U-Net: convolutional networks for biomedical image segmentation","author":"Ronneberger","year":"2015"},{"key":"2022111622065697100_bib50","article-title":"Improved techniques for training GANs","author":"Salimans","year":"2016","journal-title":"Proceedings of the 30th Conference on Neural Information Processing Systems (NIPS 2016)"}],"container-title":["PNAS Nexus"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/pnasnexus\/article-pdf\/1\/1\/pgab003\/47087227\/pgab003.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/pnasnexus\/article-pdf\/1\/1\/pgab003\/47087227\/pgab003.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,16]],"date-time":"2022-11-16T23:12:21Z","timestamp":1668640341000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/pnasnexus\/article\/doi\/10.1093\/pnasnexus\/pgab003\/6550868"}},"subtitle":[],"editor":[{"given":"Sandro","family":"Galea","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2022,3]]},"references-count":50,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2022,3,24]]}},"URL":"https:\/\/doi.org\/10.1093\/pnasnexus\/pgab003","relation":{},"ISSN":["2752-6542"],"issn-type":[{"value":"2752-6542","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,3]]},"published":{"date-parts":[[2022,3]]},"article-number":"pgab003"}}