{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T05:06:44Z","timestamp":1791522404836,"version":"4.3.4"},"reference-count":29,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAD"],"published-print":{"date-parts":[[2024,1,2]]},"abstract":"<jats:p>Background: Prognosis of future risk of dementia from neuroimaging and cognitive data is important for optimizing clinical management for patients at early stage of Alzheimer\u2019s disease (AD). However, existing studies lack an efficient way to integrate longitudinal information from both modalities to improve prognosis performance. Objective: In this study, we aim to develop and evaluate an explainable deep learning-based framework to predict mild cognitive impairment (MCI) to AD conversion within four years using longitudinal whole-brain 3D MRI and neurocognitive tests. Methods: We proposed a two-stage framework that first uses a 3D convolutional neural network to extract single-timepoint MRI-based AD-related latent features, followed by multi-modal longitudinal feature concatenation and a 1D convolutional neural network to predict the risk of future dementia onset in four years. Results: The proposed deep learning framework showed promising to predict MCI to AD conversion within 4 years using longitudinal whole-brain 3D MRI and cognitive data without extracting regional brain volumes or cortical thickness, reaching a balanced accuracy of 0.834, significantly improved from models trained from single timepoint or single modality. The post hoc model explainability revealed heatmap indicating regions that are important for predicting future risk of AD. Conclusions: The proposed framework sets the stage for future studies for using multi-modal longitudinal data to achieve optimal prediction for prognosis of AD onset, leading to better management of the diseases, thereby improving the quality of life.<\/jats:p>","DOI":"10.3233\/jad-230893","type":"journal-article","created":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T12:21:52Z","timestamp":1702988512000},"page":"459-469","source":"Crossref","is-referenced-by-count":15,"title":["Predicting Four-Year\u2019s Alzheimer\u2019s Disease Onset Using Longitudinal Neurocognitive Tests and MRI Data Using Explainable Deep Convolutional Neural Networks"],"prefix":"10.1177","volume":"97","author":[{"given":"Rohan","family":"Bapat","sequence":"first","affiliation":[{"name":"Department of Radiology, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Da","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine Section of Gerontology and Geriatric Medicine, Wake Forest University School of Medicine, Winston-Salam, NC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim Q.","family":"Duong","sequence":"additional","affiliation":[{"name":"Department of Radiology, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","reference":[{"key":"10.3233\/JAD-230893_ref1","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1016\/j.jalz.2011.03.005","article-title":"The diagnosis of dementia due to Alzheimer\u2019s disease: Recommendations from the National Institute on Aging-Alzheimer\u2019s Association workgroups on diagnostic guidelines for Alzheimer\u2019s disease","volume":"7","author":"McKhann","year":"2011","journal-title":"Alzheimers Dement"},{"key":"10.3233\/JAD-230893_ref2","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1001\/archneur.56.3.303","article-title":"Mild cognitive impairment: Clinical characterization and outcome","volume":"56","author":"Petersen","year":"1999","journal-title":"Arch Neurol"},{"key":"10.3233\/JAD-230893_ref3","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1097\/JGP.0b013e31819431d5","article-title":"Quantification of five neuropsychological approaches to defining mild cognitive impairment","volume":"17","author":"Jak","year":"2009","journal-title":"Am J Geriatr Psychiatry"},{"key":"10.3233\/JAD-230893_ref4","first-page":"771","article-title":"Alzheimer disease: Pharmacologic and nonpharmacologic therapies for cognitive and functional symptoms","volume":"95","author":"Epperly","year":"2017","journal-title":"Am Fam Physician"},{"key":"10.3233\/JAD-230893_ref5","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1016\/j.media.2016.06.032","article-title":"Machine learning approaches in medical image analysis: From detection to diagnosis","volume":"33","author":"de Bruijne","year":"2016","journal-title":"Med Image Anal"},{"key":"10.3233\/JAD-230893_ref6","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1148\/rg.2017160130","article-title":"Machine learning for medical imaging","volume":"37","author":"Erickson","year":"2017","journal-title":"Radiographics"},{"key":"10.3233\/JAD-230893_ref7","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc IEEE"},{"key":"10.3233\/JAD-230893_ref10","doi-asserted-by":"crossref","first-page":"880","DOI":"10.1109\/TPAMI.2018.2889096","article-title":"Hierarchical fully convolutional network for joint atrophy localization and Alzheimer\u2019s disease diagnosis using structural MRI","volume":"42","author":"Lian","year":"2018","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10.3233\/JAD-230893_ref11","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.media.2017.10.005","article-title":"Landmark-based deep multi-instance learning for brain disease diagnosis","volume":"43","author":"Liu","year":"2018","journal-title":"Med Image Anal"},{"key":"10.3233\/JAD-230893_ref12","doi-asserted-by":"crossref","first-page":"101694","DOI":"10.1016\/j.media.2020.101694","article-title":"Convolutional neural networks for classification of Alzheimer\u2019s disease: Overview and reproducible evaluation","volume":"63","author":"Wen","year":"2020","journal-title":"Med Image Anal"},{"key":"10.3233\/JAD-230893_ref13","doi-asserted-by":"crossref","first-page":"777","DOI":"10.3389\/fnins.2018.00777","article-title":"Convolutional neural networks-based MRI image analysis for the Alzheimer\u2019s disease prediction from mild cognitive impairment","volume":"12","author":"Lin","year":"2018","journal-title":"Front Neurosci"},{"key":"10.3233\/JAD-230893_ref15","doi-asserted-by":"crossref","first-page":"101645","DOI":"10.1016\/j.nicl.2018.101645","article-title":"Automated classification of Alzheimer\u2019s disease and mild cognitive impairment using a single MRI and deep neural networks","volume":"21","author":"Basaia","year":"2019","journal-title":"Neuroimage Clin"},{"key":"10.3233\/JAD-230893_ref16","doi-asserted-by":"crossref","first-page":"35","DOI":"10.3389\/fninf.2018.00035","article-title":"Classification of Alzheimer\u2019s disease by combination of convolutional and recurrent neural networks using FDG-PET images","volume":"12","author":"Liu","year":"2018","journal-title":"Front Neuroinform"},{"key":"10.3233\/JAD-230893_ref17","doi-asserted-by":"crossref","first-page":"217","DOI":"10.3390\/brainsci9090217","article-title":"A deep learning approach for diagnosis of mild cognitive impairment based on MRI images","volume":"9","author":"Gorji","year":"2019","journal-title":"Brain Sci"},{"key":"10.3233\/JAD-230893_ref19","doi-asserted-by":"crossref","first-page":"e560","DOI":"10.7717\/peerj-cs.560","article-title":"Deep learning prediction of mild cognitive impairment conversion to Alzheimer\u2019s disease at 3 years after diagnosis using longitudinal and whole-brain 3D MRI","volume":"7","author":"Ocasio","year":"2021","journal-title":"Peer J Comput Sci"},{"key":"10.3233\/JAD-230893_ref20","doi-asserted-by":"crossref","first-page":"106310","DOI":"10.1016\/j.nbd.2023.106310","article-title":"Deep learning combining FDG-PET and neurocognitive data accurately predicts MCI conversion to Alzheimer\u2019s dementia 3-year post MCI diagnosis","volume":"187","author":"Cao","year":"2023","journal-title":"Neurobiol Dis"},{"key":"10.3233\/JAD-230893_ref21","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.mri.2021.02.001","article-title":"A 3D densely connected convolution neural network with connection-wise attention mechanism for Alzheimer\u2019s disease classification","volume":"78","author":"Zhang","year":"2021","journal-title":"Magn Reson Imaging"},{"key":"10.3233\/JAD-230893_ref22","doi-asserted-by":"crossref","first-page":"756","DOI":"10.3389\/fneur.2019.00756","article-title":"A novel ensemble-based machine learning algorithm to predict the conversion from mild cognitive impairment to Alzheimer\u2019s disease using socio-demographic characteristics, clinical information, and neuropsychological measures","volume":"10","author":"Grassi","year":"2019","journal-title":"Front Neurol"},{"key":"10.3233\/JAD-230893_ref23","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.neuroimage.2019.01.031","article-title":"A parameter-efficient deep learning approach to predict conversion from mild cognitive impairment to Alzheimer\u2019s disease","volume":"189","author":"Spasov","year":"2019","journal-title":"Neuroimage"},{"key":"10.3233\/JAD-230893_ref24","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.neurobiolaging.2020.12.005","article-title":"Transfer learning for predicting conversion from mild cognitive impairment to dementia of Alzheimer\u2019s type based on a three-dimensional convolutional neural network","volume":"99","author":"Bae","year":"2021","journal-title":"Neurobiol Aging"},{"key":"10.3233\/JAD-230893_ref25","doi-asserted-by":"crossref","first-page":"509","DOI":"10.3389\/fnins.2019.00509","article-title":"Diagnosis of Alzheimer\u2019s disease via multi-modality 3D convolutional neural network","volume":"13","author":"Huang","year":"2019","journal-title":"Front Neurosci"},{"key":"10.3233\/JAD-230893_ref28","doi-asserted-by":"crossref","first-page":"e1006376","DOI":"10.1371\/journal.pcbi.1006376","article-title":"Modeling and prediction of clinical symptom trajectories in Alzheimer\u2019s disease using longitudinal data","volume":"14","author":"Bhagwat","year":"2018","journal-title":"PLoS Comput Biol"},{"key":"10.3233\/JAD-230893_ref30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compmedimag.2019.01.005","article-title":"RNN-based longitudinal analysis for diagnosis of Alzheimer\u2019s disease","volume":"73","author":"Cui","year":"2019","journal-title":"Comput Med Imaging Graph"},{"key":"10.3233\/JAD-230893_ref31","doi-asserted-by":"crossref","first-page":"117203","DOI":"10.1016\/j.neuroimage.2020.117203","article-title":"Predicting Alzheimer\u2019s disease progression using deep recurrent neural networks","volume":"222","author":"Nguyen","year":"2020","journal-title":"Neuroimage"},{"key":"10.3233\/JAD-230893_ref32","doi-asserted-by":"crossref","first-page":"1475","DOI":"10.3390\/s22041475","article-title":"A long short-term memory biomarker-based prediction framework for Alzheimer\u2019s disease","volume":"22","author":"Aqeel","year":"2022","journal-title":"Sensors"},{"key":"10.3233\/JAD-230893_ref33","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1023\/A:1022318921994","article-title":"A meta-analysis of structural and functional brain imaging in dementia of the Alzheimer\u2019s type: A neuroimaging profile","volume":"13","author":"Zakzanis","year":"2003","journal-title":"Neuropsychol Rev"},{"key":"10.3233\/JAD-230893_ref34","doi-asserted-by":"crossref","first-page":"1415","DOI":"10.1016\/j.neuroimage.2008.10.031","article-title":"Baseline and longitudinal patterns of brain atrophy in MCI patients, and their use in prediction of short-term conversion to AD: Results from ADNI","volume":"44","author":"Misra","year":"2009","journal-title":"NeuroImage"},{"key":"10.3233\/JAD-230893_ref35","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1186\/s13195-021-00924-2","article-title":"Improving 3D convolutional neural network comprehensibility via interactive visualization of relevance maps: Evaluation in Alzheimer\u2019s disease","volume":"13","author":"Dyrba","year":"2021","journal-title":"Alzheimers Res Ther"},{"key":"10.3233\/JAD-230893_ref37","doi-asserted-by":"crossref","first-page":"fcaa057","DOI":"10.1093\/braincomms\/fcaa057","article-title":"Prediction of conversion to Alzheimer\u2019s disease using deep survival analysis of MRI images","volume":"2","author":"Nakagawa","year":"2020","journal-title":"Brain Commun"}],"container-title":["Journal of Alzheimer's Disease"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/JAD-230893","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T10:09:03Z","timestamp":1777370943000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/JAD-230893"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,2]]},"references-count":29,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/jad-230893","relation":{},"ISSN":["1387-2877","1875-8908"],"issn-type":[{"value":"1387-2877","type":"print"},{"value":"1875-8908","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,2]]}}}