{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,10]],"date-time":"2026-05-10T06:07:07Z","timestamp":1778393227757,"version":"3.51.4"},"reference-count":63,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T00:00:00Z","timestamp":1726790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>Machine Learning (ML) has emerged as a promising approach in healthcare, outperforming traditional statistical techniques. However, to establish ML as a reliable tool in clinical practice, adherence to best practices in <jats:italic>data handling<\/jats:italic>, and <jats:italic>modeling design and assessment<\/jats:italic> is crucial. In this work, we summarize and strictly adhere to such practices to ensure reproducible and reliable ML. Specifically, we focus on Alzheimer's Disease (AD) detection, a challenging problem in healthcare. Additionally, we investigate the impact of modeling choices, including different data augmentation techniques and model complexity, on overall performance.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>We utilize Magnetic Resonance Imaging (MRI) data from the ADNI corpus to address a binary classification problem using 3D Convolutional Neural Networks (CNNs). Data processing and modeling are specifically tailored to address data scarcity and minimize computational overhead. Within this framework, we train 15 predictive models, considering three different data augmentation strategies and five distinct 3D CNN architectures with varying convolutional layers counts. The augmentation strategies involve affine transformations, such as <jats:italic>zoom, shift<\/jats:italic>, and <jats:italic>rotation<\/jats:italic>, applied either concurrently or separately.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The combined effect of data augmentation and model complexity results in up to 10% variation in prediction accuracy. Notably, when affine transformation are applied separately, the model achieves higher accuracy, regardless the chosen architecture. Across all strategies, the model accuracy exhibits a concave behavior as the number of convolutional layers increases, peaking at an intermediate value. The best model reaches excellent performance both on the internal and additional external testing set.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussions<\/jats:title><jats:p>Our work underscores the critical importance of adhering to rigorous experimental practices in the field of ML applied to healthcare. The results clearly demonstrate how data augmentation and model depth\u2014often overlooked factors\u2013 can dramatically impact final performance if not thoroughly investigated. This highlights both the necessity of exploring neglected modeling aspects and the need to comprehensively report all modeling choices to ensure reproducibility and facilitate meaningful comparisons across studies.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fncom.2024.1360095","type":"journal-article","created":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T14:53:23Z","timestamp":1726844003000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Deep learning-based Alzheimer's disease detection: reproducibility and the effect of modeling choices"],"prefix":"10.3389","volume":"18","author":[{"given":"Rosanna","family":"Turrisi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessandro","family":"Verri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Annalisa","family":"Barla","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2024,9,20]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/978-3-030-17971-7_14","article-title":"\u201cAutomatic detection of alzheimer disease based on histogram and random forest,\u201d","volume-title":"CMBEBIH 2019: Proceedings of the International Conference on Medical and Biological Engineering, 16-18 May 2019, Banja Luka, Bosnia and Herzegovina","author":"Alickovic","year":"2020"},{"key":"B2","doi-asserted-by":"publisher","first-page":"104879","DOI":"10.1016\/j.compbiomed.2021.104879","article-title":"3d shearlet-based descriptors combined with deep features for the classification of alzheimer's disease based on MRI data","volume":"138","author":"Alinsaif","year":"2021","journal-title":"Comput. Biol. Med"},{"key":"B3","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1186\/s40708-023-00195-7","article-title":"A systematic review on machine learning and deep learning techniques in the effective diagnosis of alzheimer's disease","volume":"10","author":"Arya","year":"2023","journal-title":"Brain Informat"},{"key":"B4","doi-asserted-by":"publisher","first-page":"101645","DOI":"10.1016\/j.nicl.2018.101645","article-title":"Automated classification of alzheimer's disease and mild cognitive impairment using a single MRI and deep neural networks","volume":"21","author":"Basaia","year":"2019","journal-title":"NeuroImage: Clini"},{"key":"B5","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1145\/1007730.1007735","article-title":"A study of the behavior of several methods for balancing machine learning training data","volume":"6","author":"Batista","year":"2004","journal-title":"ACM SIGKDD Explorat. Newslett"},{"key":"B6","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1001\/jama.2019.20866","article-title":"Challenges to the reproducibility of machine learning models in health care","volume":"323","author":"Beam","year":"2020","journal-title":"JAMA"},{"key":"B7","doi-asserted-by":"publisher","first-page":"3905","DOI":"10.1002\/hbm.25473","article-title":"Comparison of structural mri brain measures between 1.5 and 3 t: Data from the lothian birth cohort 1936","volume":"42","author":"Buchanan","year":"2021","journal-title":"Hum. Brain Mapp"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-021-84630-x","article-title":"Deep learning classification of lung cancer histology using ct images","volume":"11","author":"Chaunzwa","year":"2021","journal-title":"Sci. Rep"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12864-019-6413-7","article-title":"The advantages of the matthews correlation coefficient (MCC) over f1 score and accuracy in binary classification evaluation","volume":"21","author":"Chicco","year":"2020","journal-title":"BMC Genom"},{"key":"B10","doi-asserted-by":"crossref","DOI":"10.2172\/6057803","volume-title":"Multiple-Comparisons Procedures","author":"Conover","year":"1979"},{"key":"B11","doi-asserted-by":"publisher","first-page":"117693510600200030","DOI":"10.1177\/117693510600200030","article-title":"Applications of machine learning in cancer prediction and prognosis","volume":"2","author":"Cruz","year":"2006","journal-title":"Cancer Inform"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.11929","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","journal-title":"arXiv [Preprint]"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1159","DOI":"10.1093\/brain\/awm016","article-title":"Different regional patterns of cortical thinning in Alzheimer's disease and frontotemporal dementia","volume":"130","author":"Du","year":"2007","journal-title":"Brain"},{"key":"B14","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.jalz.2016.02.002","article-title":"Preclinical Alzheimer's disease: definition, natural history, and diagnostic criteria","volume":"12","author":"Dubois","year":"2016","journal-title":"Alzheimer's"},{"key":"B15","doi-asserted-by":"publisher","first-page":"20211253","DOI":"10.1259\/bjr.20211253","article-title":"Deep transfer learning-based fully automated detection and classification of Alzheimer's disease on brain mri","volume":"95","author":"Ghaffari","year":"2022","journal-title":"Br. J. Radiol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"E14","DOI":"10.1038\/s41586-020-2766-y","article-title":"Transparency and reproducibility in artificial intelligence","volume":"586","author":"Haibe-Kains","year":"2020","journal-title":"Nature"},{"key":"B17","first-page":"770","article-title":"\u201cDeep residual learning for image recognition,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He","year":"2016"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1132","DOI":"10.1038\/s41592-021-01256-7","article-title":"Reproducibility standards for machine learning in the life sciences","volume":"18","author":"Heil","year":"2021","journal-title":"Nat. Methods"},{"key":"B19","doi-asserted-by":"publisher","first-page":"e124","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLoS Med"},{"key":"B20","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1212\/WNL.55.4.484","article-title":"Rates of hippocampal atrophy correlate with change in clinical status in aging and ad","volume":"55","author":"Jack Jr","year":"2000","journal-title":"Neurology"},{"key":"B21","doi-asserted-by":"crossref","first-page":"835","DOI":"10.1109\/ISBI.2017.7950647","article-title":"\u201cResidual and plain convolutional neural networks for 3d brain MRI classification,\u201d","volume-title":"2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017)","author":"Korolev","year":"2017"},{"key":"B22","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.csbj.2014.11.005","article-title":"Machine learning applications in cancer prognosis and prediction","volume":"13","author":"Kourou","year":"2015","journal-title":"Comput. Struct. Biotechnol. J"},{"key":"B23","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1109\/RBME.2018.2855055","article-title":"Cohort harmonization and integrative analysis from a biomedical engineering perspective","volume":"12","author":"Kourou","year":"2018","journal-title":"IEEE Rev. Biomed. Eng"},{"key":"B24","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1080\/01621459.1952.10483441","article-title":"Use of ranks in one-criterion variance analysis","volume":"47","author":"Kruskal","year":"1952","journal-title":"J. Am. Stat. Assoc"},{"key":"B25","first-page":"3361","article-title":"\u201cConvolutional networks for images, speech, and time series,\u201d","author":"LeCun","year":"1995","journal-title":"The Handbook of Brain Theory and Neural Networks"},{"key":"B26","doi-asserted-by":"publisher","first-page":"e10549","DOI":"10.7717\/peerj.10549","article-title":"Comparison of machine learning approaches for enhancing Alzheimer's disease classification","volume":"9","author":"Li","year":"2021","journal-title":"PeerJ"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1487","DOI":"10.1007\/s10462-019-09709-4","article-title":"Missing value imputation: a review and analysis of the literature (2006-2017)","volume":"53","author":"Lin","year":"2020","journal-title":"Artif. Intellig. Rev"},{"key":"B28","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1109\/ISBI.2014.6868045","article-title":"\u201cEarly diagnosis of alzheimer's disease with deep learning,\u201d","volume-title":"2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI)","author":"Liu","year":"2014"},{"key":"B29","doi-asserted-by":"publisher","first-page":"e0173372","DOI":"10.1371\/journal.pone.0173372","article-title":"Prediction and classification of alzheimer disease based on quantification of mri deformation","volume":"12","author":"Long","year":"2017","journal-title":"PLoS ONE"},{"key":"B30","doi-asserted-by":"publisher","first-page":"e323","DOI":"10.2196\/jmir.5870","article-title":"Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view","volume":"18","author":"Luo","year":"2016","journal-title":"J. Med. Internet Res"},{"key":"B31","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.neuroscience.2021.01.002","article-title":"A transfer learning approach for early diagnosis of alzheimer's disease on mri images","volume":"460","author":"Mehmood","year":"2021","journal-title":"Neuroscience"},{"key":"B32","doi-asserted-by":"publisher","first-page":"81542","DOI":"10.1109\/ACCESS.2019.2923707","article-title":"Effective heart disease prediction using hybrid machine learning techniques","volume":"7","author":"Mohan","year":"2019","journal-title":"IEEE Access"},{"key":"B33","doi-asserted-by":"publisher","first-page":"104416","DOI":"10.1016\/j.compbiomed.2021.104416","article-title":"Machine learning in diagnosis and disability prediction of multiple sclerosis using optical coherence tomography","volume":"133","author":"Montol\u00edo","year":"2021","journal-title":"Comput. Biol. Med"},{"key":"B34","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1016\/j.nic.2005.09.008","article-title":"The Alzheimer's disease neuroimaging initiative","volume":"15","author":"Mueller","year":"2005","journal-title":"Neuroimaging Clin. N. Am"},{"key":"B35","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1109\/AICCSA.2008.4493524","article-title":"\u201cIntelligent heart disease prediction system using data mining techniques,\u201d","volume-title":"2008 IEEE\/ACS International Conference on Computer Systems and Applications","author":"Palaniappan","year":"2008"},{"key":"B36","doi-asserted-by":"publisher","first-page":"501050","DOI":"10.3389\/fnins.2020.00259","article-title":"Early detection of alzheimer's disease using magnetic resonance imaging: a novel approach combining convolutional neural networks and ensemble learning","volume":"14","author":"Pan","year":"2020","journal-title":"Front. Neurosci"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1109\/TAI.2024.3416420","article-title":"Decgan: Decoupling generative adversarial network for detecting abnormal neural circuits in Alzheimer's disease","author":"Pan","year":"2024","journal-title":"IEEE Trans. Artif. Intellig"},{"key":"B38","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1109\/SIBGRAPI.2016.054","article-title":"\u201cDeep learning-aided parkinson's disease diagnosis from handwritten dynamics,\u201d","volume-title":"2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI)","author":"Pereira","year":"2016"},{"key":"B39","first-page":"7459","article-title":"Improving reproducibility in machine learning research: a report from the neurips 2019 reproducibility program","volume":"22","author":"Pineau","year":"2021","journal-title":"J. Mach. Learn. Res"},{"key":"B40","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1016\/j.arr.2016.01.002","article-title":"Brain atrophy in alzheimer's disease and aging","volume":"30","author":"Pini","year":"2016","journal-title":"Ageing Res. Rev"},{"key":"B41","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1038\/s42256-021-00307-0","article-title":"Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and ct scans","volume":"3","author":"Roberts","year":"2021","journal-title":"Nat. Mach. Intellig"},{"key":"B42","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1146\/annurev.bioeng.8.061505.095802","article-title":"Machine learning for detection and diagnosis of disease","volume":"8","author":"Sajda","year":"2006","journal-title":"Annu. Rev. Biomed. Eng"},{"key":"B43","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/ICOSEC49089.2020.9215402","article-title":"\u201cA CNN model: earlier diagnosis and classification of alzheimer disease using MRI,\u201d","volume-title":"2020 International Conference on Smart Electronics and Communication (ICOSEC)","author":"Salehi","year":"2020"},{"key":"B44","first-page":"4510","article-title":"\u201cMobilenetv2: Inverted residuals and linear bottlenecks,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Sandler","year":"2018"},{"key":"B45","first-page":"553","article-title":"Data wrangling and data leakage in machine learning for healthcare","volume":"5","author":"Saravanan","year":"2018","journal-title":"JETIR."},{"key":"B46","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1093\/biomet\/52.3-4.591","article-title":"An analysis of variance test for normality (complete samples)","volume":"52","author":"Shapiro","year":"1965","journal-title":"Biometrika"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-019-48995-4","article-title":"Deep learning to improve breast cancer detection on screening mammography","volume":"9","author":"Shen","year":"2019","journal-title":"Sci. Rep"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0197-0","article-title":"A survey on image data augmentation for deep learning","volume":"6","author":"Shorten","year":"2019","journal-title":"J. Big Data"},{"key":"B49","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inform. Proc. Manage"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41746-019-0079-z","article-title":"The reproducibility crisis in the age of digital medicine","volume":"2","author":"Stupple","year":"2019","journal-title":"NPJ Digit. Med"},{"key":"B51","first-page":"6105","article-title":"\u201cEfficientnet: Rethinking model scaling for convolutional neural networks,\u201d","volume-title":"International Conference on Machine Learning","author":"Tan","year":"2019"},{"key":"B52","doi-asserted-by":"publisher","first-page":"808","DOI":"10.1016\/j.media.2014.04.006","article-title":"Multiple instance learning for classification of dementia in brain mri","volume":"18","author":"Tong","year":"2014","journal-title":"Med. Image Anal"},{"key":"B53","doi-asserted-by":"publisher","first-page":"439","DOI":"10.1136\/jnnp.2005.075341","article-title":"Hippocampal atrophy on mri in frontotemporal lobar degeneration and alzheimer's disease","volume":"77","author":"Van De Pol","year":"2006","journal-title":"J. Neurol. Neurosurg. Psychiat"},{"key":"B54","first-page":"11","article-title":"Visualizing data using t-SNE","volume":"9","author":"Van der Maaten","year":"2008","journal-title":"J. Mach. Learn. Res"},{"key":"B55","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3332186.3332209","article-title":"\u201cTraps, pitfalls and misconceptions of machine learning applied to scientific disciplines,\u201d","volume-title":"Proceedings of the Practice and Experience in Advanced Research Computing on Rise of the Machines (Learning)","author":"Vento","year":"2019"},{"key":"B56","doi-asserted-by":"publisher","first-page":"101694","DOI":"10.1016\/j.media.2020.101694","article-title":"Convolutional neural networks for classification of alzheimer's disease: Overview and reproducible evaluation","volume":"63","author":"Wen","year":"2020","journal-title":"Med. Image Anal"},{"key":"B57","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1038\/nrneurol.2017.63","article-title":"The changing prevalence and incidence of dementia over time\u2013current evidence","volume":"13","author":"Wu","year":"2017","journal-title":"Nat. Rev. Neurol"},{"key":"B58","doi-asserted-by":"publisher","first-page":"1952373","DOI":"10.1155\/2017\/1952373","article-title":"Brain mr image classification for Alzheimer's disease diagnosis based on multifeature fusion","volume":"2017","author":"Xiao","year":"2017","journal-title":"Comput. Math. Methods Med"},{"key":"B59","doi-asserted-by":"publisher","first-page":"4401","DOI":"10.1109\/TNNLS.2021.3118369","article-title":"Morphological feature visualization of alzheimer's disease via multidirectional perception gan","volume":"34","author":"Yu","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B60","doi-asserted-by":"publisher","first-page":"93752","DOI":"10.1109\/ACCESS.2019.2926288","article-title":"Hierarchical feature extraction for early alzheimer's disease diagnosis","volume":"7","author":"Yue","year":"2019","journal-title":"IEEE Access"},{"key":"B61","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1145\/3446776","article-title":"Understanding deep learning (still) requires rethinking generalization","volume":"64","author":"Zhang","year":"2021","journal-title":"Commun. ACM"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2024.3442811","article-title":"A new brain network construction paradigm for brain disorder via diffusion-based graph contrastive learning","author":"Zong","year":"2024","journal-title":"IEEE Trans. Pattern Analy. Mach. Intellig"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2023.3344641","article-title":"Prior-guided adversarial learning with hypergraph for predicting abnormal connections in Alzheimer's disease","author":"Zuo","year":"2024","journal-title":"IEEE Trans. Cybernet"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1360095\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,20]],"date-time":"2024-09-20T14:53:31Z","timestamp":1726844011000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1360095\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,20]]},"references-count":63,"alternative-id":["10.3389\/fncom.2024.1360095"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1360095","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,20]]},"article-number":"1360095"}}