{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T20:20:46Z","timestamp":1773865246930,"version":"3.50.1"},"posted":{"date-parts":[[2026,3,16]]},"group-title":"Neurology","reference-count":44,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T00:00:00Z","timestamp":1773619200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,3,16]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                <jats:p>Machine-learning (ML) models for Alzheimer\u2019s disease (AD) frequently yield divergent conclusions, raising concerns about robustness, reproducibility, and interpretability. This instability is partially driven by researcher biases and analytic variability. Coupled with the clinical heterogeneity, mixed pathologies, and cohort differences in AD research, these issues limit the reliability and validity of conclusions from individual models.<\/jats:p>\n                <jats:p>We introduce AutoML-Multiverse, an instability-aware framework characterising how analytic choices influence ML-based conclusions. The AutoML-Multiverse explores a large space of \u223c20,000 analysispipelines and by retaining the full distribution of pipelines, enables direct examination of analytic variability. We evaluate this framework across 20 classification tasks in two independent cohorts studying Alzheimer\u2019s disease progression (ADNI, N\u22641,930; NACC, N\u22641,057), using multiple data modalities: neuroimaging, clinical\/cognitive and fluid biomarkers.<\/jats:p>\n                <jats:p>AutoML-Multiverse performance was equal to or better than non-automated models across all tasks. For example, stable versus progressive mild cognitive impairment (MCI) classification accuracy was 0.68\u00b10.06 (ADNI) and 0.63\u00b10.08 (NACC), while AD versus cognitively normal (CN) classification reached 0.97\u00b10.01 (ADNI). Crucially, each modality\u2019s utility was task- and cohort-dependent: clinical measures dominated diagnostic tasks, whereas imaging better predicted progression, with modality preferences often switching between cohorts, highlighting limited generalisability of single-cohort results.<\/jats:p>\n                <jats:p>Using the AutoML-Multiverse, we obtained strong classification performance without pre-specifying key model design choices. By reducing analysis-driven variability and explicitly characterising uncertainty, instability-aware evaluation can support the development of more robust and clinically applicable prediction models in AD research.<\/jats:p>\n                <jats:sec>\n                  <jats:title>Highlights<\/jats:title>\n                  <jats:list list-type=\"bullet\">\n                    <jats:list-item>\n                      <jats:p>AutoML-Multiverse systematically quantifies analytic instability in clinical ML.<\/jats:p>\n                    <\/jats:list-item>\n                    <jats:list-item>\n                      <jats:p>Analysis of \u223c20,000 pipelines across ADNI and NACC cohorts.<\/jats:p>\n                    <\/jats:list-item>\n                    <jats:list-item>\n                      <jats:p>Pipeline choices substantially alter model rankings and biomarker importance.<\/jats:p>\n                    <\/jats:list-item>\n                    <jats:list-item>\n                      <jats:p>Cross-cohort variability highlights risks of single-dataset studies.<\/jats:p>\n                    <\/jats:list-item>\n                    <jats:list-item>\n                      <jats:p>Instability-aware evaluation improves robustness of AI-driven research.<\/jats:p>\n                    <\/jats:list-item>\n                  <\/jats:list>\n                <\/jats:sec>","DOI":"10.64898\/2026.03.13.26347938","type":"posted-content","created":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T00:25:27Z","timestamp":1773707127000},"source":"Crossref","is-referenced-by-count":0,"title":["AutoML-Multiverse: An Instability-Aware Framework for Quantifying Analytic Variability in Alzheimer\u2019s Disease Machine-Learning Studies"],"prefix":"10.64898","author":[{"given":"Maitrei","family":"Kohli","sequence":"first","affiliation":[{"name":"UCL Hawkes Institute, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gonzalo","family":"Castro Leal","sequence":"additional","affiliation":[{"name":"UCL Hawkes Institute, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Douglas","family":"Wyllie","sequence":"additional","affiliation":[{"name":"UCL Hawkes Institute, University College London, London, UK"},{"name":"Institute of Health Informatics, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Neil","family":"Oxtoby","sequence":"additional","affiliation":[{"name":"UCL Hawkes Institute, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert","family":"Leech","sequence":"additional","affiliation":[{"name":"Institute of Psychiatry, Psychology & Neuroscience, King\u2019s College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philip","family":"Weston","sequence":"additional","affiliation":[{"name":"Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1908-5588","authenticated-orcid":false,"given":"James","family":"Cole","sequence":"additional","affiliation":[{"name":"UCL Hawkes Institute, University College London, London, UK"},{"name":"Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"name":"the Alzheimer\u2019s Disease Neuroimaging Initiative","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"54368","reference":[{"issue":"16","key":"2026031805500682000_2026.03.13.26347938v1.1","doi-asserted-by":"crossref","first-page":"5913","DOI":"10.3390\/jcm14165913","article-title":"Artificial Intelligence in Alzheimer\u2019s Disease Diagnosis and Prognosis Using PET-MRI: A Narrative Review of High-Impact Literature Post-Tauvid Approval","volume":"14","year":"2025","journal-title":"J Clin Med"},{"key":"2026031805500682000_2026.03.13.26347938v1.2","doi-asserted-by":"crossref","first-page":"124780","DOI":"10.1016\/j.eswa.2024.124780","article-title":"Alzheimer\u2019s disease diagnosis from single and multimodal data using machine and deep learning models: Achievements and future directions","volume":"255","year":"2024","journal-title":"Expert Syst Appl"},{"key":"2026031805500682000_2026.03.13.26347938v1.3","doi-asserted-by":"crossref","first-page":"102989","DOI":"10.1016\/j.arr.2025.102989","article-title":"Predictive modeling approaches for Alzheimer\u2019s disease diagnosis through neuroimaging techniques","volume":"114","year":"2026","journal-title":"Ageing Res Rev"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.4","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/s40708-025-00252-3","article-title":"Machine-learning models for Alzheimer\u2019s disease diagnosis using neuroimaging data: survey, reproducibility, and generalizability evaluation","volume":"12","year":"2025","journal-title":"Brain Inform"},{"key":"2026031805500682000_2026.03.13.26347938v1.5","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-31347-8"},{"key":"2026031805500682000_2026.03.13.26347938v1.6","doi-asserted-by":"crossref","first-page":"1360095","DOI":"10.3389\/fncom.2024.1360095","article-title":"Deep learning-based Alzheimer\u2019s disease detection: reproducibility and the effect of modeling choices","volume":"18","year":"2024","journal-title":"Front Comput Neurosci"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.7","doi-asserted-by":"crossref","first-page":"17106","DOI":"10.1038\/s41598-022-20674-x","article-title":"Generalizable deep learning model for early Alzheimer\u2019s disease detection from structural MRIs","volume":"12","year":"2022","journal-title":"Sci Rep"},{"key":"2026031805500682000_2026.03.13.26347938v1.8","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-2314-9"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.9","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1038\/s42003-025-08412-1","article-title":"fNIRS reproducibility varies with data quality, analysis pipelines, and researcher experience","volume":"8","year":"2025","journal-title":"Commun Biol"},{"issue":"8","key":"2026031805500682000_2026.03.13.26347938v1.10","first-page":"780","article-title":"Reproducibility in Neuroimaging Analysis: Challenges and Solutions","volume":"8","year":"2023","journal-title":"Biol Psychiatry Cogn Neurosci Neuroimaging"},{"key":"2026031805500682000_2026.03.13.26347938v1.11","doi-asserted-by":"publisher","DOI":"10.1002\/hbm.25847"},{"key":"2026031805500682000_2026.03.13.26347938v1.12","doi-asserted-by":"publisher","DOI":"10.1007\/s00702-021-02449-2"},{"key":"2026031805500682000_2026.03.13.26347938v1.13","unstructured":"Feurer M , Klein A , Eggensperger K , Springenberg J , Blum M , Hutter F. Efficient and Robust Automated Machine Learning."},{"key":"2026031805500682000_2026.03.13.26347938v1.14","doi-asserted-by":"crossref","unstructured":"Olson RS , Moore JH . TPOT: A Tree-Based Pipeline Optimization Tool for Automating Machine Learning. In: Hutter F , Kotthoff L , Vanschoren J , editors. Automated Machine Learning [Internet]. Cham: Springer International Publishing; 2019 [cited 2026 Jan 22]. p. 151\u201360. (The Springer Series on Challenges in Machine Learning). Available from: http:\/\/link.springer.com\/10.1007\/978-3-030-05318-5_8","DOI":"10.1007\/978-3-030-05318-5_8"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.15","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s10462-025-11397-2","article-title":"A literature review on automated machine learning","volume":"59","year":"2025","journal-title":"Artif Intell Rev"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.16","doi-asserted-by":"crossref","first-page":"17682","DOI":"10.1038\/s41598-025-02149-x","article-title":"A practical evaluation of AutoML tools for binary, multiclass, and multilabel classification","volume":"15","year":"2025","journal-title":"Sci Rep"},{"issue":"5","key":"2026031805500682000_2026.03.13.26347938v1.17","doi-asserted-by":"crossref","first-page":"e254","DOI":"10.1016\/S2589-7500(23)00054-7","article-title":"Automated machine learning as a partner in predictive modelling","volume":"5","year":"2023","journal-title":"Lancet Digit Health"},{"key":"2026031805500682000_2026.03.13.26347938v1.18","doi-asserted-by":"crossref","first-page":"10483","DOI":"10.1038\/s41598-024-60429-4","article-title":"A comparative study of an on premise AutoML solution for medical image classification","volume":"14","year":"2024","journal-title":"Sci Rep"},{"issue":"3","key":"2026031805500682000_2026.03.13.26347938v1.19","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1002\/med4.75","article-title":"Automated machine learning with interpretation: A systematic review of methodologies and applications in healthcare","volume":"2","year":"2024","journal-title":"Med Adv"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.20","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13040-022-00300-2","article-title":"Benchmarking AutoML frameworks for disease prediction using medical claims","volume":"15","year":"2022","journal-title":"BioData Min"},{"issue":"3","key":"2026031805500682000_2026.03.13.26347938v1.21","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1007\/s10729-025-09718-6","article-title":"Innovations in early detection of chronic non-communicable diseases among adolescents through an easy-to-Use AutoML paradigm","volume":"28","year":"2025","journal-title":"Health Care Manag Sci"},{"key":"2026031805500682000_2026.03.13.26347938v1.22","doi-asserted-by":"crossref","unstructured":"Raj R , Kannath SK , Mathew J , Sylaja PN . AutoML accurately predicts endovascular mechanical thrombectomy in acute large vessel ischemic stroke. Front Neurol [Internet]. 2023 Sep 28 [cited 2026 Jan 23];14. Available from: https:\/\/www.frontiersin.org\/journals\/neurology\/articles\/10.3389\/fneur.2023.1259958\/full","DOI":"10.3389\/fneur.2023.1259958"},{"key":"2026031805500682000_2026.03.13.26347938v1.23","first-page":"100957","article-title":"Evaluation of machine learning models for the prediction of Alzheimer\u2019s: In search of the best performance","volume":"44","year":"2025","journal-title":"Brain Behav Immun - Health"},{"key":"2026031805500682000_2026.03.13.26347938v1.24","unstructured":"Costa PFD , Dafflon J , Pinaya WHL , Monti RP , Bzdok D , Smallwood J , et al. BUILDING A DATA-DRIVEN CONFIGURATION SPACE FOR AUTOMATED MACHINE LEARNING. 2023;"},{"key":"2026031805500682000_2026.03.13.26347938v1.25","first-page":"1","article-title":"TADPOLE Challenge: Accurate Alzheimer\u2019s disease prediction through crowdsourced forecasting of future data","volume":"11843","year":"2019","journal-title":"Predict Intell Med PRIME Workshop"},{"key":"2026031805500682000_2026.03.13.26347938v1.26","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2012.01.021"},{"issue":"7","key":"2026031805500682000_2026.03.13.26347938v1.27","doi-asserted-by":"crossref","first-page":"101317","DOI":"10.1016\/j.patter.2025.101317","article-title":"OpenML: Insights from 10 years and more than a thousand papers","volume":"6","year":"2025","journal-title":"Patterns"},{"key":"2026031805500682000_2026.03.13.26347938v1.28","unstructured":"Snoek J , Larochelle H , Adams RP . Practical Bayesian Optimization of Machine Learning Algorithms. In: Advances in Neural Information Processing Systems [Internet]. Curran Associates, Inc.; 2012 [cited 2026 Jan 23]. Available from: https:\/\/papers.nips.cc\/paper_files\/paper\/2012\/hash\/05311655a15b75fab86956663e1819cd-Abstract.html"},{"key":"2026031805500682000_2026.03.13.26347938v1.29","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(05)80023-1"},{"key":"2026031805500682000_2026.03.13.26347938v1.30","unstructured":"John GH , Langley P. Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the Eleventh conference on Uncertainty in artificial intelligence [Internet]. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.; 1995 [cited 2026 Jan 23]. p. 338\u201345. (UAI\u201995). Available from: https:\/\/dl.acm.org\/doi\/10.5555\/2074158.2074196"},{"key":"2026031805500682000_2026.03.13.26347938v1.31","doi-asserted-by":"publisher","DOI":"10.1016\/j.cpc.2010.04.018"},{"key":"2026031805500682000_2026.03.13.26347938v1.32","doi-asserted-by":"crossref","unstructured":"Chen T , Guestrin C . XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining [Internet]. New York, NY, USA: Association for Computing Machinery; 2016 [cited 2026 Jan 23]. p. 785\u201394. (KDD \u201916). Available from: https:\/\/dl.acm.org\/doi\/10.1145\/2939672.2939785","DOI":"10.1145\/2939672.2939785"},{"key":"2026031805500682000_2026.03.13.26347938v1.33","doi-asserted-by":"crossref","first-page":"105846","DOI":"10.1016\/j.neubiorev.2024.105846","article-title":"The multiverse of data preprocessing and analysis in graph-based fMRI: A systematic literature review of analytical choices fed into a decision support tool for informed analysis","volume":"165","year":"2024","journal-title":"Neurosci Biobehav Rev"},{"key":"2026031805500682000_2026.03.13.26347938v1.34","doi-asserted-by":"crossref","unstructured":"Sanz-Robinson J , Wang M , McPherson B , Chatelain Y , Kennedy D , Glatard T , et al. Open-source platforms to investigate analytical flexibility in neuroimaging. Imaging Neurosci. 2025 Jul 21;3:IMAG.a.79.","DOI":"10.1162\/IMAG.a.79"},{"key":"2026031805500682000_2026.03.13.26347938v1.35","doi-asserted-by":"crossref","unstructured":"Stahlschmidt SR , Ulfenborg B , Synnergren J . Multimodal deep learning for biomedical data fusion: a review. Brief Bioinform. 2022 Mar 1;23(2):bbab569.","DOI":"10.1093\/bib\/bbab569"},{"issue":"9","key":"2026031805500682000_2026.03.13.26347938v1.36","doi-asserted-by":"crossref","first-page":"3753","DOI":"10.1007\/s11263-024-02032-8","article-title":"Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects","volume":"132","year":"2024","journal-title":"Int J Comput Vis"},{"key":"2026031805500682000_2026.03.13.26347938v1.37","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-74399-w"},{"key":"2026031805500682000_2026.03.13.26347938v1.38","doi-asserted-by":"crossref","unstructured":"Han L. AD-Diff: enhancing Alzheimer\u2019s disease prediction accuracy through multimodal fusion. Front Comput Neurosci [Internet]. 2025 Mar 12 [cited 2026 Jan 23];19. Available from: https:\/\/www.frontiersin.org\/journals\/computational-neuroscience\/articles\/10.3389\/fncom.2025.1484540\/full","DOI":"10.3389\/fncom.2025.1484540"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.39","doi-asserted-by":"crossref","first-page":"2565166","DOI":"10.1080\/08839514.2025.2565166","article-title":"Automated Machine Learning Approach (AutoML) to Alzheimer\u2019s Disease Diagnosis and Prognosis","volume":"39","year":"2025","journal-title":"Appl Artif Intell"},{"key":"2026031805500682000_2026.03.13.26347938v1.40","first-page":"544","article-title":"Exploring Automated Machine Learning for Cognitive Outcome Prediction from Multimodal Brain Imaging using STREAMLINE","volume":"2023","year":"2023","journal-title":"AMIA Summits Transl Sci Proc"},{"issue":"1","key":"2026031805500682000_2026.03.13.26347938v1.41","doi-asserted-by":"crossref","first-page":"22721","DOI":"10.1038\/s41598-025-08092-1","article-title":"Classifying and diagnosing Alzheimer\u2019s disease with deep learning using 6735 brain MRI images","volume":"15","year":"2025","journal-title":"Sci Rep"},{"key":"2026031805500682000_2026.03.13.26347938v1.42","doi-asserted-by":"crossref","unstructured":"Raza ML , Hassan ST , Jamil S , Hyder N , Batool K , Walji S , et al. Advancements in deep learning for early diagnosis of Alzheimer\u2019s disease using multimodal neuroimaging: challenges and future directions. Front Neuroinformatics [Internet]. 2025 May 2 [cited 2026 Jan 23];19. Available from: https:\/\/www.frontiersin.org\/journals\/neuroinformatics\/articles\/10.3389\/fninf.2025.1 557177\/full","DOI":"10.3389\/fninf.2025.1557177"},{"key":"2026031805500682000_2026.03.13.26347938v1.43","doi-asserted-by":"crossref","unstructured":"Munroe L , da Silva M , Heidari F , Grigorescu I , Dahan S , Robinson EC , et al. Applications of interpretable deep learning in neuroimaging: A comprehensive review. Imaging Neurosci. 2024 Jul 12;2:imag\u20132\u201300214.","DOI":"10.1162\/imag_a_00214"},{"key":"2026031805500682000_2026.03.13.26347938v1.44","doi-asserted-by":"crossref","first-page":"100343","DOI":"10.1016\/j.medntd.2024.100343","article-title":"A systematic review on recent methods on deep learning for automatic detection of Alzheimer\u2019s disease","volume":"25","year":"2025","journal-title":"Med Nov Technol Devices"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.03.13.26347938","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T12:50:29Z","timestamp":1773838229000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.03.13.26347938"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,16]]},"references-count":44,"URL":"https:\/\/doi.org\/10.64898\/2026.03.13.26347938","relation":{},"subject":[],"published":{"date-parts":[[2026,3,16]]},"subtype":"preprint"}}