{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,11]],"date-time":"2026-07-11T04:25:02Z","timestamp":1783743902217,"version":"3.55.0"},"reference-count":42,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,11,29]],"date-time":"2024-11-29T00:00:00Z","timestamp":1732838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NINDS-IRP program"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["NeuroSci"],"abstract":"<jats:p>Classification of disease and healthy volunteer cohorts provides a useful clinical alternative to traditional group statistics due to individualized, personalized predictions. Classifiers for neurodegenerative disease can be trained on structural MRI morphometry, but require large multi-scanner datasets, introducing confounding batch effects. We test ComBat, a common harmonization model, in an example application to classify subjects with Parkinson\u2019s disease from healthy volunteers and identify common pitfalls, including data leakage. We used a multi-dataset cohort of 372 subjects (216 with Parkinson\u2019s disease, 156 healthy volunteers) from 11 identified scanners. We extracted both FreeSurfer and the determinant of Jacobian morphometry to compare single-scanner and multi-scanner classification pipelines. We confirm the presence of batch effects by running single scanner classifiers which could achieve wildly divergent AUCs on scanner-specific datasets (mean:0.651 \u00b1 0.144). Multi-scanner classifiers that considered neurobiological batch effects between sites could easily achieve a test AUC of 0.902, though pipelines that prevented data leakage could only achieve a test AUC of 0.550. We conclude that batch effects remain a major issue for classification problems, such that even impressive single-scanner classifiers are unlikely to generalize to multiple scanners, and that solving for batch effects in a classifier problem must avoid circularity and reporting overly optimistic results.<\/jats:p>","DOI":"10.3390\/neurosci5040042","type":"journal-article","created":{"date-parts":[[2024,12,4]],"date-time":"2024-12-04T11:03:08Z","timestamp":1733310188000},"page":"600-613","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Harmonization for Parkinson\u2019s Disease Multi-Dataset T1 MRI Morphometry Classification"],"prefix":"10.3390","volume":"5","author":[{"given":"Mohammed","family":"Saqib","sequence":"first","affiliation":[{"name":"University of Pennsylvania, Philadelphia, PA 19104, USA"},{"name":"National Institute of Neurological Disorders and Strokes, National Institutes of Health, Bethesda, MD 20892, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5501-5918","authenticated-orcid":false,"given":"Silvina G.","family":"Horovitz","sequence":"additional","affiliation":[{"name":"National Institute of Neurological Disorders and Strokes, National Institutes of Health, Bethesda, MD 20892, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1007\/s00702-017-1686-y","article-title":"Epidemiology of Parkinson\u2019s Disease","volume":"124","author":"Tysnes","year":"2017","journal-title":"J. Neural Transm."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"S40","DOI":"10.1016\/j.metabol.2014.10.030","article-title":"Biomarkers of Parkinson\u2019s Disease: Present and Future","volume":"64","author":"Miller","year":"2015","journal-title":"Metabolism"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3144","DOI":"10.1007\/s00415-020-09863-8","article-title":"Progression of Grey and White Matter Brain Damage in Parkinson\u2019s Disease: A Critical Review of Structural MRI Literature","volume":"268","author":"Sarasso","year":"2021","journal-title":"J. Neurol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.neurobiolaging.2014.07.010","article-title":"Parkinson\u2019s Disease and Local Atrophy in Subcortical Nuclei: Insight from Shape Analysis","volume":"36","author":"Nemmi","year":"2015","journal-title":"Neurobiol. Aging"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2583","DOI":"10.1002\/mds.28706","article-title":"International Multicenter Analysis of Brain Structure Across Clinical Stages of Parkinson\u2019s Disease","volume":"36","author":"Laansma","year":"2021","journal-title":"Mov. Disord."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1038\/nrn3475","article-title":"Power Failure: Why Small Sample Size Undermines the Reliability of Neuroscience","volume":"14","author":"Button","year":"2013","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_7","unstructured":"(2011). Parkinson Progression Marker Initiative The Parkinson Progression Marker Initiative (PPMI). Prog. Neurobiol., 95, 629\u2013635."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1016\/j.neulet.2017.04.034","article-title":"A Multilevel-ROI-Features-Based Machine Learning Method for Detection of Morphometric Biomarkers in Parkinson\u2019s Disease","volume":"651","author":"Peng","year":"2017","journal-title":"Neurosci. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Long, D., Wang, J., Xuan, M., Gu, Q., Xu, X., Kong, D., and Zhang, M. (2012). Automatic Classification of Early Parkinson\u2019s Disease with Multi-Modal MR Imaging. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0047714"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1016\/j.jneumeth.2013.11.016","article-title":"Machine Learning on Brain MRI Data for Differential Diagnosis of Parkinson\u2019s Disease and Progressive Supranuclear Palsy","volume":"222","author":"Salvatore","year":"2014","journal-title":"J. Neurosci. Methods"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1506","DOI":"10.1002\/mds.26715","article-title":"Differentiation of Neurodegenerative Parkinsonian Syndromes by Volumetric Magnetic Resonance Imaging Analysis and Support Vector Machine Classification","volume":"31","author":"Huppertz","year":"2016","journal-title":"Mov. Disord."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Badea, L., Onu, M., Wu, T., Roceanu, A., and Bajenaru, O. (2017). Exploring the Reproducibility of Functional Connectivity Alterations in Parkinson\u2019s Disease. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0188196"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1016\/j.neuroimage.2005.09.046","article-title":"Reliability in Multi-Site Structural MRI Studies: Effects of Gradient Non-Linearity Correction on Phantom and Human Data","volume":"30","author":"Jovicich","year":"2006","journal-title":"NeuroImage"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.neuroimage.2006.02.051","article-title":"Reliability of MRI-Derived Measurements of Human Cerebral Cortical Thickness: The Effects of Field Strength, Scanner Upgrade and Manufacturer","volume":"32","author":"Han","year":"2006","journal-title":"NeuroImage"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bayer, J.M.M., Thompson, P.M., Ching, C.R.K., Liu, M., Chen, A., Panzenhagen, A.C., Jahanshad, N., Marquand, A., Schmaal, L., and S\u00e4mann, P.G. (2022). Site Effects How-to and When: An Overview of Retrospective Techniques to Accommodate Site Effects in Multi-Site Neuroimaging Analyses. Front. Neurol., 13.","DOI":"10.3389\/fneur.2022.923988"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.neuroimage.2017.11.024","article-title":"Harmonization of Cortical Thickness Measurements across Scanners and Sites","volume":"167","author":"Fortin","year":"2018","journal-title":"NeuroImage"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1179","DOI":"10.1002\/hbm.25688","article-title":"Mitigating Site Effects in Covariance for Machine Learning in Neuroimaging Data","volume":"43","author":"Chen","year":"2022","journal-title":"Hum. Brain Mapp."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Reynolds, M., Chaudhary, T., Torbati, M.E., Tudorascu, D.L., and Batmanghelich, K. (2022). Alzheimer\u2019s Disease Neuroimaging Initiative. ComBat Harmonization: Empirical Bayes versus Fully Bayes Approaches. bioRxiv.","DOI":"10.1101\/2022.07.13.499561"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tafuri, B., Lombardi, A., Nigro, S., Urso, D., Monaco, A., Pantaleo, E., Diacono, D., De Blasi, R., Bellotti, R., and Tangaro, S. (2022). The Impact of Harmonization on Radiomic Features in Parkinson\u2019s Disease and Healthy Controls: A Multicenter Study. Front. Neurosci., 16.","DOI":"10.3389\/fnins.2022.1012287"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1007\/s00429-014-0981-8","article-title":"Deficits in Task-Set Maintenance and Execution Networks in Parkinson\u2019s Disease","volume":"221","author":"Tinaz","year":"2016","journal-title":"Brain Struct. Funct."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.nicl.2016.12.019","article-title":"Changes in Functional Organization and White Matter Integrity in the Connectome in Parkinson\u2019s Disease","volume":"13","author":"Tinaz","year":"2017","journal-title":"Neuroimage Clin."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"103100","DOI":"10.1016\/j.nicl.2022.103100","article-title":"Neural Correlates of Working Memory and Compensation at Different Stages of Cognitive Impairment in Parkinson\u2019s Disease","volume":"35","author":"Hattori","year":"2022","journal-title":"Neuroimage Clin."},{"key":"ref_23","first-page":"S34","article-title":"What features improve the accuracy of clinical diagnosis in Parkinson\u2019s disease: A clinicopathologic study","volume":"57","author":"Hughes","year":"1992","journal-title":"Neurology"},{"key":"ref_24","unstructured":"Ashburner, J., and Friston, K. (2004). Chapter 6. Morphometry. Human Brain Function, Elsevier."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1016\/j.neuroimage.2012.01.021","article-title":"FreeSurfer","volume":"62","author":"Fischl","year":"2012","journal-title":"Neuroimage"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1006\/cbmr.1996.0014","article-title":"AFNI: Software for Analysis and Visualization of Functional Magnetic Resonance Neuroimages","volume":"29","author":"Cox","year":"1996","journal-title":"Comput. Biomed. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1016\/j.neuroimage.2014.09.034","article-title":"Accurate Automatic Estimation of Total Intracranial Volume: A Nuisance Variable with Less Nuisance","volume":"104","author":"Malone","year":"2015","journal-title":"NeuroImage"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1093\/biostatistics\/kxj037","article-title":"Adjusting Batch Effects in Microarray Expression Data Using Empirical Bayes Methods","volume":"8","author":"Johnson","year":"2007","journal-title":"Biostatistics"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from Imbalanced Data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic Minority Over-Sampling Technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","first-page":"2825","article-title":"Scikit-Learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_32","first-page":"1","article-title":"Imbalanced-Learn: A Python Toolbox to Tackle the Curse of Imbalanced Datasets in Machine Learning","volume":"18","author":"Nogueira","year":"2017","journal-title":"J. Mach. Learn. Res."},{"key":"ref_33","first-page":"281","article-title":"Random Search for Hyper-Parameter Optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","unstructured":"Guan, H., and Liu, M. (2021). Domain Adaptation for Medical Image Analysis: A Survey. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"116450","DOI":"10.1016\/j.neuroimage.2019.116450","article-title":"Harmonization of Large MRI Datasets for the Analysis of Brain Imaging Patterns throughout the Lifespan","volume":"208","author":"Pomponio","year":"2020","journal-title":"NeuroImage"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Stein, C.K., Qu, P., Epstein, J., Buros, A., Rosenthal, A., Crowley, J., Morgan, G., and Barlogie, B. (2015). Removing Batch Effects from Purified Plasma Cell Gene Expression Microarrays with Modified ComBat. BMC Bioinform., 16.","DOI":"10.1186\/s12859-015-0478-3"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40708-020-00112-2","article-title":"Application of Deep Learning in Detecting Neurological Disorders from Magnetic Resonance Images: A Survey on the Detection of Alzheimer\u2019s Disease, Parkinson\u2019s Disease and Schizophrenia","volume":"7","author":"Noor","year":"2020","journal-title":"Brain Inform."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Segato, A., Marzullo, A., Calimeri, F., and De Momi, E. (2020). Artificial Intelligence for Brain Diseases: A Systematic Review. APL Bioeng., 4.","DOI":"10.1063\/5.0011697"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/s41597-023-02421-7","article-title":"Efficacy of MRI Data Harmonization in the Age of Machine Learning: A Multicenter Study across 36 Datasets","volume":"11","author":"Marzi","year":"2024","journal-title":"Sci. Data"},{"key":"ref_40","first-page":"1","article-title":"Clinical Utility of DaTscan in Patients with Suspected Parkinsonian Syndrome: A Systematic Review and Meta-Analysis","volume":"7","author":"Bega","year":"2021","journal-title":"NPJ Park. Dis."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"117127","DOI":"10.1016\/j.neuroimage.2020.117127","article-title":"Neuroharmony: A New Tool for Harmonizing Volumetric MRI Data from Unseen Scanners","volume":"220","author":"Scarpazza","year":"2020","journal-title":"Neuroimage"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Seoni, S., Shahini, A., Meiburger, K.M., Marzola, F., Rotunno, G., Acharya, U.R., Molinari, F., and Salvi, M. (2024). All You Need Is Data Preparation: A Systematic Review of Image Harmonization Techniques in Multi-Center\/Device Studies for Medical Support Systems. Comput. Methods Programs Biomed., 250.","DOI":"10.1016\/j.cmpb.2024.108200"}],"container-title":["NeuroSci"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2673-4087\/5\/4\/42\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:42:47Z","timestamp":1760114567000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2673-4087\/5\/4\/42"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,29]]},"references-count":42,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["neurosci5040042"],"URL":"https:\/\/doi.org\/10.3390\/neurosci5040042","relation":{},"ISSN":["2673-4087"],"issn-type":[{"value":"2673-4087","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,29]]}}}