{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T01:12:33Z","timestamp":1778634753848,"version":"3.51.4"},"reference-count":56,"publisher":"SAGE Publications","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IDA"],"published-print":{"date-parts":[[2021,4,20]]},"abstract":"<jats:p>Early and precise diagnosis of schizophrenia disorder (SZ) has an essential role in the quality of a patient\u2019s life and future treatments. Structural and functional neuroimaging provides robust biomarkers for understanding the anatomical and functional changes associated with SZ. Each of the neuroimaging techniques shows only a different perspective on the functional or structural of the brain, while multi-modal fusion can reveal latent connections in the brain. In this paper, we propose an approach for the fusion of structural and functional brain data with a deep learning-based model to take advantage of data fusion and increase the accuracy of schizophrenia disorder diagnosis. The proposed method consists of an architecture of 3D convolutional neural networks (CNNs) that applied to magnetic resonance imaging (MRI), functional magnetic resonance imaging (fMRI), and diffusion tensor imaging (DTI) extracted features. We use 3D MRI patches, fMRI spatial independent component analysis (ICA) map, and DTI fractional anisotropy (FA) as model inputs. Our method is validated on the COBRE dataset, and an average accuracy of 99.35% is obtained. The proposed method demonstrates promising classification performance and can be applied to real data.<\/jats:p>","DOI":"10.3233\/ida-205113","type":"journal-article","created":{"date-parts":[[2021,4,23]],"date-time":"2021-04-23T14:43:42Z","timestamp":1619189022000},"page":"527-540","source":"Crossref","is-referenced-by-count":12,"title":["Multi-modal neuroimaging feature fusion via 3D Convolutional Neural Network architecture for schizophrenia diagnosis"],"prefix":"10.1177","volume":"25","author":[{"given":"Babak","family":"Masoudi","sequence":"first","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sabalan","family":"Daneshvar","sequence":"additional","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran"},{"name":"Department of Electronic and Computer Engineering, College of Engineering, Design and Physical Sciences, Brunel University, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seyed Naser","family":"Razavi","sequence":"additional","affiliation":[{"name":"Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","reference":[{"issue":"9","key":"10.3233\/IDA-205113_ref1","doi-asserted-by":"publisher","first-page":"5195","DOI":"10.1007\/s00521-018-3360-1","article-title":"Detection of Schizophrenia in brain MR images based on segmented ventricle region and deep belief networks","volume":"31","author":"Latha","year":"2019","journal-title":"Neural Computing and Applications"},{"issue":"2","key":"10.3233\/IDA-205113_ref2","doi-asserted-by":"publisher","first-page":"525","DOI":"10.3389\/fnins.2018.00525","article-title":"Classification and prediction of brain disorders using functional connectivity: promising but challenging","volume":"12","author":"Du","year":"2018","journal-title":"Frontiers in Neuroscience"},{"issue":"2","key":"10.3233\/IDA-205113_ref3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0191202","article-title":"Combining multi-modality data for searching biomarkers in schizophrenia","volume":"13","author":"Guo","year":"2018","journal-title":"PLoS ONE"},{"issue":"9","key":"10.3233\/IDA-205113_ref4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/JBHI.2018.2796588","article-title":"Identifying resting-state multi-frequency biomarkers via tree-guided group sparse learning for schizophrenia classification","volume":"31","author":"Huang","year":"2018","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"10.3233\/IDA-205113_ref5","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.schres.2017.05.038","article-title":"Abnormal regional homogeneity as a potential imaging biomarker for adolescent-onset schizophrenia: a resting-state fMRI study and support vector machine analysis","volume":"192","author":"Wang","year":"2018","journal-title":"Schizophrenia Research"},{"key":"10.3233\/IDA-205113_ref6","doi-asserted-by":"publisher","DOI":"10.1038\/nn.4478"},{"key":"10.3233\/IDA-205113_ref7","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.jneumeth.2016.11.005","article-title":"A robust sparse-modeling framework for estimating schizophrenia biomarkers from fMRI","volume":"276","author":"Dillon","year":"2017","journal-title":"Journal of Neuroscience Methods"},{"issue":"3","key":"10.3233\/IDA-205113_ref8","doi-asserted-by":"publisher","first-page":"1550007","DOI":"10.1142\/S0129065715500070","article-title":"Discrimination of schizophrenia auditory hallucinators by machine learning of resting-state functional MRI","volume":"25","author":"Chyzhyk","year":"2015","journal-title":"International Journal of Neural Systems"},{"key":"10.3233\/IDA-205113_ref9","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1016\/j.neuroimage.2016.04.051","article-title":"Classification of schizophrenia and bipolar patients using static and dynamic resting-state fMRI brain connectivity","volume":"134","author":"Rashid","year":"2016","journal-title":"NeuroImage"},{"key":"10.3233\/IDA-205113_ref10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fninf.2017.00059","article-title":"Multimodal discrimination of schizophrenia using hybrid weighted feature concatenation of brain functional connectivity and anatomical features with an extreme learning machine","volume":"11","author":"Qureshi","year":"2017","journal-title":"Frontiers in Neuroinformatics"},{"issue":"4","key":"10.3233\/IDA-205113_ref11","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/j.bj.2017.05.002","article-title":"PET and MRI image fusion based on combination of 2-D Hilbert transform and IHS method","volume":"40","author":"Haddadpour","year":"2017","journal-title":"Biomedical Journa"},{"issue":"1","key":"10.3233\/IDA-205113_ref12","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1109\/JBHI.2017.2655720","article-title":"Multimodal neuroimaging feature learning with multimodal stacked deep polynomial networks for diagnosis of alzheimer\u2019s disease","volume":"22","author":"Shi","year":"2018","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"key":"10.3233\/IDA-205113_ref13","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.nicl.2016.01.002","article-title":"Combining anatomical, diffusion, and resting state functional magnetic resonance imaging for individual classification of mild and moderate Alzheimer\u2019s disease","volume":"11","author":"Schouten","year":"2016","journal-title":"NeuroImage: Clinical"},{"key":"10.3233\/IDA-205113_ref14","doi-asserted-by":"publisher","DOI":"10.3389\/fncom.2017.00117"},{"key":"10.3233\/IDA-205113_ref15","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0129250"},{"key":"10.3233\/IDA-205113_ref16","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2017.00380"},{"key":"10.3233\/IDA-205113_ref17","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1016\/j.cortex.2015.02.008","article-title":"Multimodal neuroimaging based classification of autism spectrum disorder using anatomical, neurochemical, and white matter correlates","volume":"66","author":"Libero","year":"2015","journal-title":"Cortex"},{"issue":"6","key":"10.3233\/IDA-205113_ref18","doi-asserted-by":"publisher","first-page":"895","DOI":"10.1007\/s10278-018-0093-8","article-title":"Combination of rs-fMRI and sMRI data to discriminate autism spectrum disorders in young children using deep belief network","volume":"31","author":"Akhavan Aghdam","year":"2018","journal-title":"Journal of Digital Imaging"},{"key":"10.3233\/IDA-205113_ref19","doi-asserted-by":"publisher","first-page":"101858","DOI":"10.1016\/j.nicl.2019.101858","article-title":"A totally data-driven whole-brain multimodal pipeline for the discrimination of Parkinson\u2019s disease, multiple system atrophy and healthy control","volume":"23","author":"Nemmi","year":"2019","journal-title":"NeuroImage: Clinical"},{"key":"10.3233\/IDA-205113_ref20","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2016.00466"},{"issue":"1","key":"10.3233\/IDA-205113_ref21","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","article-title":"Deep learning in medical image analysis","volume":"19","author":"Shen","year":"2017","journal-title":"Annual Review of Biomedical Engineering"},{"issue":"3\u20134","key":"10.3233\/IDA-205113_ref22","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1007\/s12021-018-9370-4","article-title":"Multi-modality cascaded convolutional neural networks for alzheimer\u2019s disease diagnosis","volume":"16","author":"Liu","year":"2018","journal-title":"Neuroinformatics"},{"issue":"2","key":"10.3233\/IDA-205113_ref23","doi-asserted-by":"publisher","first-page":"841","DOI":"10.1007\/s00429-013-0687-3","article-title":"Latent feature representation with stacked auto-encoder for AD\/MCI diagnosis","volume":"220","author":"Suk","year":"2015","journal-title":"Brain Structure and Function"},{"key":"10.3233\/IDA-205113_ref24","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1016\/j.ebiom.2018.03.017","article-title":"Multi-site diagnostic classification of schizophrenia using discriminant deep learning with functional connectivity MRI","volume":"30","author":"Zeng","year":"2018","journal-title":"EBioMedicine"},{"key":"10.3233\/IDA-205113_ref25","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.neuroimage.2015.05.018","article-title":"Deep neural network with weight sparsity control and pre-training extracts hierarchical features and enhances classification performance: evidence from whole-brain resting-state functional connectivity patterns of schizophrenia","volume":"124","author":"Kim","year":"2016","journal-title":"NeuroImage"},{"key":"10.3233\/IDA-205113_ref26","doi-asserted-by":"crossref","first-page":"584","DOI":"10.2741\/4606","article-title":"Alzheimer\u2019s disease diagnostics by a 3D deeply supervised adaptable convolutional network","volume":"23","author":"Hosseini-Asl","year":"2018","journal-title":"Front Biosci (Landmark Ed)"},{"key":"10.3233\/IDA-205113_ref27","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.neunet.2015.04.002","article-title":"Computer aided diagnosis of schizophrenia on resting state fMRI data by ensembles of ELM","volume":"68","author":"Chyzhyk","year":"2015","journal-title":"Neural Networks"},{"key":"10.3233\/IDA-205113_ref28","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1016\/j.neucom.2015.01.079","article-title":"Local activity features for computer aided diagnosis of schizophrenia on resting-state fMRI","volume":"164","author":"Savio","year":"2015","journal-title":"Neurocomputing"},{"issue":"2","key":"10.3233\/IDA-205113_ref29","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.braindev.2006.07.002","article-title":"Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI","volume":"29","author":"Yu-Feng","year":"2007","journal-title":"Brain and Development"},{"key":"10.3233\/IDA-205113_ref30","doi-asserted-by":"publisher","DOI":"10.1097\/MD.0000000000003973"},{"issue":"7","key":"10.3233\/IDA-205113_ref31","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1109\/TNB.2017.2751074","article-title":"Classification of schizophrenia based on individual hierarchical brain networks constructed from structural MRI images","volume":"16","author":"Liu","year":"2017","journal-title":"IEEE Transactions on NanoBioscience"},{"key":"10.3233\/IDA-205113_ref32","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.artmed.2019.06.003","article-title":"3D-CNN based discrimination of schizophrenia using resting-state fMRI","volume":"98","author":"Qureshi","year":"2019","journal-title":"Artificial Intelligence in Medicine"},{"issue":"6","key":"10.3233\/IDA-205113_ref33","doi-asserted-by":"publisher","first-page":"1326","DOI":"10.1093\/schbul\/sbv060","article-title":"Disintegration of sensorimotor brain networks in schizophrenia","volume":"41","author":"Kaufmann","year":"2015","journal-title":"Schizophr Bul"},{"issue":"6","key":"10.3233\/IDA-205113_ref35","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1111\/acps.12964","article-title":"Identifying a neuroanatomical signature of schizophrenia, reproducible across sites and stages, using machine learning with structured sparsity","volume":"138","author":"de Pierrefeu","year":"2018","journal-title":"Acta Psychiatrica Scandinavica"},{"issue":"2","key":"10.3233\/IDA-205113_ref36","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1016\/j.neuroimage.2008.04.255","article-title":"Application of principal component analysis to distinguish patients with schizophrenia from healthy controls based on fractional anisotropy measurements","volume":"42","author":"Caprihan","year":"2008","journal-title":"NeuroImage"},{"key":"10.3233\/IDA-205113_ref37","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1016\/J.PNPBP.2018.06.010","article-title":"Tractography-based classification in distinguishing patients with first-episode schizophrenia from healthy individuals","volume":"88","author":"Deng","year":"2019","journal-title":"Progress in Neuro-Psychopharmacology and Biological Psychiatry"},{"key":"10.3233\/IDA-205113_ref38","doi-asserted-by":"publisher","DOI":"10.1016\/j.ebiom.2019.08.023"},{"issue":"22","key":"10.3233\/IDA-205113_ref39","doi-asserted-by":"publisher","first-page":"29651","DOI":"10.1007\/s11042-017-5470-7","article-title":"MMM: classification of schizophrenia using multi-modality multi-atlas feature representation and multi-kernel learning","volume":"77","author":"Liu","year":"2018","journal-title":"Multimedia Tools and Applications"},{"key":"10.3233\/IDA-205113_ref40","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnhum.2013.00235","article-title":"Combination of resting state fMRI, DTI, and sMRI data to discriminate schizophrenia by n-way MCCa+jICA","volume":"7","author":"Sui","year":"2013","journal-title":"Frontiers in Human Neuroscience"},{"issue":"1","key":"10.3233\/IDA-205113_ref41","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1007\/s12021-014-9238-1","article-title":"Clinical prediction from structural brain MRI scans: a large-scale empirical study","volume":"13","author":"Sabuncu","year":"2015","journal-title":"Neuroinformatics"},{"issue":"1","key":"10.3233\/IDA-205113_ref42","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1109\/TPAMI.2012.59","article-title":"3D convolutional neural networks for human action recognition","volume":"35","author":"Ji","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"7","key":"10.3233\/IDA-205113_ref44","doi-asserted-by":"publisher","first-page":"1483","DOI":"10.1162\/neco.1997.9.7.1483","article-title":"A fast fixed-point algorithm for independent component analysis","volume":"9","author":"Hyv","year":"1997","journal-title":"Neural Comput"},{"issue":"1","key":"10.3233\/IDA-205113_ref46","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res"},{"key":"10.3233\/IDA-205113_ref47","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res"},{"key":"10.3233\/IDA-205113_ref48","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyt.2011.00075"},{"issue":"9","key":"10.3233\/IDA-205113_ref49","doi-asserted-by":"publisher","first-page":"2302","DOI":"10.1002\/hbm.22065","article-title":"Functional imaging of the hemodynamic sensory gating response in schizophrenia","volume":"34","author":"Mayer","year":"2013","journal-title":"Human Brain Mapping"},{"key":"10.3233\/IDA-205113_ref50","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2004.07.051"},{"issue":"3","key":"10.3233\/IDA-205113_ref52","doi-asserted-by":"publisher","first-page":"S169","DOI":"10.3233\/JAD-2012-120412","article-title":"Structural brain atlases: design, rationale, and applications in normal and pathological cohorts","volume":"3","author":"Mandal","year":"2012","journal-title":"Journal of Alzheimer\u2019s Disease: JAD 31 Suppl"},{"key":"10.3233\/IDA-205113_ref53","unstructured":"M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean et al., TensorFlow: a system for large-scale machine learning, in: Presented at the Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, Savannah, GA, USA, 2016."},{"issue":"13","key":"10.3233\/IDA-205113_ref54","doi-asserted-by":"publisher","first-page":"3795","DOI":"10.1002\/hbm.24632","article-title":"Parallel group ICA+ICA: joint estimation of linked functional network variability and structural covariation with application to schizophrenia","volume":"40","author":"Qi","year":"2019","journal-title":"Human Brain Mapping"},{"issue":"4","key":"10.3233\/IDA-205113_ref55","doi-asserted-by":"publisher","first-page":"1475","DOI":"10.1002\/hbm.23906","article-title":"Four-way multimodal fusion of 7 T imaging data using an mCCA+jICA model in first-episode schizophrenia","volume":"39","author":"Lottman","year":"2018","journal-title":"Human Brain Mapping"},{"key":"10.3233\/IDA-205113_ref56","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1016\/j.neuroimage.2015.07.054","article-title":"A group ICA based framework for evaluating resting fMRI markers when disease categories are unclear: application to schizophrenia, bipolar, and schizoaffective disorders","volume":"122","author":"Du","year":"2015","journal-title":"NeuroImage"},{"issue":"7","key":"10.3233\/IDA-205113_ref57","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1007\/s00234-017-1844-9","article-title":"Schizophrenia symptomatic associations with diffusion tensor imaging measured fractional anisotropy of brain: a meta-analysis","volume":"59","author":"Yang","year":"2017","journal-title":"Neuroradiology"},{"key":"10.3233\/IDA-205113_ref58","doi-asserted-by":"publisher","DOI":"10.1016\/j.pnpbp.2013.04.019"},{"key":"10.3233\/IDA-205113_ref59","doi-asserted-by":"publisher","first-page":"1494","DOI":"10.1093\/brain\/awq040","article-title":"Diffusion tensor tractography findings in schizophrenia across the adult lifespan","volume":"133","author":"Voineskos","year":"2010","journal-title":"Brain: A Journal of Neurology"},{"key":"10.3233\/IDA-205113_ref60","doi-asserted-by":"publisher","first-page":"1261","DOI":"10.1038\/mp.2017.170","article-title":"Widespread white matter microstructural differences in schizophrenia across 4322 individuals: results from the ENIGMA Schizophrenia DTI Working Group","volume":"23","author":"Kelly","year":"2017","journal-title":"Molecular Psychiatry"}],"container-title":["Intelligent Data Analysis"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/IDA-205113","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:19:03Z","timestamp":1777454343000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/IDA-205113"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,20]]},"references-count":56,"journal-issue":{"issue":"3"},"URL":"https:\/\/doi.org\/10.3233\/ida-205113","relation":{},"ISSN":["1088-467X","1571-4128"],"issn-type":[{"value":"1088-467X","type":"print"},{"value":"1571-4128","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,20]]}}}