{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,10]],"date-time":"2026-03-10T22:35:34Z","timestamp":1773182134025,"version":"3.50.1"},"reference-count":54,"publisher":"Wiley","issue":"17","license":[{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"},{"start":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T00:00:00Z","timestamp":1695772800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2112455"],"award-info":[{"award-number":["2112455"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01EB006841"],"award-info":[{"award-number":["R01EB006841"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006778","name":"Georgia Institute of Technology","doi-asserted-by":"publisher","award":["T32EB025816"],"award-info":[{"award-number":["T32EB025816"]}],"id":[{"id":"10.13039\/100006778","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Human Brain Mapping"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>This work proposes a novel generative multimodal approach to jointly analyze multimodal data while linking the multimodal information to colors. We apply our proposed framework, which disentangles multimodal data into private and shared sets of features from pairs of structural (sMRI), functional (sFNC and ICA), and diffusion MRI data (FA maps). With our approach, we find that heterogeneity in schizophrenia is potentially a function of modality pairs. Results show (1) schizophrenia is highly multimodal and includes changes in specific networks, (2) non\u2010linear relationships with schizophrenia are observed when interpolating among shared latent dimensions, and (3) we observe a decrease in the modularity of functional connectivity and decreased visual\u2010sensorimotor connectivity for schizophrenia patients for the FA\u2010sFNC and sMRI\u2010sFNC modality pairs, respectively. Additionally, our results generally indicate decreased fractional corpus callosum anisotropy, and decreased spatial ICA map and voxel\u2010based morphometry strength in the superior frontal lobe as found in the FA\u2010sFNC, sMRI\u2010FA, and sMRI\u2010ICA modality pair clusters. In sum, we introduce a powerful new multimodal neuroimaging framework designed to provide a rich and intuitive understanding of the data which we hope challenges the reader to think differently about how modalities interact.<\/jats:p>","DOI":"10.1002\/hbm.26479","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T05:28:52Z","timestamp":1695792532000},"page":"5828-5845","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Chromatic fusion: Generative multimodal neuroimaging data fusion provides multi\u2010informed insights into schizophrenia"],"prefix":"10.1002","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5448-6358","authenticated-orcid":false,"given":"Eloy P.T.","family":"Geenjaar","sequence":"first","affiliation":[{"name":"School of Electrical and Computer Engineering Georgia Institute of Technology  Atlanta Georgia USA"},{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5712-2434","authenticated-orcid":false,"given":"Noah L.","family":"Lewis","sequence":"additional","affiliation":[{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"},{"name":"School of Computational Science and Engineering Georgia Institute of Technology  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alex","family":"Fedorov","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering Georgia Institute of Technology  Atlanta Georgia USA"},{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9705-1829","authenticated-orcid":false,"given":"Lei","family":"Wu","sequence":"additional","affiliation":[{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Judith M.","family":"Ford","sequence":"additional","affiliation":[{"name":"San Francisco Veterans Affairs Medical Center  San Francisco California USA"},{"name":"Department of Psychiatry and Behavioral Sciences University of California San Francisco  San Francisco California USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adrian","family":"Preda","sequence":"additional","affiliation":[{"name":"Department of Psychiatry and Human Behavior University of California Irvine  Irvine California USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sergey M.","family":"Plis","sequence":"additional","affiliation":[{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"},{"name":"Department of Computer Science Georgia State University  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Vince D.","family":"Calhoun","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering Georgia Institute of Technology  Atlanta Georgia USA"},{"name":"Tri\u2010Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State, Georgia Tech, Emory  Atlanta Georgia USA"},{"name":"School of Computational Science and Engineering Georgia Institute of Technology  Atlanta Georgia USA"},{"name":"Department of Computer Science Georgia State University  Atlanta Georgia USA"},{"name":"Department of Psychology Georgia State University  Atlanta Georgia USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2023,9,27]]},"reference":[{"key":"e_1_2_9_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2015.10.019"},{"issue":"365","key":"e_1_2_9_3_1","first-page":"1","article-title":"Advanced normalization tools (ANTS)","volume":"2","author":"Avants B. B.","year":"2009","journal-title":"Insight j"},{"key":"e_1_2_9_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2798607"},{"key":"e_1_2_9_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/S2215-0366(17)30333-4"},{"key":"e_1_2_9_6_1","doi-asserted-by":"publisher","DOI":"10.1097\/00001504-200503000-00008"},{"key":"e_1_2_9_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.bpsc.2015.12.005"},{"key":"e_1_2_9_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2008.10.057"},{"key":"e_1_2_9_9_1","doi-asserted-by":"publisher","DOI":"10.1089\/brain.2011.0015"},{"key":"e_1_2_9_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.schres.2015.06.014"},{"key":"e_1_2_9_11_1","unstructured":"Clevert D. A. Unterthiner T. &Hochreiter S.(2015).Fast and accurate deep network learning by exponential linear units (elus). arXiv Preprint arXiv:151107289."},{"key":"e_1_2_9_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nicl.2014.07.003"},{"key":"e_1_2_9_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.nicl.2020.102375"},{"key":"e_1_2_9_14_1","unstructured":"Fedorov A. Geenjaar E. Wu L. DeRamus T. P. Calhoun V. D. &Plis S. M.(2021).Tasting the cake: Evaluating self\u2010supervised generalization on out\u2010of\u2010distribution multimodal MRI data. arXiv preprint arXiv:210315914."},{"key":"e_1_2_9_15_1","article-title":"Self\u2010supervised multimodal domino: In search of biomarkers for alzheimer's disease","author":"Fedorov A.","year":"2021","journal-title":"2021 IEEE 9th International Conference on Health Informatics (ICHI)"},{"key":"e_1_2_9_16_1","volume-title":"Structured clinical interview for DSM\u2010IV\u2010TR axis I disorders, research version","author":"First M. B.","year":"2002"},{"key":"e_1_2_9_17_1","doi-asserted-by":"publisher","DOI":"10.1093\/schbul\/sbu031"},{"key":"e_1_2_9_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.schres.2008.12.011"},{"key":"e_1_2_9_19_1","doi-asserted-by":"publisher","DOI":"10.1176\/ajp.2007.164.3.450"},{"key":"e_1_2_9_20_1","doi-asserted-by":"publisher","DOI":"10.1049\/cp:19991075"},{"key":"e_1_2_9_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2020.3013825"},{"key":"e_1_2_9_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.05.079"},{"key":"e_1_2_9_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2015.09.003"},{"key":"e_1_2_9_24_1","first-page":"165","volume-title":"Independent vector analysis: An extension of ICA to multivariate components. In: Independent component analysis and blind signal separation: 6th international conference, ICA 2006","author":"Kim T.","year":"2006"},{"key":"e_1_2_9_25_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.1412.6980"},{"key":"e_1_2_9_26_1","volume-title":"2nd international conference on learning representations, ICLR 2014\u2014Conference track proceedings","author":"Kingma D. P.","year":"2014"},{"key":"e_1_2_9_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00406-009-0087-0"},{"key":"e_1_2_9_28_1","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_2_9_29_1","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800030404"},{"key":"e_1_2_9_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2007.11.019"},{"key":"e_1_2_9_31_1","first-page":"1692","article-title":"Private\u2010shared disentangled multimodal VAE for learning of latent representations","author":"Lee M.","year":"2021","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops"},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1097\/01.wnr.0000198434.06518.b8"},{"key":"e_1_2_9_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40708-015-0019-x"},{"key":"e_1_2_9_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2017.123"},{"key":"e_1_2_9_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.12.063"},{"key":"e_1_2_9_36_1","unstructured":"Peide L. Sofuoglu S. E. Maiti T. &Aviyente S.(2022).Coupled support tensor machine classification for multimodal neuroimaging data. arXiv preprint arXiv:220107683."},{"key":"e_1_2_9_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2018.07.047"},{"key":"e_1_2_9_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2017.2725306"},{"key":"e_1_2_9_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/hbm.24632"},{"key":"e_1_2_9_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2019.02.024"},{"key":"e_1_2_9_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11682-016-9666-7"},{"key":"e_1_2_9_42_1","doi-asserted-by":"publisher","DOI":"10.1176\/ajp.2007.164.3.467"},{"key":"e_1_2_9_43_1","article-title":"Variational mixture\u2010of\u2010experts autoencoders for multi\u2010modal deep generative models","volume":"32","author":"Shi Y.","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_9_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3028452"},{"issue":"1","key":"e_1_2_9_45_1","first-page":"1929","article-title":"Dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava N.","year":"2014","journal-title":"The Journal of Machine Learning Research"},{"key":"e_1_2_9_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2012.10.051"},{"key":"e_1_2_9_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopsych.2020.02.016"},{"issue":"2","key":"e_1_2_9_48_1","first-page":"240","article-title":"Depression and schizophrenia: Cause, consequence, or trans\u2010diagnostic issue?","volume":"43","author":"Upthegrove R.","year":"2017","journal-title":"Schizophrenia Bulletin"},{"issue":"11","key":"e_1_2_9_49_1","first-page":"2579","article-title":"Visualizing data using t\u2010SNE","volume":"9","author":"Maaten L.","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_9_50_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-019-0686-2"},{"key":"e_1_2_9_51_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1440-1614.2005.01678.x"},{"key":"e_1_2_9_52_1","doi-asserted-by":"publisher","DOI":"10.1002\/hbm.22945"},{"key":"e_1_2_9_53_1","doi-asserted-by":"crossref","unstructured":"Wu Y. &He K.(2018).Group normalization. Proceedings of the European conference on computer vision (ECCV) pp.3\u201319.","DOI":"10.1007\/978-3-030-01261-8_1"},{"key":"e_1_2_9_54_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnsys.2011.00103"},{"key":"e_1_2_9_55_1","first-page":"613","volume-title":"Deep representation learning for multimodal brain networks","author":"Zhang W.","year":"2020"}],"container-title":["Human Brain Mapping"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/hbm.26479","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/hbm.26479","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,1]],"date-time":"2023-11-01T13:07:44Z","timestamp":1698844064000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/hbm.26479"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,27]]},"references-count":54,"journal-issue":{"issue":"17","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10.1002\/hbm.26479"],"URL":"https:\/\/doi.org\/10.1002\/hbm.26479","archive":["Portico"],"relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.05.18.23290184","asserted-by":"object"}]},"ISSN":["1065-9471","1097-0193"],"issn-type":[{"value":"1065-9471","type":"print"},{"value":"1097-0193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,27]]},"assertion":[{"value":"2023-03-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-24","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}