{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T16:09:52Z","timestamp":1753891792653,"version":"3.41.2"},"reference-count":64,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neuroinform."],"abstract":"<jats:p>Although the anatomical arrangement of brain regions and the functional structures within them are similar across individuals, the representation of neural information, such as recorded brain activity, varies among individuals owing to various factors. Therefore, appropriate conversion and translation of brain information is essential when decoding neural information using a model trained using another person\u2019s data or to achieving brain-to-brain communication. We propose a brain representation transfer method that involves transforming a data representation obtained from one person\u2019s brain into that obtained from another person\u2019s brain, without relying on corresponding label information between the transferred datasets. We defined the requirements to enable such brain representation transfer and developed an algorithm that distills the assumption of common similarity structure across the brain datasets into a rotational and reflectional transformation across low-dimensional hyperspheres using encoders for non-linear dimensional reduction. We first validated our proposed method using data from artificial neural networks as substitute neural activity and examining various experimental factors. We then evaluated the applicability of our method to real brain activity using functional magnetic resonance imaging response data acquired from human participants. The results of these validation experiments showed that our method successfully performed representation transfer and achieved transformations in some cases that were similar to those obtained when using corresponding label information. Additionally, we reconstructed images from individuals\u2019 data without training personalized decoders by performing brain representation transfer. The results suggest that our unsupervised transfer method is useful for the reapplication of existing models personalized to specific participants and datasets to decode brain information from other individuals. Our findings also serve as a proof of concept for the methodology, enabling the exchange of the latent properties of neural information representing individuals\u2019 sensations.<\/jats:p>","DOI":"10.3389\/fninf.2024.1470845","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T06:35:27Z","timestamp":1732775727000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Unsupervised method for representation transfer from one brain to another"],"prefix":"10.3389","volume":"18","author":[{"given":"Daiki","family":"Nakamura","sequence":"first","affiliation":[]},{"given":"Shizuo","family":"Kaji","sequence":"additional","affiliation":[]},{"given":"Ryota","family":"Kanai","sequence":"additional","affiliation":[]},{"given":"Ryusuke","family":"Hayashi","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","first-page":"1881","DOI":"10.18653\/v1\/D18-1214","article-title":"Gromov-wasserstein alignment of word embedding spaces","author":"Alvarez-Melis","year":"2018","journal-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"},{"key":"B2","doi-asserted-by":"publisher","first-page":"704","DOI":"10.1162\/089892903322307429","article-title":"Patterns of activity in the categorical representations of objects.","volume":"15","author":"Carlson","year":"2003","journal-title":"J. Cogn. Neurosc."},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2002.05709","article-title":"A simple framework for contrastive learning of visual representations.","author":"Chen","year":"2020","journal-title":"arxiv"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2021.655019","article-title":"Research on the modality transfer method of brain imaging based on generative adversarial network.","volume":"15","author":"Cheng","year":"2021","journal-title":"Front. Neurosci."},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1038\/srep27755","article-title":"Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence.","volume":"6","author":"Cichy","year":"2016","journal-title":"Sci. Rep."},{"key":"B6","doi-asserted-by":"crossref","first-page":"2608","DOI":"10.1523\/JNEUROSCI.5547-11.2012","article-title":"The representation of biological classes in the human brain.","volume":"32","author":"Connolly","year":"2012","journal-title":"J. Neurosci."},{"key":"B7","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1109\/TIT.1967.1053964","article-title":"Nearest neighbor pattern classification.","volume":"13","author":"Cover","year":"1967","journal-title":"IEEE Trans. Inf. Theory"},{"key":"B8","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/BF02569044","article-title":"Gordon holmes, the cortical retina, and the wounds of war.","volume":"93","author":"Fishman","year":"1997","journal-title":"Documenta Ophthalmol."},{"key":"B9","doi-asserted-by":"publisher","first-page":"238","DOI":"10.2307\/1403797","article-title":"Discriminatory analysis. nonparametric discrimination: Consistency properties.","volume":"57","author":"Fix","year":"1989","journal-title":"Int. Stat. Rev."},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0105225","article-title":"Conscious brain-to-brain communication in humans using non-invasive technologies.","volume":"9","author":"Grau","year":"2014","journal-title":"PLoS One"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1126\/science.1063736","article-title":"Distributed and overlapping representations of faces and objects in ventral temporal cortex.","volume":"293","author":"Haxby","year":"2001","journal-title":"Science"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.56601","article-title":"Hyperalignment: Modeling shared information encoded in idiosyncratic cortical topographies.","volume":"9","author":"Haxby","year":"2020","journal-title":"ELife"},{"key":"B13","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/SMC.2018.00028","article-title":"Image reconstruction from neural activity recorded from monkey inferior temporal cortex using generative adversarial networks","author":"Hayashi","year":"2018","journal-title":"Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC)"},{"key":"B14","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","journal-title":"Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770\u2013778"},{"key":"B15","doi-asserted-by":"publisher","first-page":"482","DOI":"10.1016\/j.neuroimage.2011.04.056","article-title":"Transformation from image-based to perceptual representation of materials along the human ventral visual pathway.","volume":"57","author":"Hiramatsu","year":"2011","journal-title":"NeuroImage"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.11239","article-title":"Denoising diffusion probabilistic models.","author":"Ho","year":"2020","journal-title":"arxiv"},{"key":"B17","doi-asserted-by":"publisher","first-page":"321","DOI":"10.2307\/2333955","article-title":"Relations between two sets of variates.","volume":"28","author":"Hotelling","year":"1936","journal-title":"Biometrika"},{"key":"B18","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1113\/jphysiol.1962.sp006837","article-title":"Receptive fields, binocular interaction and functional architecture in the cat\u2019s visual cortex.","volume":"160","author":"Hubel","year":"1962","journal-title":"J. Physiol."},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-41895-7","article-title":"BrainNet: A multi-person brain-to-brain interface for direct collaboration between brains.","volume":"9","author":"Jiang","year":"2019","journal-title":"Sci. Rep."},{"key":"B20","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1038\/nn1444","article-title":"Decoding the visual and subjective contents of the human brain.","volume":"8","author":"Kamitani","year":"2005","journal-title":"Nat. Neurosci."},{"key":"B21","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1007\/s00221-004-1992-0","article-title":"Transcranial magnetic stimulation in the visual system. II. Characterization of induced phosphenes and scotomas.","volume":"160","author":"Kammer","year":"2005","journal-title":"Exp. Brain Res."},{"key":"B22","doi-asserted-by":"publisher","first-page":"4217","DOI":"10.1109\/TPAMI.2020.2970919","article-title":"A style-based generator architecture for generative adversarial networks.","volume":"43","author":"Karras","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2107.00630","article-title":"Variational diffusion models.","author":"Kingma","year":"2021","journal-title":"arxiv"},{"key":"B24","doi-asserted-by":"crossref","first-page":"408","DOI":"10.1523\/JNEUROSCI.12-02-00408.1992","article-title":"Color selectivity of neurons in the inferior temporal cortex of the awake macaque monkey.","volume":"12","author":"Komatsu","year":"1992","journal-title":"J. Neurosci."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational similarity analysis - connecting the branches of systems neuroscience.","volume":"2","author":"Kriegeskorte","year":"","journal-title":"Front. Syst. Neurosci."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1126","DOI":"10.1016\/j.neuron.2008.10.043","article-title":"Matching categorical object representations in inferior temporal cortex of man and monkey.","volume":"60","author":"Kriegeskorte","year":"","journal-title":"Neuron"},{"journal-title":"Learning Multiple Layers of Features from Tiny Images.","year":"2009","author":"Krizhevsky","key":"B27"},{"key":"B28","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/BF02289694","article-title":"Nonmetric multidimensional scaling: A numerical method.","volume":"29","author":"Kruskal","year":"","journal-title":"Psychometrika"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/BF02289565","article-title":"Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis.","volume":"29","author":"Kruskal","year":"","journal-title":"Psychometrika"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0178476","article-title":"Non-invasive transmission of sensorimotor information in humans using an EEG\/focused ultrasound brain-to-brain interface.","volume":"12","author":"Lee","year":"2017","journal-title":"PLoS One"},{"key":"B31","doi-asserted-by":"publisher","first-page":"208","DOI":"10.1016\/j.brainres.2015.08.038","article-title":"Electrical stimulation of the brain and the development of cortical visual prostheses: An historical perspective.","volume":"1630","author":"Lewis","year":"2016","journal-title":"Brain Res."},{"key":"B32","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1016\/j.neuron.2009.02.025","article-title":"Timing, timing, timing: Fast decoding of object information from intracranial field potentials in human visual cortex.","volume":"62","author":"Liu","year":"2009","journal-title":"Neuron"},{"key":"B33","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1016\/j.ejor.2005.09.032","article-title":"A survey for the quadratic assignment problem","volume":"176","author":"Loiola","year":"2007","journal-title":"Eur. J. Oper. Res"},{"key":"B34","first-page":"2579","article-title":"Visualizing data using t-SNE.","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Machine Learn. Res."},{"key":"B35","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1093\/schbul\/sbac157","article-title":"Disorganization of semantic brain networks in schizophrenia revealed by fMRI.","volume":"49","author":"Matsumoto","year":"2023","journal-title":"Schizophrenia Bull."},{"key":"B36","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1007\/s10208-011-9093-5","article-title":"Gromov\u2013wasserstein distances and the metric approach to object matching.","volume":"11","author":"M\u00e9moli","year":"2011","journal-title":"Foundations Comput. Math."},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2023.120079","article-title":"Inter-individual single-trial classification of MEG data using M-CCA.","volume":"273","author":"Michalke","year":"2023","journal-title":"NeuroImage"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1310.4546","article-title":"Distributed representations of words and phrases and their compositionality.","author":"Mikolov","year":"2013","journal-title":"arxiv"},{"key":"B39","doi-asserted-by":"publisher","first-page":"915","DOI":"10.1016\/j.neuron.2008.11.004","article-title":"Visual image reconstruction from human brain activity using a combination of multiscale local image decoders.","volume":"60","author":"Miyawaki","year":"2008","journal-title":"Neuron"},{"key":"B40","first-page":"195","author":"Nielsen","year":"2016","journal-title":"Hierarchical Clustering. In Introduction to HPC with MPI for Data Science."},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1101\/2020.06.03.132928","article-title":"Reduced intra- and inter-individual diversity of semantic representations in the brains of schizophrenia patients.","author":"Nishida","year":"2020","journal-title":"BioRxiv"},{"key":"B42","doi-asserted-by":"publisher","first-page":"1641","DOI":"10.1016\/j.cub.2011.08.031","article-title":"Reconstructing visual experiences from brain activity evoked by natural movies.","volume":"21","author":"Nishimoto","year":"2011","journal-title":"Curr. Biol."},{"key":"B43","doi-asserted-by":"publisher","first-page":"E351","DOI":"10.1073\/pnas.1415146112","article-title":"Image statistics underlying natural texture selectivity of neurons in macaque V4.","volume":"112","author":"Okazawa","year":"2015","journal-title":"Proc. Natl. Acad. Sci."},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1807.03748","article-title":"Representation learning with contrastive predictive coding.","author":"Oord","year":"2018","journal-title":"arxiv"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-42891-8","article-title":"Natural scene reconstruction from fMRI signals using generative latent diffusion.","volume":"13","author":"Ozcelik","year":"2023","journal-title":"Sci. Rep."},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1038\/srep01319","article-title":"A Brain-to-brain interface for real-time sharing of sensorimotor information.","volume":"3","author":"Pais-Vieira","year":"2013","journal-title":"Sci. Rep."},{"key":"B47","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1093\/brain\/60.4.389","article-title":"Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation.","volume":"60","author":"Penfield","year":"1937","journal-title":"Brain"},{"key":"B48","doi-asserted-by":"crossref","first-page":"1809","DOI":"10.1109\/TPAMI.2009.184","article-title":"Kernelized Sorting.","volume":"32","author":"Quadrianto","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B49","doi-asserted-by":"publisher","first-page":"1339","DOI":"10.1126\/science.7079770","article-title":"Tonotopic organization of the human auditory cortex.","volume":"216","author":"Romani","year":"1982","journal-title":"Science"},{"key":"B50","doi-asserted-by":"publisher","first-page":"10674","DOI":"10.1088\/1361-6560\/ad229e","article-title":"High-resolution image synthesis with latent diffusion models","author":"Rombach","year":"2022","journal-title":"Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022-June"},{"key":"B51","doi-asserted-by":"publisher","first-page":"4713","DOI":"10.1109\/TPAMI.2022.3204461","article-title":"Image super-resolution via iterative refinement.","volume":"45","author":"Saharia","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.3389\/fncom.2019.00021","article-title":"End-to-end deep image reconstruction from human brain activity.","volume":"13","author":"Shen","year":"","journal-title":"Front. Comput. Neurosci."},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1006633","article-title":"Deep image reconstruction from human brain activity.","volume":"15","author":"Shen","year":"","journal-title":"PLoS Comput. Biol."},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1503.03585","article-title":"Deep unsupervised learning using nonequilibrium thermodynamics.","author":"Sohl-Dickstein","year":"2015","journal-title":"arxiv"},{"key":"B55","doi-asserted-by":"crossref","first-page":"14453","DOI":"10.1109\/CVPR52729.2023.01389","article-title":"High-resolution image reconstruction with latent diffusion models from human brain activity","author":"Takagi","year":"2023","journal-title":"Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B56","first-page":"1","article-title":"Self-supervised learning facilitates neural representation structures that can be unsupervisedly aligned to human behaviors","author":"Takahashi","year":"2024","journal-title":"Proceedings of the ICLR 2024 Workshop on Representational Alignment"},{"key":"B57","first-page":"1","article-title":"Unsupervised alignment reveals structural commonalities and differences in neural representations of natural scenes across individuals and brain areas","author":"Takeda","year":"2024","journal-title":"Proceedings of the ICLR 2024 Workshop on Representational Alignment"},{"key":"B58","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1098\/rstb.2005.1628","article-title":"Visual field map clusters in human cortex.","volume":"360","author":"Wandell","year":"2005","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"B59","first-page":"9871","article-title":"Understanding contrastive representation learning through alignment and uniformity on the hypersphere","author":"Wang","year":"2020","journal-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, PartF16814"},{"key":"B60","doi-asserted-by":"publisher","first-page":"4136","DOI":"10.1093\/cercor\/bhx268","article-title":"Neural encoding and decoding with deep learning for dynamic natural vision.","volume":"28","author":"Wen","year":"2018","journal-title":"Cereb. Cortex"},{"key":"B61","first-page":"3733","article-title":"Unsupervised feature learning via non-parametric instance discrimination","author":"Wu","year":"2018","journal-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition"},{"key":"B62","first-page":"807","article-title":"Cross-domain object matching with model selection","volume":"15","author":"Yamada","year":"2011","journal-title":"Proc. Mach. Learn. Res"},{"key":"B63","doi-asserted-by":"publisher","first-page":"1267","DOI":"10.1016\/j.neuroimage.2011.02.058","article-title":"Focused ultrasound modulates region-specific brain activity.","volume":"56","author":"Yoo","year":"2011","journal-title":"NeuroImage"},{"key":"B64","doi-asserted-by":"crossref","first-page":"2242","DOI":"10.1109\/ICCV.2017.244","article-title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","author":"Zhu","year":"2017","journal-title":"Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), 2017-Octob"}],"container-title":["Frontiers in Neuroinformatics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2024.1470845\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T06:35:41Z","timestamp":1732775741000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fninf.2024.1470845\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":64,"alternative-id":["10.3389\/fninf.2024.1470845"],"URL":"https:\/\/doi.org\/10.3389\/fninf.2024.1470845","relation":{},"ISSN":["1662-5196"],"issn-type":[{"type":"electronic","value":"1662-5196"}],"subject":[],"published":{"date-parts":[[2024,11,28]]},"article-number":"1470845"}}