{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T10:28:07Z","timestamp":1787567287869,"version":"build-2736575974"},"reference-count":135,"publisher":"eLife Sciences Publications, Ltd","license":[{"start":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T00:00:00Z","timestamp":1776384000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T00:00:00Z","timestamp":1776384000000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T00:00:00Z","timestamp":1776384000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Research Council","award":["Advanced grant GLoW (101096017)"],"award-info":[{"award-number":["Advanced grant GLoW (101096017)"]}]},{"award":["Advanced grant GLoW (101096017)"],"award-info":[{"award-number":["Advanced grant GLoW (101096017)"]}],"id":[{"id":"https:\/\/ror.org\/0472cxd90","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["AI-REPS (ANR-18-CE37-0007-01)"],"award-info":[{"award-number":["AI-REPS (ANR-18-CE37-0007-01)"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]},{"award":["AI-REPS (ANR-18-CE37-0007-01)"],"award-info":[{"award-number":["AI-REPS (ANR-18-CE37-0007-01)"]}],"id":[{"id":"https:\/\/ror.org\/00rbzpz17","id-type":"ROR","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001665","name":"Agence Nationale de la Recherche","doi-asserted-by":"crossref","award":["ANITI (ANR-19-PI3A-0004)"],"award-info":[{"award-number":["ANITI (ANR-19-PI3A-0004)"]}],"id":[{"id":"10.13039\/501100001665","id-type":"DOI","asserted-by":"crossref"}]},{"award":["ANITI (ANR-19-PI3A-0004)"],"award-info":[{"award-number":["ANITI (ANR-19-PI3A-0004)"]}],"id":[{"id":"https:\/\/ror.org\/00rbzpz17","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":["elifesciences.org"],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>Humans perform tasks involving the manipulation of inputs regardless of how these signals are perceived by the brain, thanks to representations that are invariant to the stimulus modality. In this paper, we present modality-agnostic decoders that leverage such modality-invariant representations to predict which stimulus a subject is seeing, irrespective of the modality in which the stimulus is presented. Training these modality-agnostic decoders is made possible thanks to our new large-scale fMRI dataset SemReps-8K, released publicly along with this paper. It comprises six subjects watching both images and short text descriptions of such images, as well as the conditions during which the subjects were imagining visual scenes. We find that modality-agnostic decoders can perform as well as modality-specific decoders and even outperform them when decoding captions and mental imagery. Furthermore, a searchlight analysis revealed that large areas of the brain contain modality-invariant representations. Such areas are also particularly suitable for decoding visual scenes from the mental imagery condition.<\/jats:p>","DOI":"10.7554\/elife.107933","type":"journal-article","created":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T15:31:02Z","timestamp":1756913462000},"update-policy":"https:\/\/doi.org\/10.7554\/elife.107933","source":"Crossref","is-referenced-by-count":0,"title":["Modality-agnostic decoding of vision and language from fMRI"],"prefix":"10.7554","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5609-6628","authenticated-orcid":true,"given":"Mitja","family":"Nikolaus","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01ahyrz84","id-type":"ROR","asserted-by":"publisher"}],"name":"Universit\u00e9 de Toulouse, CNRS, CerCo","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"corresponding-author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4521-1640","authenticated-orcid":true,"given":"Milad","family":"Mozafari","sequence":"first","affiliation":[{"name":"Torus AI","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"given":"Isabelle","family":"Berry","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01ahyrz84","id-type":"ROR","asserted-by":"publisher"}],"name":"Universit\u00e9 de Toulouse, CNRS, CerCo","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"data-curation","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"}]},{"given":"Nicholas","family":"Asher","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01ahyrz84","id-type":"ROR","asserted-by":"publisher"}],"name":"Universit\u00e9 de Toulouse, IRIT","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]},{"given":"Leila","family":"Reddy","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01ahyrz84","id-type":"ROR","asserted-by":"publisher"}],"name":"Universit\u00e9 de Toulouse, CNRS, CerCo","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"resources","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3611-7716","authenticated-orcid":true,"given":"Rufin","family":"VanRullen","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01ahyrz84","id-type":"ROR","asserted-by":"publisher"}],"name":"Universit\u00e9 de Toulouse, CNRS, CerCo","place":["Toulouse, France"]}],"role":[{"role":"author","vocabulary":"crossref"},{"role":"conceptualization","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"funding-acquisition","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"4374","published-online":{"date-parts":[[2026,4,17]]},"reference":[{"key":"bib1","doi-asserted-by":"publisher","first-page":"1427","DOI":"10.1016\/j.cub.2013.05.065","type":"journal-article","article-title":"Shared representations for working memory and mental imagery in early visual cortex","volume":"23","author":"Albers","year":"2013","journal-title":"Current Biology"},{"key":"bib2","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-00680-2","type":"journal-article","article-title":"A naturalistic neuroimaging database for understanding the brain using ecological stimuli","volume":"7","author":"Aliko","year":"2020","journal-title":"Scientific Data"},{"key":"bib3","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/s41593-021-00962-x","type":"journal-article","article-title":"A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence","volume":"25","author":"Allen","year":"2022","journal-title":"Nature Neuroscience"},{"key":"bib4","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1111\/tops.12096","type":"journal-article","article-title":"Reconciling embodied and distributional accounts of meaning in language","volume":"6","author":"Andrews","year":"2014","journal-title":"Topics in Cognitive Science"},{"key":"bib5","volume-title":"SPM12","author":"Ashburner","year":"2014","type":"book"},{"key":"bib6","volume-title":"A Cognitive Theory of Consciousness","author":"Baars","year":"1993","type":"book"},{"key":"bib7","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/S0079-6123(05)50004-9","type":"book-chapter","article-title":"Global workspace theory of consciousness: toward a cognitive neuroscience of human experience","volume-title":"In Progress in Brain Research","author":"Baars","year":"2005"},{"key":"bib8","doi-asserted-by":"publisher","first-page":"1122","DOI":"10.3758\/s13423-016-1028-3","type":"journal-article","article-title":"On staying grounded and avoiding quixotic dead ends","volume":"23","author":"Barsalou","year":"2016","journal-title":"Psychonomic Bulletin & Review"},{"key":"bib9","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/s0010-9452(08)70665-8","type":"journal-article","article-title":"The relationship between visual perception and visual mental imagery: a reappraisal of the neuropsychological evidence","volume":"38","author":"Bartolomeo","year":"2002","journal-title":"Cortex; a Journal Devoted to the Study of the Nervous System and Behavior"},{"key":"bib10","type":"conference-paper","article-title":"From voxels to pixels and back: self-supervision in natural-image reconstruction from fmri","author":"Beliy","year":"2019","unstructured":"Beliy R, Gaziv G, Hoogi A, Strappini F, Golan T, Irani M. 2019. From voxels to pixels and back: self-supervision in natural-image reconstruction from fmri. Proceedings of the 33rd International Conference on Neural Information Processing Systems."},{"key":"bib11","type":"conference-paper","article-title":"Brain decoding: toward real-time reconstruction of visual perception","author":"Benchetrit","year":"2024","unstructured":"Benchetrit Y, Banville H, King JR. 2024. Brain decoding: toward real-time reconstruction of visual perception. In The Twelfth International Conference on Learning Representations."},{"key":"bib12","doi-asserted-by":"publisher","first-page":"2767","DOI":"10.1093\/cercor\/bhp055","type":"journal-article","article-title":"Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies","volume":"19","author":"Binder","year":"2009","journal-title":"Cerebral Cortex"},{"key":"bib13","doi-asserted-by":"publisher","first-page":"1096","DOI":"10.3758\/s13423-015-0909-1","type":"journal-article","article-title":"In defense of abstract conceptual representations","volume":"23","author":"Binder","year":"2016","journal-title":"Psychonomic Bulletin & Review"},{"key":"bib14","doi-asserted-by":"publisher","first-page":"2449","DOI":"10.1002\/hbm.24535","type":"journal-article","article-title":"The dynamic contribution of the high-level visual cortex to imagery and perception","volume":"40","author":"Boccia","year":"2019","journal-title":"Human Brain Mapping"},{"key":"bib15","type":"conference-paper","article-title":"The Courtois project on neuronal modelling-first data release","author":"Boyle","year":"2020","unstructured":"Boyle JA, Pinsard B, Boukhdhir A, Belleville S, Brambatti S, Chen J, Cohen-Adad J, Cyr A, Fuente Rainville P, Bellec P. 2020. The Courtois project on neuronal modelling-first data release. In 26th annual meeting of the organization for human brain mapping."},{"key":"bib16","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0207741","type":"journal-article","article-title":"Hierarchical structure guides rapid linguistic predictions during naturalistic listening","volume":"14","author":"Brennan","year":"2019","journal-title":"PLOS ONE"},{"key":"bib17","volume-title":"Language and the Brain: A Slim Guide to Neurolinguistics","author":"Brennan","year":"2022","type":"book"},{"key":"bib18","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1016\/j.bandl.2004.01.010","type":"journal-article","article-title":"Unitary vs multiple semantics: PET studies of word and picture processing","volume":"89","author":"Bright","year":"2004","journal-title":"Brain and Language"},{"key":"bib19","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2020.117408","type":"journal-article","article-title":"Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals","volume":"224","author":"Carota","year":"2021","journal-title":"NeuroImage"},{"key":"bib20","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0052-3","type":"journal-article","article-title":"BOLD5000, a public fMRI dataset while viewing 5000 visual images","volume":"6","author":"Chang","year":"2019","journal-title":"Scientific Data"},{"key":"bib21","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/s1053-8119(03)00049-1","type":"journal-article","article-title":"Functional magnetic resonance imaging (fMRI) \u201cbrain reading\u201d: detecting and classifying distributed patterns of fMRI activity in human visual cortex","volume":"19","author":"Cox","year":"2003","journal-title":"NeuroImage"},{"key":"bib22","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1162\/neco.1989.1.1.123","type":"journal-article","article-title":"The brain binds entities and events by multiregional activation from convergence zones","volume":"1","author":"Damasio","year":"1989","journal-title":"Neural Computation"},{"key":"bib23","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.cognition.2002.07.001","type":"journal-article","article-title":"Neural systems behind word and concept retrieval","volume":"92","author":"Damasio","year":"2004","journal-title":"Cognition"},{"key":"bib24","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1016\/j.neuroimage.2006.01.021","type":"journal-article","article-title":"An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest","volume":"31","author":"Desikan","year":"2006","journal-title":"NeuroImage"},{"key":"bib25","doi-asserted-by":"publisher","first-page":"18906","DOI":"10.1523\/JNEUROSCI.3809-13.2013","type":"journal-article","article-title":"Representational similarity analysis reveals commonalities and differences in the semantic processing of words and objects","volume":"33","author":"Devereux","year":"2013","journal-title":"The Journal of Neuroscience"},{"key":"bib26","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1006\/nimg.2000.0595","type":"journal-article","article-title":"Susceptibility-induced loss of signal: comparing PET and fMRI on a semantic task","volume":"11","author":"Devlin","year":"2000","journal-title":"NeuroImage"},{"key":"bib27","first-page":"4171","type":"conference-paper","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin","year":"2019","unstructured":"Devlin J, Chang MW, Lee K, Toutanova K. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies."},{"key":"bib28","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1016\/j.tics.2019.02.004","type":"journal-article","article-title":"Shared neural mechanisms of visual perception and imagery","volume":"23","author":"Dijkstra","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"bib29","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0326-24.2024","type":"journal-article","article-title":"MEG Evidence that modality-independent conceptual representations contain semantic and visual features","volume":"44","author":"Dirani","year":"2024","journal-title":"The Journal of Neuroscience"},{"key":"bib30","type":"conference-paper","article-title":"An image is worth 16x16 Words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N. 2020. An image is worth 16x16 Words: transformers for image recognition at scale. In International Conference on Learning Representations."},{"key":"bib31","doi-asserted-by":"publisher","first-page":"1570","DOI":"10.1002\/hbm.20959","type":"journal-article","article-title":"Distortion correction for diffusion-weighted MRI tractography and fMRI in the temporal lobes","volume":"31","author":"Embleton","year":"2010","journal-title":"Human Brain Mapping"},{"key":"bib32","doi-asserted-by":"publisher","first-page":"10552","DOI":"10.1523\/JNEUROSCI.0051-13.2013","type":"journal-article","article-title":"Brain regions that represent amodal conceptual knowledge","volume":"33","author":"Fairhall","year":"2013","journal-title":"The Journal of Neuroscience"},{"key":"bib33","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/0010-0277(84)90026-x","type":"journal-article","article-title":"The neurological basis of mental imagery: a componential analysis","volume":"18","author":"Farah","year":"1984","journal-title":"Cognition"},{"key":"bib34","doi-asserted-by":"publisher","first-page":"1177","DOI":"10.1152\/jn.00032.2010","type":"journal-article","article-title":"New method for fMRI investigations of language: defining ROIs functionally in individual subjects","volume":"104","author":"Fedorenko","year":"2010","journal-title":"Journal of Neurophysiology"},{"key":"bib35","doi-asserted-by":"publisher","first-page":"16428","DOI":"10.1073\/pnas.1112937108","type":"journal-article","article-title":"Functional specificity for high-level linguistic processing in the human brain","volume":"108","author":"Fedorenko","year":"2011","journal-title":"PNAS"},{"key":"bib36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/cercor\/1.1.1-a","type":"journal-article","article-title":"Distributed hierarchical processing in the primate cerebral cortex","volume":"1","author":"Felleman","year":"1991","journal-title":"Cerebral Cortex"},{"key":"bib37","doi-asserted-by":"publisher","first-page":"2018","DOI":"10.1093\/cercor\/bhv020","type":"journal-article","article-title":"Concept representation reflects multimodal abstraction: a framework for embodied semantics","volume":"26","author":"Fernandino","year":"2016","journal-title":"Cerebral Cortex"},{"key":"bib38","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2108091119","type":"journal-article","article-title":"Decoding the information structure underlying the neural representation of concepts","volume":"119","author":"Fernandino","year":"2022","journal-title":"PNAS"},{"key":"bib39","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1016\/j.neuroimage.2012.01.021","type":"journal-article","article-title":"FreeSurfer","volume":"62","author":"Fischl","year":"2012","journal-title":"NeuroImage"},{"key":"bib40","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/11173.001.0001","type":"book","volume-title":"Language in Our Brain: The Origins of a Uniquely Human Capacity","author":"Friederici","year":"2017"},{"key":"bib41","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1016\/j.tics.2022.12.006","type":"journal-article","article-title":"Decoding semantic representations in mind and brain","volume":"27","author":"Frisby","year":"2023","journal-title":"Trends in Cognitive Sciences"},{"key":"bib42","doi-asserted-by":"publisher","first-page":"539","DOI":"10.1016\/s0010-9452(08)70537-9","type":"journal-article","article-title":"What the locus of brain lesion tells us about the nature of the cognitive defect underlying category-specific disorders: a review","volume":"36","author":"Gainotti","year":"2000","journal-title":"Cortex; a Journal Devoted to the Study of the Nervous System and Behavior"},{"key":"bib43","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01457","type":"conference-paper","article-title":"ImageBind one embedding space to bind them all","author":"Girdhar","year":"2023","unstructured":"Girdhar R, El-Nouby A, Liu Z, Singh M, Alwala KV, Joulin A, Misra I. 2023. ImageBind one embedding space to bind them all. 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"bib44","doi-asserted-by":"publisher","DOI":"10.3389\/fninf.2011.00013","type":"journal-article","article-title":"Nipype: a flexible, lightweight and extensible neuroimaging data processing framework in python","volume":"5","author":"Gorgolewski","year":"2011","journal-title":"Frontiers in Neuroinformatics"},{"key":"bib45","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1146\/annurev.neuro.27.070203.144220","type":"journal-article","article-title":"The human visual cortex","volume":"27","author":"Grill-Spector","year":"2004","journal-title":"Annual Review of Neuroscience"},{"key":"bib46","doi-asserted-by":"publisher","first-page":"3075","DOI":"10.1007\/s00429-022-02505-x","type":"journal-article","article-title":"The connectional anatomy of visual mental imagery: evidence from a patient with left occipito-temporal damage","volume":"227","author":"Hajhajate","year":"2022","journal-title":"Brain Structure & Function"},{"key":"bib47","doi-asserted-by":"publisher","first-page":"232","DOI":"10.1016\/j.neuroimage.2016.04.063","type":"journal-article","article-title":"How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge","volume":"135","author":"Handjaras","year":"2016","journal-title":"NeuroImage"},{"key":"bib48","doi-asserted-by":"publisher","first-page":"2425","DOI":"10.1126\/science.1063736","type":"journal-article","article-title":"Distributed and overlapping representations of faces and objects in ventral temporal cortex","volume":"293","author":"Haxby","year":"2001","journal-title":"Science"},{"key":"bib49","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90","type":"conference-paper","article-title":"Deep residual learning for image recognition","author":"He","year":"2016","unstructured":"He K, Zhang X, Ren S, Sun J. 2016. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"bib50","doi-asserted-by":"publisher","first-page":"1210","DOI":"10.1016\/j.neuron.2012.10.014","type":"journal-article","article-title":"A continuous semantic space describes the representation of thousands of object and action categories across the human brain","volume":"76","author":"Huth","year":"2012","journal-title":"Neuron"},{"key":"bib51","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1037\/a0015452","type":"journal-article","article-title":"Comprehension of concrete and abstract words in semantic dementia","volume":"23","author":"Jefferies","year":"2009","journal-title":"Neuropsychology"},{"key":"bib52","doi-asserted-by":"publisher","first-page":"782","DOI":"10.1016\/j.neuroimage.2011.09.015","type":"journal-article","article-title":"FSL","volume":"62","author":"Jenkinson","year":"2012","journal-title":"NeuroImage"},{"key":"bib53","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.06825","type":"preprint","article-title":"Mistral7B","volume-title":"arXiv","author":"Jiang","year":"2023"},{"key":"bib54","doi-asserted-by":"publisher","DOI":"10.3389\/fnhum.2014.00059","type":"journal-article","article-title":"Decoding individual natural scene representations during perception and imagery","volume":"8","author":"Johnson","year":"2014","journal-title":"Frontiers in Human Neuroscience"},{"key":"bib55","doi-asserted-by":"publisher","first-page":"5969","DOI":"10.1523\/JNEUROSCI.0272-18.2018","type":"journal-article","article-title":"Modality-independent coding of scene categories in prefrontal cortex","volume":"38","author":"Jung","year":"2018","journal-title":"The Journal of Neuroscience"},{"key":"bib56","doi-asserted-by":"publisher","first-page":"11163","DOI":"10.1073\/pnas.1005062107","type":"journal-article","article-title":"Functional specificity in the human brain: A window into the functional architecture of the mind","volume":"107","author":"Kanwisher","year":"2010","journal-title":"PNAS"},{"key":"bib57","doi-asserted-by":"publisher","first-page":"352","DOI":"10.1038\/nature06713","type":"journal-article","article-title":"Identifying natural images from human brain activity","volume":"452","author":"Kay","year":"2008","journal-title":"Nature"},{"key":"bib58","doi-asserted-by":"publisher","first-page":"805","DOI":"10.1016\/j.cortex.2011.04.006","type":"journal-article","article-title":"Conceptual representations in mind and brain: theoretical developments, current evidence and future directions","volume":"48","author":"Kiefer","year":"2012","journal-title":"Cortex; a Journal Devoted to the Study of the Nervous System and Behavior"},{"key":"bib59","first-page":"5583","type":"conference-paper","article-title":"ViLT: Vision-and-Language Transformer without convolution or region supervision","author":"Kim","year":"2021","unstructured":"Kim W, Son B, Kim I. 2021. ViLT: Vision-and-Language Transformer without convolution or region supervision. In Proceedings of the 38th International Conference on Machine Learning."},{"key":"bib60","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1126\/science.284.5411.167","type":"journal-article","article-title":"The role of area 17 in visual imagery: convergent evidence from PET and rTMS","volume":"284","author":"Kosslyn","year":"1999","journal-title":"Science"},{"key":"bib61","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1037\/0033-2909.129.5.723","type":"journal-article","article-title":"When is early visual cortex activated during visual mental imagery?","volume":"129","author":"Kosslyn","year":"2003","journal-title":"Psychological Bulletin"},{"key":"bib62","doi-asserted-by":"publisher","first-page":"5729","DOI":"10.18653\/v1\/P19-1573","type":"conference-paper","article-title":"Empirical linguistic study of sentence embeddings","author":"Krasnowska-Kiera\u015b","year":"2019","unstructured":"Krasnowska-Kiera\u015b K, Wr\u00f3blewska A. 2019. Empirical linguistic study of sentence embeddings. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics."},{"key":"bib63","doi-asserted-by":"publisher","DOI":"10.3389\/neuro.06.004.2008","type":"journal-article","article-title":"Representational similarity analysis - connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Frontiers in Systems Neuroscience"},{"key":"bib64","doi-asserted-by":"publisher","first-page":"1127","DOI":"10.1093\/brain\/awm025","type":"journal-article","article-title":"Neural basis of category-specific semantic deficits for living things: evidence from semantic dementia, HSVE and a neural network model","volume":"130","author":"Lambon Ralph","year":"2006","journal-title":"Brain"},{"key":"bib65","doi-asserted-by":"publisher","first-page":"4064","DOI":"10.1016\/j.neuroimage.2011.10.055","type":"journal-article","article-title":"Disentangling visual imagery and perception of real-world objects","volume":"59","author":"Lee","year":"2012","journal-title":"NeuroImage"},{"key":"bib66","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1908.03557","type":"preprint","article-title":"VisualBERT: A simple and performant baseline for vision and language","volume-title":"arXiv","author":"Li","year":"2019"},{"key":"bib67","type":"conference-paper","article-title":"BLIP-2: Bootstrapping Language-Image Pre-training with frozen image encoders and large language models","author":"Li","year":"2023","unstructured":"Li J, Li D, Savarese S, Hoi S. 2023. BLIP-2: Bootstrapping Language-Image Pre-training with frozen image encoders and large language models. In International Conference on Machine Learning."},{"key":"bib68","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","type":"book-chapter","article-title":"Microsoft COCO: common objects in context","volume-title":"In Computer Vision \u2013 ECCV","author":"Lin","year":"2014"},{"key":"bib69","doi-asserted-by":"publisher","first-page":"29624","DOI":"10.52202\/068431-2148","type":"conference-paper","article-title":"Mind reader: reconstructing complex images from brain activities","author":"Lin","year":"2022","unstructured":"Lin S, Singh AK, Sprague T. 2022. Mind reader: reconstructing complex images from brain activities. Advances in Neural Information Processing Systems 35."},{"key":"bib70","doi-asserted-by":"publisher","first-page":"697","DOI":"10.1007\/s00429-021-02277-w","type":"journal-article","article-title":"Hemispheric asymmetries in visual mental imagery","volume":"227","author":"Liu","year":"2022","journal-title":"Brain Structure & Function"},{"key":"bib71","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1016\/j.cortex.2025.01.013","type":"journal-article","article-title":"Visual mental imagery in typical imagers and in aphantasia: A millimeter-scale 7-T fMRI study","volume":"185","author":"Liu","year":"2025","journal-title":"Cortex; a Journal Devoted to the Study of the Nervous System and Behavior"},{"key":"bib72","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1016\/j.neuroimage.2017.02.032","type":"journal-article","article-title":"Cross-modal representation of spoken and written word meaning in left pars triangularis","volume":"150","author":"Liuzzi","year":"2017","journal-title":"NeuroImage"},{"key":"bib73","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-65906-0","type":"journal-article","article-title":"General and feature-based semantic representations in the semantic network","volume":"10","author":"Liuzzi","year":"2020","journal-title":"Scientific Reports"},{"key":"bib74","doi-asserted-by":"publisher","first-page":"16629","DOI":"10.1523\/JNEUROSCI.2342-12.2012","type":"journal-article","article-title":"Sight and sound converge to form modality-invariant representations in temporoparietal cortex","volume":"32","author":"Man","year":"2012","journal-title":"The Journal of Neuroscience"},{"key":"bib75","doi-asserted-by":"publisher","first-page":"1031","DOI":"10.7551\/mitpress\/8029.003.0091","type":"book-chapter","article-title":"Circuits in mind: the neural foundations for object concepts","volume-title":"The Cognitive Neurosciences","author":"Martin","year":"2009"},{"key":"bib76","doi-asserted-by":"publisher","first-page":"979","DOI":"10.3758\/s13423-015-0842-3","type":"journal-article","article-title":"GRAPES\u2014Grounding representations in action, perception, and emotion systems: How object properties and categories are represented in the human brain","volume":"23","author":"Martin","year":"2016","journal-title":"Psychonomic Bulletin & Review"},{"key":"bib77","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.31873","type":"journal-article","article-title":"Integrative and distinctive coding of visual and conceptual object features in the ventral visual stream","volume":"7","author":"Martin","year":"2018","journal-title":"eLife"},{"key":"bib78","type":"conference-paper","article-title":"Mapping multimodal conceptual representations within the lexical-semantic brain system","author":"Meschke","year":"2024","unstructured":"Meschke E, Gallant J. 2024. Mapping multimodal conceptual representations within the lexical-semantic brain system. In CCN."},{"key":"bib79","doi-asserted-by":"publisher","first-page":"376","DOI":"10.1016\/j.tins.2009.04.002","type":"journal-article","article-title":"Convergence and divergence in a neural architecture for recognition and memory","volume":"32","author":"Meyer","year":"2009","journal-title":"Trends in Neurosciences"},{"key":"bib80","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1006\/nimg.1999.0450","type":"journal-article","article-title":"Three distinct ventral occipitotemporal regions for reading and object naming","volume":"10","author":"Moore","year":"1999","journal-title":"NeuroImage"},{"key":"bib81","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1016\/j.neuron.2009.09.006","type":"journal-article","article-title":"Bayesian reconstruction of natural images from human brain activity","volume":"63","author":"Naselaris","year":"2009","journal-title":"Neuron"},{"key":"bib82","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1016\/j.neuroimage.2010.07.073","type":"journal-article","article-title":"Encoding and decoding in fMRI","volume":"56","author":"Naselaris","year":"2011","journal-title":"NeuroImage"},{"key":"bib83","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.neuroimage.2014.10.018","type":"journal-article","article-title":"A voxel-wise encoding model for early visual areas decodes mental images of remembered scenes","volume":"105","author":"Naselaris","year":"2015","journal-title":"NeuroImage"},{"key":"bib84","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-021-01033-3","type":"journal-article","article-title":"The \u201cNarratives\u201d fMRI dataset for evaluating models of naturalistic language comprehension","volume":"8","author":"Nastase","year":"2021","journal-title":"Scientific Data"},{"key":"bib85","type":"software","article-title":"Multimodal_decoding","volume-title":"Software Heritage","author":"Nikolaus","year":"2026","unstructured":"Nikolaus M. 2026. Multimodal_decoding. Software Heritage. swh:1:rev:57f7a1c4a5da1c96887ed79f0507cbd286c727fd. https:\/\/archive.softwareheritage.org\/swh:1:dir:358f81a6a99cdcdf3ba0129b4e5627b7170b3273;origin=https:\/\/github.com\/mitjanikolaus\/multimodal_decoding;visit=swh:1:snp:840f7a3b5e83b249f545ad61c124df2dab366bac;anchor=swh:1:rev:57f7a1c4a5da1c96887ed79f0507cbd286c727fd."},{"key":"bib86","doi-asserted-by":"publisher","first-page":"1641","DOI":"10.1016\/j.cub.2011.08.031","type":"journal-article","article-title":"Reconstructing visual experiences from brain activity evoked by natural movies","volume":"21","author":"Nishimoto","year":"2011","journal-title":"Current Biology"},{"key":"bib87","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1162\/08989290051137549","type":"journal-article","article-title":"Mental imagery of faces and places activates corresponding stiimulus-specific brain regions","volume":"12","author":"O\u2019Craven","year":"2000","journal-title":"Journal of Cognitive Neuroscience"},{"key":"bib88","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.07193","type":"preprint","article-title":"DINOv2: Learning robust visual features without supervision","volume-title":"arXiv","author":"Oquab","year":"2023"},{"key":"bib89","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-42891-8","type":"journal-article","article-title":"Natural scene reconstruction from fMRI signals using generative latent diffusion","volume":"13","author":"Ozcelik","year":"2023","journal-title":"Scientific Reports"},{"key":"bib90","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1016\/B978-0-12-407794-2.00061-4","type":"book-chapter","article-title":"The hub-and-spoke hypothesis of semantic memory","volume-title":"In Neurobiology of Language","author":"Patterson","year":"2016"},{"key":"bib91","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1038\/s41583-019-0202-9","type":"journal-article","article-title":"The human imagination: the cognitive neuroscience of visual mental imagery","volume":"20","author":"Pearson","year":"2019","journal-title":"Nature Reviews. Neuroscience"},{"key":"bib92","first-page":"2825","type":"journal-article","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"The Journal of Machine Learning Research"},{"key":"bib93","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-03068-4","type":"journal-article","article-title":"Toward a universal decoder of linguistic meaning from brain activation","volume":"9","author":"Pereira","year":"2018","journal-title":"Nature Communications"},{"key":"bib94","doi-asserted-by":"publisher","first-page":"964","DOI":"10.1016\/j.cub.2010.03.070","type":"journal-article","article-title":"Category-specific versus category-general semantic impairment induced by transcranial magnetic stimulation","volume":"20","author":"Pobric","year":"2010","journal-title":"Current Biology"},{"key":"bib95","doi-asserted-by":"publisher","first-page":"1628","DOI":"10.1038\/s41593-021-00921-6","type":"journal-article","article-title":"Visual and linguistic semantic representations are aligned at the border of human visual cortex","volume":"24","author":"Popham","year":"2021","journal-title":"Nature Neuroscience"},{"key":"bib96","doi-asserted-by":"publisher","first-page":"2142","DOI":"10.1016\/j.neuroimage.2011.10.018","type":"journal-article","article-title":"Spurious but systematic correlations in functional connectivity MRI networks arise from subject motion","volume":"59","author":"Power","year":"2012","journal-title":"NeuroImage"},{"key":"bib97","first-page":"1","type":"journal-article","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford","year":"2019","journal-title":"OpenAI Blog"},{"key":"bib98","type":"conference-paper","article-title":"Learning transferable visual models from natural language supervision","author":"Radford","year":"2021","unstructured":"Radford A, Kim JW, Hallacy C, Ramesh A, Goh G, Agarwal S, Sastry G, Askell A, Mishkin P, Clark J, Krueger G, Sutskever I. 2021. Learning transferable visual models from natural language supervision. Proceedings of the 38th International Conference on Machine Learning."},{"key":"bib99","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1038\/nrn.2016.150","type":"journal-article","article-title":"The neural and computational bases of semantic cognition","volume":"18","author":"Ralph","year":"2017","journal-title":"Nature Reviews. Neuroscience"},{"key":"bib100","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1016\/j.neuroimage.2009.11.084","type":"journal-article","article-title":"Reading the mind\u2019s eye: decoding category information during mental imagery","volume":"50","author":"Reddy","year":"2010","journal-title":"NeuroImage"},{"key":"bib101","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410","type":"conference-paper","article-title":"Sentence-BERT: sentence embeddings using siamese BERT-networks","author":"Reimers","year":"2019","unstructured":"Reimers N, Gurevych I. 2019. Sentence-BERT: sentence embeddings using siamese BERT-networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)."},{"key":"bib102","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1037\/0033-295X.111.1.205","type":"journal-article","article-title":"Structure and deterioration of semantic memory: a neuropsychological and computational investigation","volume":"111","author":"Rogers","year":"2004","journal-title":"Psychological Review"},{"key":"bib103","doi-asserted-by":"publisher","first-page":"7437","DOI":"10.1073\/pnas.1912584117","type":"journal-article","article-title":"Decoding across sensory modalities reveals common supramodal signatures of conscious perception","volume":"117","author":"Sanchez","year":"2020","journal-title":"PNAS"},{"key":"bib104","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-019-0020-y","type":"journal-article","article-title":"A 204-subject multimodal neuroimaging dataset to study language processing","volume":"6","author":"Schoffelen","year":"2019","journal-title":"Scientific Data"},{"key":"bib105","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1126\/science.7754376","type":"journal-article","article-title":"Borders of multiple visual areas in humans revealed by functional magnetic resonance imaging","volume":"268","author":"Sereno","year":"1995","journal-title":"Science"},{"key":"bib106","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1006633","type":"journal-article","article-title":"Deep image reconstruction from human brain activity","volume":"15","author":"Shen","year":"2019","journal-title":"PLOS Computational Biology"},{"key":"bib107","doi-asserted-by":"publisher","first-page":"2418","DOI":"10.1016\/j.neuroimage.2010.10.042","type":"journal-article","article-title":"Commonality of neural representations of words and pictures","volume":"54","author":"Shinkareva","year":"2011","journal-title":"NeuroImage"},{"key":"bib108","doi-asserted-by":"publisher","first-page":"426","DOI":"10.1093\/cercor\/bhs324","type":"journal-article","article-title":"Modality-independent decoding of semantic information from the human brain","volume":"24","author":"Simanova","year":"2014","journal-title":"Cerebral Cortex"},{"key":"bib109","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1080\/02643290342000032","type":"journal-article","article-title":"The similarity-in-topography principle: reconciling theories of conceptual deficits","volume":"20","author":"Simmons","year":"2003","journal-title":"Cognitive Neuropsychology"},{"key":"bib110","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01519","type":"conference-paper","article-title":"FLAVA: A Foundational Language And Vision Alignment Model","author":"Singh","year":"2022","unstructured":"Singh A, Hu R, Goswami V, Couairon G, Galuba W, Rohrbach M, Kiela D. 2022. FLAVA: A Foundational Language And Vision Alignment Model. 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"bib111","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.neuroimage.2008.03.061","type":"journal-article","article-title":"Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference","volume":"44","author":"Smith","year":"2009","journal-title":"NeuroImage"},{"key":"bib112","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1155\/1989\/124043","type":"journal-article","article-title":"Semantic dementia: A form of circumscribed cerebral atrophy","volume":"2","author":"Snowden","year":"1989","journal-title":"Behavioural Neurology"},{"key":"bib113","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1016\/j.neubiorev.2020.12.029","type":"journal-article","article-title":"Visual mental imagery engages the left fusiform gyrus, but not the early visual cortex: A meta-analysis of neuroimaging evidence","volume":"122","author":"Spagna","year":"2021","journal-title":"Neuroscience & Biobehavioral Reviews"},{"key":"bib114","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-22848-z","type":"journal-article","article-title":"A network linking scene perception and spatial memory systems in posterior cerebral cortex","volume":"12","author":"Steel","year":"2021","journal-title":"Nature Communications"},{"key":"bib115","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2412.03555","type":"preprint","article-title":"PaliGemma 2: A family of versatile VLMs for transfer","volume-title":"arXiv","author":"Steiner","year":"2024"},{"key":"bib116","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/j.neuroimage.2012.09.063","type":"journal-article","article-title":"Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control","volume":"65","author":"Stelzer","year":"2013","journal-title":"NeuroImage"},{"key":"bib117","doi-asserted-by":"publisher","first-page":"1565","DOI":"10.1523\/JNEUROSCI.4657-08.2009","type":"journal-article","article-title":"Top-down activation of shape-specific population codes in visual cortex during mental imagery","volume":"29","author":"Stokes","year":"2009","journal-title":"The Journal of Neuroscience"},{"key":"bib118","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01389","type":"conference-paper","article-title":"High-resolution image reconstruction with latent diffusion models from human brain activity","author":"Takagi","year":"2023","unstructured":"Takagi Y, Nishimoto S. 2023. High-resolution image reconstruction with latent diffusion models from human brain activity. 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"bib119","type":"conference-paper","article-title":"Brain encoding models based on multimodal transformers can transfer across language and vision","author":"Tang","year":"2023","unstructured":"Tang J, Du M, Vo VA. 2023a. Brain encoding models based on multimodal transformers can transfer across language and vision. In NeurIPS."},{"key":"bib120","doi-asserted-by":"publisher","first-page":"858","DOI":"10.1038\/s41593-023-01304-9","type":"journal-article","article-title":"Semantic reconstruction of continuous language from non-invasive brain recordings","volume":"26","author":"Tang","year":"2023","journal-title":"Nature Neuroscience"},{"key":"bib121","doi-asserted-by":"publisher","first-page":"7121","DOI":"10.1523\/JNEUROSCI.1243-21.2022","type":"journal-article","article-title":"A distributed network for multimodal experiential representation of concepts","volume":"42","author":"Tong","year":"2022","journal-title":"The Journal of Neuroscience"},{"key":"bib122","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288","type":"preprint","article-title":"Llama 2: Open foundation and fine-tuned chat models","volume-title":"arXiv","author":"Touvron","year":"2023"},{"key":"bib123","doi-asserted-by":"publisher","first-page":"1319","DOI":"10.1016\/s0028-3932(97)00085-7","type":"journal-article","article-title":"A neural basis for the retrieval of conceptual knowledge","volume":"35","author":"Tranel","year":"1997","journal-title":"Neuropsychologia"},{"key":"bib124","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1038\/383254a0","type":"journal-article","article-title":"Functional anatomy of a common semantic system for words and pictures","volume":"383","author":"Vandenberghe","year":"1996","journal-title":"Nature"},{"key":"bib125","doi-asserted-by":"publisher","DOI":"10.1038\/s42003-019-0438-y","type":"journal-article","article-title":"Reconstructing faces from fMRI patterns using deep generative neural networks","volume":"2","author":"VanRullen","year":"2019","journal-title":"Communications Biology"},{"key":"bib126","doi-asserted-by":"publisher","DOI":"10.1038\/s41597-020-00735-4","type":"journal-article","article-title":"An fMRI dataset in response to \u201cThe Grand Budapest Hotel\u201d, a socially-rich, naturalistic movie","volume":"7","author":"Visconti di Oleggio Castello","year":"2020","journal-title":"Scientific Data"},{"key":"bib127","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1080\/14640747508400525","type":"journal-article","article-title":"The selective impairment of semantic memory","volume":"27","author":"Warrington","year":"1975","journal-title":"The Quarterly Journal of Experimental Psychology"},{"key":"bib128","doi-asserted-by":"crossref","DOI":"10.1093\/brain\/107.3.829","type":"book","volume-title":"Category Specific Semantic Impairments","author":"Warrington","year":"1984"},{"key":"bib129","doi-asserted-by":"crossref","DOI":"10.1093\/brain\/110.5.1273","type":"book","volume-title":"Categories of Knowledge: Further Fractionations and an Attempted Integration","author":"Warrington","year":"1987"},{"key":"bib130","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6","type":"conference-paper","article-title":"Transformers: State-of-the-art natural language processing","author":"Wolf","year":"2020","unstructured":"Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M, Davison J, Shleifer S, von Platen P, Ma C, Jernite Y, Plu J, Xu C, Le Scao T, Gugger S, Drame M, Lhoest Q, Rush A. 2020. Transformers: State-of-the-art natural language processing. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing."},{"key":"bib131","doi-asserted-by":"publisher","first-page":"10637","DOI":"10.1609\/aaai.v37i9.26263","type":"journal-article","article-title":"Bridgetower: building bridges between encoders in vision-language representation learning","volume":"37","author":"Xu","year":"2023","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"bib132","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01100","type":"conference-paper","article-title":"Sigmoid loss for language image pre-training","author":"Zhai","year":"2023","unstructured":"Zhai X, Mustafa B, Kolesnikov A, Beyer L. 2023. Sigmoid loss for language image pre-training. 2023 IEEE\/CVF International Conference on Computer Vision (ICCV)."},{"key":"bib133","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1146\/annurev.ne.18.030195.002043","type":"journal-article","article-title":"Localization of brain function: the legacy of Franz Joseph Gall (1758-1828)","volume":"18","author":"Zola-Morgan","year":"1995","journal-title":"Annual Review of Neuroscience"},{"key":"bib134","doi-asserted-by":"publisher","first-page":"1028","DOI":"10.3758\/s13423-015-0864-x","type":"journal-article","article-title":"Situation models, mental simulations, and abstract concepts in discourse comprehension","volume":"23","author":"Zwaan","year":"2016","journal-title":"Psychonomic Bulletin & Review"},{"key":"dataset1","doi-asserted-by":"publisher","DOI":"10.18112\/openneuro.ds007272.v1.0.0","type":"dataset","article-title":"SemReps-8K","volume-title":"OpenNeuro","author":"Nikolaus","year":"2025","unstructured":"Nikolaus M, Mozafari M, Berry I, Asher N, Reddy L, VanRullen R. 2025. SemReps-8K. OpenNeuro."}],"container-title":["eLife"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/cdn.elifesciences.org\/articles\/107933\/elife-107933-v1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/cdn.elifesciences.org\/articles\/107933\/elife-107933-v1.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/elifesciences.org\/articles\/107933","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,24]],"date-time":"2026-08-24T09:01:51Z","timestamp":1787562111000},"score":1,"resource":{"primary":{"URL":"https:\/\/elifesciences.org\/articles\/107933"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,17]]},"references-count":135,"alternative-id":["10.7554\/eLife.107933"],"URL":"https:\/\/doi.org\/10.7554\/elife.107933","archive":["CLOCKSS"],"relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2025.06.08.658221","asserted-by":"subject"}],"is-same-as":[{"id-type":"doi","id":"10.7554\/eLife.107933.3","asserted-by":"subject"}],"is-version-of":[{"id-type":"doi","id":"10.7554\/eLife.107933.1","asserted-by":"subject"},{"id-type":"doi","id":"10.7554\/eLife.107933.2","asserted-by":"subject"}],"is-supplemented-by":[{"id-type":"doi","id":"10.18112\/openneuro.ds007272.v1.0.0","asserted-by":"subject"},{"id-type":"uri","id":"https:\/\/archive.softwareheritage.org\/swh:1:dir:358f81a6a99cdcdf3ba0129b4e5627b7170b3273;origin=https:\/\/github.com\/mitjanikolaus\/multimodal_decoding;visit=swh:1:snp:840f7a3b5e83b249f545ad61c124df2dab366bac;anchor=swh:1:rev:57f7a1c4a5da1c96887ed79f0507cbd286c727fd","asserted-by":"subject"}]},"ISSN":["2050-084X"],"issn-type":[{"value":"2050-084X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,17]]},"assertion":[{"value":"single anonymised","name":"peer_review_transparency","label":"Peer review transparency","group":{"name":"peer_review_taxonomy"}},{"value":"other reviewer(s), editor","name":"peer_review_interaction","label":"Peer review interaction","group":{"name":"peer_review_taxonomy"}},{"value":"review summaries, review reports, author\/editor communication, reviewer identities reviewer opt in, editor identities","name":"peer_review_published","label":"Peer review published","group":{"name":"peer_review_taxonomy"}},{"value":"open (sign in with ORCID iD required)","name":"post_publication_commenting","label":"Post publication commenting","group":{"name":"post_publication_commenting"}}],"article-number":"RP107933"}}