{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T19:21:31Z","timestamp":1791400891893,"version":"4.3.3"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2016,11,24]],"date-time":"2016-11-24T00:00:00Z","timestamp":1479945600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2016,11,24]],"date-time":"2016-11-24T00:00:00Z","timestamp":1479945600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recognizing familiar individuals is achieved by the brain by combining cues from several sensory modalities, including the face of a person and her voice. Here we used functional magnetic resonance (fMRI) and a whole-brain, searchlight multi-voxel pattern analysis (MVPA) to search for areas in which local fMRI patterns could result in identity classification as a function of sensory modality. We found several areas supporting face or voice stimulus classification based on fMRI responses, consistent with previous reports; the classification maps overlapped across modalities in a single area of right posterior superior temporal sulcus (pSTS). Remarkably, we also found several cortical areas, mostly located along the middle temporal gyrus, in which local fMRI patterns resulted in identity \u201ccross-classification\u201d: vocal identity could be classified based on fMRI responses to the faces, or the reverse, or both. These findings are suggestive of a series of cortical identity representations increasingly abstracted from the input modality.<\/jats:p>","DOI":"10.1038\/srep37494","type":"journal-article","created":{"date-parts":[[2016,11,24]],"date-time":"2016-11-24T10:05:46Z","timestamp":1479981946000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["\u201cHearing faces and seeing voices\u201d: Amodal coding of person identity in the human brain"],"prefix":"10.1038","volume":"6","author":[{"given":"Bashar","family":"Awwad Shiekh Hasan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mitchell","family":"Valdes-Sosa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joachim","family":"Gross","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pascal","family":"Belin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2016,11,24]]},"reference":[{"key":"BFsrep37494_CR1","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1016\/j.neubiorev.2014.10.022","volume":"47","author":"H Blank","year":"2014","unstructured":"Blank, H., Wieland, N. & von Kriegstein, K. Person recognition and the brain: merging evidence from patients and healthy individuals. Neurosci Biobehav Rev 47, 717\u2013734, doi: 10.1016\/j.neubiorev.2014.10.022 (2014).","journal-title":"Neurosci Biobehav Rev"},{"key":"BFsrep37494_CR2","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1016\/j.tics.2015.09.002","volume":"19","author":"C Perrodin","year":"2015","unstructured":"Perrodin, C., Kayser, C., Abel, T. J., Logothetis, N. K. & Petkov, C. I. Who is That? Brain Networks and Mechanisms for Identifying Individuals. Trends Cogn Sci 19, 783\u2013796, doi: 10.1016\/j.tics.2015.09.002 (2015).","journal-title":"Trends Cogn Sci"},{"key":"BFsrep37494_CR3","doi-asserted-by":"publisher","first-page":"1446","DOI":"10.1080\/17470210601063589","volume":"60","author":"SR Schweinberger","year":"2007","unstructured":"Schweinberger, S. R., Robertson, D. & Kaufmann, J. M. Hearing facial identities. Quart J Exp Psychol 60, 1446\u20131456 (2007).","journal-title":"Quart J Exp Psychol"},{"key":"BFsrep37494_CR4","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1016\/j.tics.2007.10.001","volume":"11","author":"S Campanella","year":"2007","unstructured":"Campanella, S. & Belin, P. Integrating face and voice in person perception. Trends Cogn Sci 11, 535\u2013543 (2007).","journal-title":"Trends Cogn Sci"},{"key":"BFsrep37494_CR5","doi-asserted-by":"publisher","first-page":"266","DOI":"10.5334\/pb.ar","volume":"54","author":"SV Stevenage","year":"2014","unstructured":"Stevenage, S. V. & Neil, G. J. Hearing Faces and Seeing Voices: The Integration and Interaction of Face and Voice Processing. Psychol Belg 54, 266\u2013281, doi: 10.5334\/pb.ar (2014).","journal-title":"Psychol Belg"},{"key":"BFsrep37494_CR6","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1111\/j.2044-8295.1986.tb02199.x","volume":"77","author":"V Bruce","year":"1986","unstructured":"Bruce, V. & Young, A. Understanding face recognition. Br J Psychol 77, 305\u2013327 (1986).","journal-title":"Br J Psychol"},{"key":"BFsrep37494_CR7","doi-asserted-by":"publisher","first-page":"959","DOI":"10.1111\/j.2044-8295.2011.02045.x","volume":"102","author":"AW Young","year":"2011","unstructured":"Young, A. W. & Bruce, V. Understanding person perception. Br J Psychol 102, 959\u2013974, doi: 10.1111\/j.2044-8295.2011.02045.x (2011).","journal-title":"Br J Psychol"},{"key":"BFsrep37494_CR8","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1093\/brain\/124.4.804","volume":"124","author":"NJ Shah","year":"2001","unstructured":"Shah, N. J. et al. The neural correlates of person familiarity. A functional magnetic resonance imaging study with clinical implications. Brain 124, 804\u2013815 (2001).","journal-title":"Brain"},{"key":"BFsrep37494_CR9","doi-asserted-by":"publisher","first-page":"S688","DOI":"10.1016\/S1053-8119(01)92031-2","volume":"13","author":"F Joassin","year":"2001","unstructured":"Joassin, F. et al. The neural substrates sustaining the retrieval of face-name associations: a PET activation study. Neuroimage 13, S688\u2013S688 (2001).","journal-title":"Neuroimage"},{"key":"BFsrep37494_CR10","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/j.neulet.2004.07.067","volume":"369","author":"F Joassin","year":"2004","unstructured":"Joassin, F., Maurage, P., Bruyer, R., Crommelinck, M. & Campanella, S. When audition alters vision: an event-related potential study of the cross-modal interactions between faces and voices. Neurosci Lett 369, 132\u2013137 (2004).","journal-title":"Neurosci Lett"},{"key":"BFsrep37494_CR11","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1016\/j.cortex.2010.03.003S0010-9452(10)00118-8","volume":"47","author":"F Joassin","year":"2011","unstructured":"Joassin, F. et al. Cross-modal interactions between human faces and voices involved in person recognition. Cortex 47, 367\u2013376, doi: 10.1016\/j.cortex.2010.03.003S0010-9452(10)00118-8 (2011).","journal-title":"Cortex"},{"key":"BFsrep37494_CR12","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1016\/j.cortex.2013.07.011S0010-9452(13)00183-4","volume":"50","author":"R Watson","year":"2014","unstructured":"Watson, R., Latinus, M., Charest, I., Crabbe, F. & Belin, P. People-selectivity, audiovisual integration and heteromodality in the superior temporal sulcus. Cortex 50, 125\u2013136, doi: 10.1016\/j.cortex.2013.07.011S0010-9452(13)00183-4 (2014).","journal-title":"Cortex"},{"key":"BFsrep37494_CR13","doi-asserted-by":"publisher","first-page":"4457","DOI":"10.1523\/JNEUROSCI.0541-08.2008","volume":"28","author":"AA Ghazanfar","year":"2008","unstructured":"Ghazanfar, A. A., Chandrasekaran, C. & Logothetis, N. K. Interactions between the superior temporal sulcus and auditory cortex mediate dynamica face\/voice integration in rhesus monkeys. J Neurosci 28, 4457\u20134469 (2008).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR14","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.neuropsychologia.2015.09.002","volume":"77","author":"G Gainotti","year":"2015","unstructured":"Gainotti, G. Implications of recent findings for current cognitive models of familiar people recognition. Neuropsychologia 77, 279\u2013287, doi: 10.1016\/j.neuropsychologia.2015.09.002 (2015).","journal-title":"Neuropsychologia"},{"key":"BFsrep37494_CR15","doi-asserted-by":"publisher","first-page":"12906","DOI":"10.1523\/JNEUROSCI.2091-11.201131\/36\/12906","volume":"31","author":"H Blank","year":"2011","unstructured":"Blank, H., Anwander, A. & von Kriegstein, K. Direct structural connections between voice- and face-recognition areas. J Neurosci 31, 12906\u201312915, doi: 10.1523\/JNEUROSCI.2091-11.201131\/36\/12906 (2011).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR16","doi-asserted-by":"publisher","first-page":"1314","DOI":"10.1093\/cercor\/bhj073","volume":"16","author":"K von Kriegstein","year":"2006","unstructured":"von Kriegstein, K., Kleinschmidt, A. & Giraud, A. L. Voice Recognition and Cross-Modal Responses to Familiar Speakers\u2019 Voices in Prosopagnosia. Cereb Cortex 16, 1314\u20131322 (2006).","journal-title":"Cereb Cortex"},{"key":"BFsrep37494_CR17","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/S1053-8119(03)00049-1","volume":"19","author":"DD Cox","year":"2003","unstructured":"Cox, D. D. & Savoy, R. L. Functional magnetic resonance imaging (fMRI) \u201cbrain reading\u201d: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261\u2013270 (2003).","journal-title":"Neuroimage"},{"key":"BFsrep37494_CR18","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1146\/annurev-neuro-062012-170325","volume":"37","author":"JV Haxby","year":"2014","unstructured":"Haxby, J. V., Connolly, A. C. & Guntupalli, J. S. Decoding neural representational spaces using multivariate pattern analysis. Annu Rev Neurosci 37, 435\u2013456, doi: 10.1146\/annurev-neuro-062012-170325 (2014).","journal-title":"Annu Rev Neurosci"},{"key":"BFsrep37494_CR19","doi-asserted-by":"publisher","first-page":"1988","DOI":"10.1093\/cercor\/bht046","volume":"24","author":"S Anzellotti","year":"2014","unstructured":"Anzellotti, S., Fairhall, S. L. & Caramazza, A. Decoding representations of face identity that are tolerant to rotation. Cereb Cortex 24, 1988\u20131995, doi: 10.1093\/cercor\/bht046 (2014).","journal-title":"Cereb Cortex"},{"key":"BFsrep37494_CR20","doi-asserted-by":"publisher","first-page":"9998","DOI":"10.1073\/pnas.1102433108","volume":"108","author":"A Nestor","year":"2011","unstructured":"Nestor, A., Plaut, D. C. & Behrmann, M. Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis. Proc Natl Acad Sci USA 108, 9998\u201310003, doi: 10.1073\/pnas.1102433108 (2011).","journal-title":"Proc Natl Acad Sci USA"},{"key":"BFsrep37494_CR21","doi-asserted-by":"publisher","first-page":"e0117126","DOI":"10.1371\/journal.pone.0117126","volume":"10","author":"V Axelrod","year":"2015","unstructured":"Axelrod, V. & Yovel, G. Successful decoding of famous faces in the fusiform face area. PLoS One 10, e0117126, doi: 10.1371\/journal.pone.0117126 (2015).","journal-title":"PLoS One"},{"key":"BFsrep37494_CR22","doi-asserted-by":"publisher","first-page":"2791","DOI":"10.1523\/JNEUROSCI.4037-14.2015","volume":"35","author":"J Dubois","year":"2015","unstructured":"Dubois, J., de Berker, A. O. & Tsao, D. Y. Single-unit recordings in the macaque face patch system reveal limitations of fMRI MVPA. J Neurosci 35, 2791\u20132802, doi: 10.1523\/JNEUROSCI.4037-14.2015 (2015).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR23","doi-asserted-by":"publisher","first-page":"20600","DOI":"10.1073\/pnas.0705654104","volume":"104","author":"N Kriegeskorte","year":"2007","unstructured":"Kriegeskorte, N., Formisano, E., Sorger, B. & Goebel, R. Individual faces elicit distinct response patterns in human anterior temporal cortex. Proceedings of the National Academy of Sciences of the United States of America 104, 20600\u201320605 (2007).","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"BFsrep37494_CR24","doi-asserted-by":"publisher","first-page":"970","DOI":"10.1126\/science.1164318","volume":"322","author":"E Formisano","year":"2008","unstructured":"Formisano, E., De Martino, F., Bonte, M. & Goebel, R. \u201cWho\u201d Is Saying \u201cWhat\u201d\u2009?\u2009Brain-Based Decoding of Human Voice and Speech. Science 322, 970\u2013973 (2008).","journal-title":"Science"},{"key":"BFsrep37494_CR25","doi-asserted-by":"publisher","first-page":"4548","DOI":"10.1523\/JNEUROSCI.4339-13.2014 34\/13\/4548","volume":"34","author":"M Bonte","year":"2014","unstructured":"Bonte, M., Hausfeld, L., Scharke, W., Valente, G. & Formisano, E. Task-dependent decoding of speaker and vowel identity from auditory cortical response patterns. J Neurosci 34, 4548\u20134557, doi: 10.1523\/JNEUROSCI.4339-13.2014 34\/13\/4548 (2014).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR26","doi-asserted-by":"publisher","first-page":"10127","DOI":"10.1523\/JNEUROSCI.2161-10.2010","volume":"30","author":"MV Peelen","year":"2010","unstructured":"Peelen, M. V., Atkinson, A. P. & Vuilleumier, P. Supramodal representations of perceived emotions in the human brain. J Neurosci 30, 10127\u201310134, doi: 10.1523\/JNEUROSCI.2161-10.2010 (2010).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR27","doi-asserted-by":"publisher","first-page":"3863","DOI":"10.1073\/pnas.0600244103","volume":"103","author":"N Kriegeskorte","year":"2006","unstructured":"Kriegeskorte, N., Goebel, R. & Bandettini, P. Information-based functional brain mapping. Proceedings of the National Academy of Sciences of the United States of America 103, 3863\u20133868 (2006).","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"key":"BFsrep37494_CR28","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1016\/j.neuroimage.2008.06.037","volume":"43","author":"F De Martino","year":"2008","unstructured":"De Martino, F. et al. Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage 43, 44\u201358 (2008).","journal-title":"Neuroimage"},{"key":"BFsrep37494_CR29","doi-asserted-by":"publisher","first-page":"809","DOI":"10.1016\/S0896-6273(04)00070-4","volume":"41","author":"MS Beauchamp","year":"2004","unstructured":"Beauchamp, M. S., Lee, K. E., Argall, B. D. & Martin, A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41, 809\u2013823 (2004).","journal-title":"Neuron"},{"key":"BFsrep37494_CR30","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1016\/S1364-6613(00)01501-1","volume":"4","author":"T Allison","year":"2000","unstructured":"Allison, T., Puce, A. & McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn Sci 4, 267\u2013278 (2000).","journal-title":"Trends Cogn Sci"},{"key":"BFsrep37494_CR31","doi-asserted-by":"publisher","first-page":"1445","DOI":"10.1016\/j.neuroimage.2007.06.020","volume":"37","author":"B Kreifelts","year":"2007","unstructured":"Kreifelts, B., Ethofer, T., Grodd, W., Erb, M. & Wildgruber, D. Audiovisual integration of emotional signals in voice and face: An event-related fMRI study. Neuroimage 37, 1445\u20131456 (2007).","journal-title":"Neuroimage"},{"key":"BFsrep37494_CR32","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1002\/hbm.21222","volume":"33","author":"MH Grosbras","year":"2012","unstructured":"Grosbras, M. H., Beaton, S. & Eickhoff, S. B. Brain regions involved in human movement perception: a quantitative voxel-based meta-analysis. Hum Brain Mapp 33, 431\u2013454, doi: 10.1002\/hbm.21222 (2012).","journal-title":"Hum Brain Mapp"},{"key":"BFsrep37494_CR33","doi-asserted-by":"publisher","first-page":"6813","DOI":"10.1523\/JNEUROSCI.4478-13.2014JNEUROSCI.4478-13.2014","volume":"34","author":"R Watson","year":"2014","unstructured":"Watson, R. et al. Crossmodal adaptation in right posterior superior temporal sulcus during face-voice emotional integration. J Neurosci 34, 6813\u20136821, doi: 10.1523\/JNEUROSCI.4478-13.2014JNEUROSCI.4478-13.2014 (2014).","journal-title":"J Neurosci"},{"key":"BFsrep37494_CR34","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1163\/156856897X00357","volume":"10","author":"DH Brainard","year":"1997","unstructured":"Brainard, D. H. The Psychophysics Toolbox. Spat Vis 10, 433\u2013436 (1997).","journal-title":"Spat Vis"},{"key":"BFsrep37494_CR35","unstructured":"Bishop, C. M. Pattern Recognition and Machine Learning. (Springer, 2006)."},{"key":"BFsrep37494_CR36","doi-asserted-by":"publisher","first-page":"27:21","DOI":"10.1145\/1961189.1961199","volume":"2","author":"CC Chang","year":"2011","unstructured":"Chang, C. C. & Lin, C.-J. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 27:21\u201327:27 (2011).","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"BFsrep37494_CR37","doi-asserted-by":"publisher","first-page":"e69328","DOI":"10.1371\/journal.pone.0069328","volume":"8","author":"K Schreiber","year":"2013","unstructured":"Schreiber, K. & Krekelberg, B. The statistical analysis of multi-voxel patterns in functional imaging. PLoS One 8, e69328, doi: 10.1371\/journal.pone.0069328 (2013).","journal-title":"PLoS One"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/srep37494.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/srep37494","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/srep37494.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,5]],"date-time":"2023-01-05T02:30:07Z","timestamp":1672885807000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/srep37494"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,11,24]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2016,12,23]]}},"alternative-id":["BFsrep37494"],"URL":"https:\/\/doi.org\/10.1038\/srep37494","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,11,24]]},"assertion":[{"value":"13 May 2016","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 October 2016","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 November 2016","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing financial interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"37494"}}