{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T06:21:49Z","timestamp":1772173309153,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011887","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000}}],"reference-count":52,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The Clare Boothe Luce Program for Women"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that \u201cdisentangles\u201d different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model\u2019s learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few \"entangled\" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard \u201cblack box\u201d deep learning approaches for modeling and interpreting human brain data.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011887","type":"journal-article","created":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T13:41:18Z","timestamp":1708954878000},"page":"e1011887","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Disentangled deep generative models reveal coding principles of the human face processing network"],"prefix":"10.1371","volume":"20","author":[{"given":"Paul","family":"Soulos","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9255-0151","authenticated-orcid":true,"given":"Leyla","family":"Isik","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2024,2,26]]},"reference":[{"issue":"49","key":"pcbi.1011887.ref001","doi-asserted-by":"crossref","first-page":"19514","DOI":"10.1073\/pnas.0809662105","article-title":"Comparing face patch systems in macaques and humans","volume":"105","author":"D. Y. Tsao","year":"2008","journal-title":"Proc. Natl. Acad. Sci. U. S. A."},{"issue":"1","key":"pcbi.1011887.ref002","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1146\/annurev-vision-082114-035518","article-title":"A Revised Neural Framework for Face Processing","volume":"1","author":"B. Duchaine","year":"2015","journal-title":"Annu. Rev. Vis. Sci."},{"key":"pcbi.1011887.ref003","first-page":"325","article-title":"Face Processing Systems: From Neurons to Real-World Social Perception","volume":"39","author":"W. Freiwald","year":"2016"},{"issue":"4","key":"pcbi.1011887.ref004","doi-asserted-by":"crossref","first-page":"2356","DOI":"10.1016\/j.neuroimage.2011.03.067","article-title":"Differential selectivity for dynamic versus static information in face-selective cortical regions","volume":"56","author":"D. Pitcher","year":"2011","journal-title":"Neuroimage"},{"issue":"29","key":"pcbi.1011887.ref005","doi-asserted-by":"crossref","first-page":"11768","DOI":"10.1523\/JNEUROSCI.5402-11.2013","article-title":"Faces in Motion: Selectivity of Macaque and Human Face Processing Areas for Dynamic Stimuli","volume":"33","author":"P. Polosecki","year":"2013","journal-title":"J. Neurosci."},{"issue":"6005","key":"pcbi.1011887.ref006","doi-asserted-by":"crossref","first-page":"845","DOI":"10.1126\/science.1194908","article-title":"Functional compartmentalization and viewpoint generalization within the macaque face-processing system.","volume":"330","author":"W. A. Freiwald","year":"2010","journal-title":"Science"},{"issue":"51","key":"pcbi.1011887.ref007","doi-asserted-by":"crossref","first-page":"20600","DOI":"10.1073\/pnas.0705654104","article-title":"Individual faces elicit distinct response patterns in human anterior temporal cortex","volume":"104","author":"N. Kriegeskorte","year":"2007","journal-title":"Proc. Natl. Acad. Sci. U. S. A."},{"issue":"9","key":"pcbi.1011887.ref008","doi-asserted-by":"crossref","first-page":"794","DOI":"10.1016\/j.tics.2018.06.006","article-title":"Face Space Representations in Deep Convolutional Neural Networks","volume":"22","author":"A. J. O\u2019Toole","year":"2018","journal-title":"Trends Cogn. Sci."},{"key":"pcbi.1011887.ref009","first-page":"543","article-title":"Face Recognition by Humans and Machines: Three Fundamental Advances from Deep Learning","volume":"7","author":"A. J. O\u2019Toole","year":"2021"},{"issue":"43","key":"pcbi.1011887.ref010","doi-asserted-by":"crossref","first-page":"e2304085120","DOI":"10.1073\/pnas.2304085120","article-title":"Modeling naturalistic face processing in humans with deep convolutional neural networks","volume":"120","author":"G. Jiahui","year":"2023","journal-title":"Proc. Natl. Acad. Sci."},{"issue":"1","key":"pcbi.1011887.ref011","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1038\/s42003-019-0438-y","article-title":"Reconstructing faces from fMRI patterns using deep generative neural networks","volume":"2","author":"R. VanRullen","year":"2019","journal-title":"Commun. Biol."},{"key":"pcbi.1011887.ref012","article-title":"Deep Convolutional Inverse Graphics Network","volume":"28","author":"T. D. Kulkarni","year":"2015","journal-title":"Advances in {Neural} {Information} {Processing} {Systems}"},{"key":"pcbi.1011887.ref013","article-title":"Infogan: Interpretable representation learning by information maximizing generative adversarial nets","volume":"29","author":"X. Chen","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"10","key":"pcbi.1011887.ref014","doi-asserted-by":"crossref","DOI":"10.1167\/13.10.1","article-title":"Representational dynamics of object vision: the first 1000 ms.","volume":"13","author":"T. Carlson","year":"2013","journal-title":"J. Vis."},{"key":"pcbi.1011887.ref015","article-title":"Isolating sources of disentanglement in variational autoencoders","volume":"31","author":"R. T. Q. Chen","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"pcbi.1011887.ref016","article-title":"Disentangling by Factorising","author":"H. Kim","year":"2018"},{"key":"pcbi.1011887.ref017","article-title":"beta-{VAE}: Learning Basic Visual Concepts with a Constrained Variational Framework","author":"I. Higgins","year":"2017","journal-title":"International Conference on Learning Representations"},{"key":"pcbi.1011887.ref018","article-title":"Auto-Encoding Variational Bayes","author":"D. P. Kingma","year":"2014","journal-title":"no. arXiv:1312.6114."},{"issue":"8","key":"pcbi.1011887.ref019","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Y. Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"pcbi.1011887.ref020","first-page":"3730","article-title":"Deep Learning Face Attributes in the Wild","author":"Z. Liu","year":"2015"},{"key":"pcbi.1011887.ref021","article-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","author":"K. Simonyan","year":"2014"},{"key":"pcbi.1011887.ref022","first-page":"1","article-title":"Deep face recognition","author":"O. Parkhi","year":"2015","journal-title":"BMVC 2015\u2014Proceedings of the British Machine Vision Conference 2015"},{"key":"pcbi.1011887.ref023","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.cobeha.2020.12.008","article-title":"Extensive sampling for complete models of individual brains","volume":"40","author":"T. Naselaris","year":"2021","journal-title":"Curr. Opin. Behav. Sci."},{"issue":"2","key":"pcbi.1011887.ref024","doi-asserted-by":"crossref","first-page":"1177","DOI":"10.1152\/jn.00032.2010","article-title":"New method for fMRI investigations of language: defining ROIs functionally in individual subjects.","volume":"104","author":"E. Fedorenko","year":"2010","journal-title":"J. Neurophysiol."},{"key":"pcbi.1011887.ref025","doi-asserted-by":"crossref","first-page":"118741","DOI":"10.1016\/j.neuroimage.2021.118741","article-title":"Functional selectivity for social interaction perception in the human superior temporal sulcus during natural viewing","volume":"245","author":"H. Lee Masson","year":"2021","journal-title":"Neuroimage"},{"issue":"6","key":"pcbi.1011887.ref026","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/S1364-6613(00)01482-0","article-title":"The distributed human neural system for face perception","volume":"4","author":"J. V. Haxby","year":"2000","journal-title":"Trends Cogn. Sci."},{"issue":"11","key":"pcbi.1011887.ref027","doi-asserted-by":"crossref","first-page":"2381","DOI":"10.1093\/brain\/awg241","article-title":"A network of occipito-temporal face-sensitive areas besides the right middle fusiform gyrus is necessary for normal face processing","volume":"126","author":"B. Rossion","year":"2003","journal-title":"Brain"},{"issue":"9","key":"pcbi.1011887.ref028","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.1523\/JNEUROSCI.1449-20.2020","article-title":"FFA and OFA Encode Distinct Types of Face Identity Information","volume":"41","author":"M. Tsantani","year":"2021","journal-title":"J. Neurosci."},{"issue":"7","key":"pcbi.1011887.ref029","doi-asserted-by":"crossref","first-page":"e69684","DOI":"10.1371\/journal.pone.0069684","article-title":"Multivoxel Patterns in Fusiform Face Area Differentiate Faces by Sex and Race","volume":"8","author":"J. M. Contreras","year":"2013","journal-title":"PLoS One"},{"key":"pcbi.1011887.ref030","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.neuroimage.2016.01.045","article-title":"Face-selective regions differ in their ability to classify facial expressions","volume":"130","author":"H. Zhang","year":"2016","journal-title":"Neuroimage"},{"key":"pcbi.1011887.ref031","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.neuroimage.2014.04.032","article-title":"Brain regions involved in processing facial identity and expression are differentially selective for surface and edge information","volume":"97","author":"R. J. Harris","year":"2014","journal-title":"Neuroimage"},{"key":"pcbi.1011887.ref032","article-title":"Evaluating the Disentanglement of Deep Generative Models through Manifold Topology","author":"S. Zhou","year":"2020","journal-title":"ICLR 2021 - 9th Int. Conf. Learn. Represent."},{"key":"pcbi.1011887.ref033","first-page":"2021.11.17.469009","article-title":"Modeling naturalistic face processing in humans with deep convolutional neural networks","author":"G. Jiahui","year":"2022","journal-title":"bioRxiv"},{"issue":"10","key":"pcbi.1011887.ref034","doi-asserted-by":"crossref","first-page":"eaax5979","DOI":"10.1126\/sciadv.aax5979","article-title":"Efficient inverse graphics in biological face processing","volume":"6","author":"I. Yildirim","year":"2020","journal-title":"Sci. Adv."},{"issue":"35","key":"pcbi.1011887.ref035","doi-asserted-by":"crossref","first-page":"eadg1736","DOI":"10.1126\/sciadv.adg1736","article-title":"The neural code for \u2018face cells\u2019 is not face-specific","volume":"9","author":"K. Vinken","year":"2023","journal-title":"Sci. Adv."},{"key":"pcbi.1011887.ref036","first-page":"248","article-title":"ImageNet: A large-scale hierarchical image database","author":"R. Socher","year":"2009","journal-title":"in 2009 IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"pcbi.1011887.ref037","first-page":"2022.07.13.499969","article-title":"High-performing neural network models of visual cortex benefit from high latent dimensionality","author":"E. Elmoznino","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011887.ref038","first-page":"2022.03.28.485868","article-title":"What can 1.8 billion regressions tell us about the pressures shaping high-level visual representation in brains and machines?","author":"C. Conwell","year":"2023","journal-title":"bioRxiv"},{"key":"pcbi.1011887.ref039","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.neuroimage.2019.05.039","article-title":"Variational autoencoder: An unsupervised model for encoding and decoding fMRI activity in visual cortex","volume":"198","author":"K. Han","year":"2019","journal-title":"Neuroimage"},{"key":"pcbi.1011887.ref040","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1016\/j.neuroimage.2018.07.043","article-title":"Generative adversarial networks for reconstructing natural images from brain activity","volume":"181","author":"K. Seeliger","year":"2018","journal-title":"Neuroimage"},{"key":"pcbi.1011887.ref041","article-title":"Reconstructing perceived faces from brain activations with deep adversarial neural decoding","volume":"30","author":"Y. G\u00fc\u00e7l\u00fct\u00fcrk","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"9","key":"pcbi.1011887.ref042","doi-asserted-by":"crossref","first-page":"1187","DOI":"10.1038\/nn.2363","article-title":"A face feature space in the macaque temporal lobe","volume":"12","author":"W. A. Freiwald","year":"2009","journal-title":"Nat. Neurosci."},{"issue":"1","key":"pcbi.1011887.ref043","first-page":"1","article-title":"Unsupervised deep learning identifies semantic disentanglement in single inferotemporal face patch neurons","volume":"12","author":"I. Higgins","year":"2021","journal-title":"Nat. Commun. 2021 121"},{"issue":"6","key":"pcbi.1011887.ref044","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1016\/j.cell.2017.05.011","article-title":"The Code for Facial Identity in the Primate Brain","volume":"169","author":"L. Chang","year":"2017","journal-title":"Cell"},{"key":"pcbi.1011887.ref045","doi-asserted-by":"crossref","article-title":"Towards decolonising computational sciences","author":"A. Birhane","DOI":"10.7146\/kkf.v29i2.124899"},{"key":"pcbi.1011887.ref046","first-page":"8771","article-title":"Domain-Invariant Disentangled Network for Generalizable Object Detection","author":"C. Lin","year":"2021"},{"key":"pcbi.1011887.ref047","first-page":"616","article-title":"BlobGAN: Spatially Disentangled Scene Representations","author":"D. Epstein","year":"2022"},{"key":"pcbi.1011887.ref048","article-title":"Unsupervised Learning of Disentangled Representations from Video","volume":"30","author":"E. L. Denton","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"pcbi.1011887.ref049","first-page":"4114","article-title":"Challenging common assumptions in the unsupervised learning of disentangled representations","author":"F. Locatello","year":"2019","journal-title":"in international conference on machine learning"},{"key":"pcbi.1011887.ref050","article-title":"Unsupervised Model Selection for Variational Disentangled Representation Learning","author":"S. Duan","year":"2019"},{"issue":"2","key":"pcbi.1011887.ref051","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1006\/nimg.1998.0396","article-title":"Cortical Surface-Based Analysis","volume":"9","author":"B. Fischl","year":"1999","journal-title":"Neuroimage"},{"issue":"4","key":"pcbi.1011887.ref052","doi-asserted-by":"crossref","first-page":"2357","DOI":"10.1016\/j.neuroimage.2012.02.055","article-title":"An algorithmic method for functionally defining regions of interest in the ventral visual pathway.","volume":"60","author":"J. B. Julian","year":"2012","journal-title":"Neuroimage"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011887","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T00:00:00Z","timestamp":1709769600000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011887","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,7]],"date-time":"2024-03-07T14:05:20Z","timestamp":1709820320000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011887"}},"subtitle":[],"editor":[{"given":"Jean","family":"Daunizeau","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2024,2,26]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2,26]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011887","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.02.15.528489","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,26]]}}}