{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T15:48:26Z","timestamp":1753890506325,"version":"3.41.2"},"reference-count":56,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006502","name":"Defense Sciences Office, DARPA","doi-asserted-by":"publisher","award":["HR00112290074"],"award-info":[{"award-number":["HR00112290074"]}],"id":[{"id":"10.13039\/100006502","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Sci."],"abstract":"<jats:p>Artificial neural networks (ANNs) are sensitive to perturbations and adversarial attacks. One hypothesized solution to adversarial robustness is to align manifolds in the embedded space of neural networks with biologically grounded manifolds. Recent state-of-the-art works that emphasize learning robust neural representations, rather than optimizing for a specific target task like classification, support the idea that researchers should investigate this hypothesis. While works have shown that fine-tuning ANNs to coincide with biological vision does increase robustness to both perturbations and adversarial attacks, these works have relied on proprietary datasets\u2014the lack of publicly available biological benchmarks makes it difficult to evaluate the efficacy of these claims. Here, we deliver a curated dataset consisting of biological representations of images taken from two commonly used computer vision datasets, ImageNet and COCO, that can be easily integrated into model training and evaluation. Specifically, we take a large functional magnetic resonance imaging (fMRI) dataset (BOLD5000), preprocess it into representational dissimilarity matrices (RDMs), and establish an infrastructure that anyone can use to train models with biologically grounded representations. Using this infrastructure, we investigate the representations of several popular neural networks and find that as networks have been optimized for tasks, their correspondence with biological fidelity has decreased. Additionally, we use a previously unexplored graph-based technique, Fiedler partitioning, to showcase the viability of the biological data, and the potential to extend these analyses by extending RDMs into Laplacian matrices. Overall, our findings demonstrate the potential of utilizing our new biological benchmark to effectively enhance the robustness of models.<\/jats:p>","DOI":"10.3389\/fcomp.2023.1275026","type":"journal-article","created":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T05:47:18Z","timestamp":1702964838000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring fMRI RDMs: enhancing model robustness through neurobiological data"],"prefix":"10.3389","volume":"5","author":[{"given":"William","family":"Pickard","sequence":"first","affiliation":[]},{"given":"Kelsey","family":"Sikes","sequence":"additional","affiliation":[]},{"given":"Huma","family":"Jamil","sequence":"additional","affiliation":[]},{"given":"Nicholas","family":"Chaffee","sequence":"additional","affiliation":[]},{"given":"Nathaniel","family":"Blanchard","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Kirby","sequence":"additional","affiliation":[]},{"given":"Chris","family":"Peterson","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,12,19]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/j.neunet.2023.04.032","article-title":"Contrast sensitivity function in deep networks","volume":"164","author":"Akbarinia","year":"2023","journal-title":"Neural Netw."},{"key":"B2","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1038\/s41597-020-00680-2","article-title":"A naturalistic neuroimaging database for understanding the brain using ecological stimuli","volume":"7","author":"Aliko","year":"2020","journal-title":"Sci. Data"},{"key":"B3","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/s41593-021-00962-x","article-title":"A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence","volume":"25","author":"Allen","year":"2022","journal-title":"Nat. Neurosci."},{"key":"B4","doi-asserted-by":"crossref","first-page":"5319","DOI":"10.1109\/ICCV.2019.00542","article-title":"\u201cTeacher guided architecture search,\u201d","volume-title":"2019 IEEE\/CVF International Conference on Computer Vision (ICCV)","author":"Bashivan","year":"2019"},{"key":"B5","doi-asserted-by":"crossref","first-page":"5399","DOI":"10.1109\/CVPR.2019.00555","article-title":"\u201cA neurobiological evaluation metric for neural network model search,\u201d","volume-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Blanchard","year":"2019"},{"key":"B6","unstructured":"BlanchardN. T.\n          Ph.D. thesisQuantifying internal representation for use in model search2019"},{"key":"B7","unstructured":"ChangN.\n            PylesJ.\n            PrinceJ.\n            TarrM.\n            AminoffE.\n          10.1184\/R1\/14456124.2021"},{"key":"B8","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1038\/s41597-019-0052-3","article-title":"BOLD5000, a public fMRI dataset while viewing 5000 visual images","volume":"6","author":"Chang","year":"2019","journal-title":"Sci. Data"},{"key":"B9","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","article-title":"\u201cImageNet: a large-scale hierarchical image database,\u201d","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng","year":"2009"},{"key":"B10","doi-asserted-by":"publisher","first-page":"1997","DOI":"10.48550\/arXiv.1808.05377","article-title":"Neural architecture search: a survey","volume":"20","author":"Elsken","year":"2018","journal-title":"J. Mach. Learn. Res."},{"key":"B11","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1038\/s41592-018-0235-4","article-title":"fMRIPrep: a robust preprocessing pipeline for functional MRI","volume":"16","author":"Esteban","year":"2019","journal-title":"Nat. Methods"},{"key":"B12","doi-asserted-by":"crossref","first-page":"298","DOI":"10.21136\/CMJ.1973.101168","article-title":"Algebraic connectivity of graphs","volume":"23","author":"Fiedler","year":"1973","journal-title":"Czechoslovak Math. J."},{"key":"B13","doi-asserted-by":"crossref","first-page":"619","DOI":"10.21136\/CMJ.1975.101357","article-title":"A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory","volume":"25","author":"Fiedler","year":"1975","journal-title":"Czechoslovak Math. J."},{"key":"B14","doi-asserted-by":"publisher","first-page":"774","DOI":"10.1016\/j.neuroimage.2012.01.021","article-title":"FreeSurfer","volume":"62","author":"Fischl","year":"2012","journal-title":"Neuroimage"},{"key":"B15","article-title":"\u201cGeneralisation in humans and deep neural networks,\u201d","author":"Geirhos","year":"2018","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.23915\/distill.00030","article-title":"Multimodal neurons in artificial neural networks","author":"Goh","year":"2021","journal-title":"Distill"},{"key":"B17","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.visres.2020.07.010","article-title":"Color illusions also deceive CNNs for low-level vision tasks: analysis and implications","volume":"176","author":"Gomez-Villa","year":"2020","journal-title":"Vision Res."},{"key":"B18","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"\u201cDeep residual learning for image recognition,\u201d","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"B19","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.patrec.2023.07.017","article-title":"Neural networks with divisive normalization for image segmentation","volume":"173","author":"Hern\u00e1ndez-C\u00e1mara","year":"2023","journal-title":"Pattern Recogn. Lett."},{"key":"B20","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1038\/nn.4247","article-title":"Explicit information for category-orthogonal object properties increases along the ventral stream","volume":"19","author":"Hong","year":"2016","journal-title":"Nat. Neurosci."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1806.10332","article-title":"MONAS: multi-objective neural architecture search using reinforcement learning","author":"Hsu","year":"2018","journal-title":"arXiv [Preprint]. arXiv:1806.10332"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1872","DOI":"10.1038\/s41467-021-22078-3","article-title":"Qualitative similarities and differences in visual object representations between brains and deep networks","volume":"12","author":"Jacob","year":"2021","journal-title":"Nat. Commun."},{"key":"B23","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1109\/CVPRW59228.2023.00066","article-title":"\u201cHamming similarity and graph Laplacians for class partitioning and adversarial image detection,\u201d","volume-title":"2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","author":"Jamil","year":"2023"},{"key":"B24","doi-asserted-by":"crossref","first-page":"4302","DOI":"10.1523\/JNEUROSCI.17-11-04302.1997","article-title":"The fusiform face area: a module in human extrastriate cortex specialized for face perception","volume":"17","author":"Kanwisher","year":"1997","journal-title":"J. Neurosci."},{"key":"B25","doi-asserted-by":"publisher","first-page":"32672","DOI":"10.1038\/srep32672","article-title":"Deep networks can resemble human feed-forward vision in invariant object recognition","volume":"6","author":"Kheradpisheh","year":"2016","journal-title":"Sci. Rep."},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1312.6114","article-title":"Auto-encoding variational Bayes","author":"Kingma","year":"2013","journal-title":"arXiv [Preprint]. arXiv:1312.6114"},{"key":"B27","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational similarity analysis - connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Front. Syst. Neurosci."},{"key":"B28","article-title":"\u201cImageNet classification with deep convolutional neural networks,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Krizhevsky","year":"2012"},{"year":"2022","author":"Kumar","key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.04946"},{"key":"B30","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1167\/jov.22.6.8","article-title":"Contrast sensitivity functions in autoencoders","volume":"22","author":"Li","year":"2022","journal-title":"J. Vision"},{"key":"B31","article-title":"\u201cLearning from brains how to regularize machines,\u201d","author":"Li","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B32","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"\u201cMicrosoft COCO: common objects in context,\u201d","author":"Lin","year":"2014","journal-title":"Computer Vision \u2013 ECCV 2014"},{"key":"B33","first-page":"19","article-title":"\u201cProgressive neural architecture search,\u201d","author":"Liu","year":"2018","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"B34","doi-asserted-by":"publisher","first-page":"e2011417118","DOI":"10.1073\/pnas.2011417118","article-title":"An ecologically motivated image dataset for deep learning yields better models of human vision","volume":"118","author":"Mehrer","year":"2021","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B35","article-title":"\u201cDivisive feature normalization improves image recognition performance in AlexNet,\u201d","author":"Miller","year":"2021","journal-title":"International Conference on Learning Representations"},{"key":"B36","doi-asserted-by":"publisher","first-page":"e1003553","DOI":"10.1371\/journal.pcbi.1003553","article-title":"A toolbox for representational similarity analysis","volume":"10","author":"Nili","year":"2014","journal-title":"PLoS Comput. Biol."},{"key":"B37","first-page":"4095","article-title":"\u201cEfficient neural architecture search via parameters sharing,\u201d","volume-title":"Proceedings of the 35th International Conference on Machine Learning","author":"Pham","year":"2018"},{"key":"B38","doi-asserted-by":"publisher","DOI":"10.1101\/2022.01.31.478431","article-title":"GLMsingle: a toolbox for improving single-trial fMRI response estimates","author":"Prince","year":"2022","journal-title":"bioRxiv [Preprint]"},{"key":"B39","first-page":"8748","article-title":"\u201cLearning transferable visual models from natural language supervision,\u201d","volume-title":"Proceedings of the 38th International Conference on Machine Learning","author":"Radford","year":"2021"},{"key":"B40","doi-asserted-by":"publisher","first-page":"2280","DOI":"10.1109\/TPAMI.2018.2849989","article-title":"PsyPhy: a psychophysics driven evaluation framework for visual recognition","volume":"41","author":"RichardWebster","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"B41","first-page":"252","article-title":"\u201cVisual psychophysics for making face recognition algorithms more explainable,\u201d","author":"RichardWebster","year":"2018","journal-title":"Proceedings of the European Conference on Computer Vision (ECCV)"},{"key":"B42","doi-asserted-by":"crossref","first-page":"3546","DOI":"10.1109\/CVPR46437.2021.00355","article-title":"\u201cEnriching ImageNet with human similarity judgments and psychological embeddings,\u201d","volume-title":"2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Roads","year":"2021"},{"key":"B43","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1093\/cercor\/bhaa246","article-title":"A probabilistic functional atlas of human occipito-temporal visual cortex","volume":"31","author":"Rosenke","year":"2021","journal-title":"Cereb. Cortex"},{"key":"B44","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1016\/j.neuroimage.2017.02.040","article-title":"A cross-validated cytoarchitectonic atlas of the human ventral visual stream","volume":"170","author":"Rosenke","year":"2018","journal-title":"Neuroimage"},{"key":"B45","doi-asserted-by":"crossref","first-page":"4510","DOI":"10.1109\/CVPR.2018.00474","article-title":"\u201cMobileNetV2: inverted residuals and linear bottlenecks,\u201d","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Sandler","year":"2018"},{"year":"2018","author":"Schrimpf","key":"B46","doi-asserted-by":"publisher","DOI":"10.1101\/407007"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abm2219","article-title":"Reassessing hierarchical correspondences between brain and deep networks through direct interface","author":"Sexton","year":"2022","journal-title":"Sci. Adv."},{"key":"B48","doi-asserted-by":"publisher","first-page":"2124","DOI":"10.3389\/fpsyg.2017.02124","article-title":"A developmental approach to machine learning?","volume":"8","author":"Smith","year":"2017","journal-title":"Front. Psychol."},{"key":"B49","doi-asserted-by":"publisher","first-page":"2044","DOI":"10.1162\/jocn_a_01755","article-title":"Diverse deep neural networks all predict human inferior temporal cortex well, after training and fitting","volume":"33","author":"Storrs","year":"2021","journal-title":"J. Cogn. Neurosci."},{"key":"B50","first-page":"6105","article-title":"\u201cEfficientNet: rethinking model scaling for convolutional neural networks,\u201d","author":"Tan","year":"2019","journal-title":"Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research"},{"key":"B51","article-title":"\u201cBio-inspired learnable divisive normalization for ANNs,\u201d","author":"Veerabadran","year":"2021","journal-title":"SVRHM 2021 Workshop@NeurIPS"},{"key":"B52","doi-asserted-by":"crossref","first-page":"3485","DOI":"10.1109\/CVPR.2010.5539970","article-title":"\u201cSUN database: large-scale scene recognition from abbey to zoo,\u201d","volume-title":"2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Xiao","year":"2010"},{"key":"B53","article-title":"\u201cHierarchical modular optimization of convolutional networks achieves representations similar to macaque IT and human ventral stream,\u201d","author":"Yamins","year":"2013","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B54","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1038\/nn.4244","article-title":"Using goal-driven deep learning models to understand sensory cortex","volume":"19","author":"Yamins","year":"2016","journal-title":"Nat. Neurosci."},{"key":"B55","doi-asserted-by":"publisher","first-page":"8619","DOI":"10.1073\/pnas.1403112111","article-title":"Performance-optimized hierarchical models predict neural responses in higher visual cortex","volume":"111","author":"Yamins","year":"2014","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B56","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1109\/CVPR.2018.00068","article-title":"\u201cThe unreasonable effectiveness of deep features as a perceptual metric,\u201d","volume-title":"2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhang","year":"2018"}],"container-title":["Frontiers in Computer Science"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1275026\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T05:47:35Z","timestamp":1702964855000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fcomp.2023.1275026\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":56,"alternative-id":["10.3389\/fcomp.2023.1275026"],"URL":"https:\/\/doi.org\/10.3389\/fcomp.2023.1275026","relation":{},"ISSN":["2624-9898"],"issn-type":[{"type":"electronic","value":"2624-9898"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"article-number":"1275026"}}