{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T19:34:23Z","timestamp":1772307263176,"version":"3.50.1"},"reference-count":62,"publisher":"MIT Press","issue":"3","funder":[{"DOI":"10.13039\/501100001827","name":"University of Amsterdam","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001827","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["339374"],"award-info":[{"award-number":["339374"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Deep convolutional neural networks (DCNNs) are able to partially predict brain activity during object categorization tasks, but factors contributing to this predictive power are not fully understood. Our study aimed to investigate the factors contributing to the predictive power of DCNNs in object categorization tasks. We compared the activity of four DCNN architectures with EEG recordings obtained from 62 human participants during an object categorization task. Previous physiological studies on object categorization have highlighted the importance of figure-ground segregation\u2014the ability to distinguish objects from their backgrounds. Therefore, we investigated whether figure-ground segregation could explain the predictive power of DCNNs. Using a stimulus set consisting of identical target objects embedded in different backgrounds, we examined the influence of object background versus object category within both EEG and DCNN activity. Crucially, the recombination of naturalistic objects and experimentally controlled backgrounds creates a challenging and naturalistic task, while retaining experimental control. Our results showed that early EEG activity (&amp;lt; 100 msec) and early DCNN layers represent object background rather than object category. We also found that the ability of DCNNs to predict EEG activity is primarily influenced by how both systems process object backgrounds, rather than object categories. We demonstrated the role of figure-ground segregation as a potential prerequisite for recognition of object features, by contrasting the activations of trained and untrained (i.e., random weights) DCNNs. These findings suggest that both human visual cortex and DCNNs prioritize the segregation of object backgrounds and target objects to perform object categorization. Altogether, our study provides new insights into the mechanisms underlying object categorization as we demonstrated that both human visual cortex and DCNNs care deeply about object background.<\/jats:p>","DOI":"10.1162\/jocn_a_02098","type":"journal-article","created":{"date-parts":[[2024,1,2]],"date-time":"2024-01-02T11:52:11Z","timestamp":1704196331000},"page":"551-566","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":5,"title":["Human Visual Cortex and Deep Convolutional Neural Network Care Deeply about Object Background"],"prefix":"10.1162","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8859-3802","authenticated-orcid":true,"given":"Jessica","family":"Loke","sequence":"first","affiliation":[{"name":"University of Amsterdam"}]},{"given":"Noor","family":"Seijdel","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"given":"Lukas","family":"Snoek","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5560-6486","authenticated-orcid":true,"given":"Lynn K. A.","family":"S\u00f6rensen","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9851-0409","authenticated-orcid":true,"given":"Ron","family":"van de Klundert","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"given":"Matthew","family":"van der Meer","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"given":"Eva","family":"Quispel","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"given":"Natalie","family":"Cappaert","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9839-1788","authenticated-orcid":true,"given":"H. Steven","family":"Scholte","sequence":"additional","affiliation":[{"name":"University of Amsterdam"}]}],"member":"281","published-online":{"date-parts":[[2024,3,1]]},"reference":[{"key":"2024020716171334000_bib1","doi-asserted-by":"publisher","first-page":"14410","DOI":"10.1109\/ACCESS.2018.2807385","article-title":"Threat of adversarial attacks on deep learning in computer vision: A survey","volume":"6","author":"Akhtar","year":"2018","journal-title":"IEEE Access"},{"key":"2024020716171334000_bib2","doi-asserted-by":"publisher","first-page":"e1006613","DOI":"10.1371\/journal.pcbi.1006613","article-title":"Deep convolutional networks do not classify based on global object shape","volume":"14","author":"Baker","year":"2018","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2103.05137","article-title":"Contemplating real-world object classification","author":"Borji","year":"2021","journal-title":"ArXiv"},{"key":"2024020716171334000_bib4","doi-asserted-by":"publisher","first-page":"e1003963","DOI":"10.1371\/journal.pcbi.1003963","article-title":"Deep neural networks rival the representation of primate IT cortex for core visual object recognition","volume":"10","author":"Cadieu","year":"2014","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2104.01490","article-title":"Explanatory models in neuroscience: Part 1\u2014Taking mechanistic abstraction seriously","author":"Cao","year":"2021","journal-title":"ArXiv"},{"key":"2024020716171334000_bib6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2104.01489","article-title":"Explanatory models in neuroscience: Part 2\u2014Constraint-based intelligibility","author":"Cao","year":"2021","journal-title":"ArXiv"},{"key":"2024020716171334000_bib7","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1016\/j.tics.2019.01.009","article-title":"Deep neural networks as scientific models","volume":"23","author":"Cichy","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2024020716171334000_bib8","doi-asserted-by":"publisher","first-page":"27755","DOI":"10.1038\/srep27755","article-title":"Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence","volume":"6","author":"Cichy","year":"2016","journal-title":"Scientific Reports"},{"key":"2024020716171334000_bib9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2209.11737","article-title":"Semantic scene descriptions as an objective of human vision","author":"Doerig","year":"2022","journal-title":"ArXiv"},{"key":"2024020716171334000_bib10","doi-asserted-by":"publisher","first-page":"e1009267","DOI":"10.1371\/journal.pcbi.1009267","article-title":"Unveiling functions of the visual cortex using task-specific deep neural networks","volume":"17","author":"Dwivedi","year":"2021","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib11","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/j.neuroimage.2016.10.001","article-title":"Seeing it all: Convolutional network layers map the function of the human visual system","volume":"152","author":"Eickenberg","year":"2017","journal-title":"Neuroimage"},{"key":"2024020716171334000_bib12","doi-asserted-by":"publisher","first-page":"1488","DOI":"10.1162\/jocn.2007.19.9.1488","article-title":"Masking disrupts reentrant processing in human visual cortex","volume":"19","author":"Fahrenfort","year":"2007","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2024020716171334000_bib13","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1706.06969","article-title":"Comparing deep neural networks against humans: Object recognition when the signal gets weaker","author":"Geirhos","year":"2017","journal-title":"ArXiv"},{"key":"2024020716171334000_bib14","article-title":"Beyond accuracy: Quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency","author":"Geirhos","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"2024020716171334000_bib15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1811.12231","article-title":"ImageNet-trained CNNs are biased towards texture; Increasing shape bias improves accuracy and robustness","author":"Geirhos","year":"2018","journal-title":"ArXiv"},{"key":"2024020716171334000_bib16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6572","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow","year":"2014","journal-title":"ArXiv"},{"key":"2024020716171334000_bib17","doi-asserted-by":"publisher","first-page":"e1006690","DOI":"10.1371\/journal.pcbi.1006690","article-title":"Scene complexity modulates degree of feedback activity during object detection in natural scenes","volume":"14","author":"Groen","year":"2018","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib18","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1016\/j.neuroimage.2018.12.046","article-title":"The representational dynamics of visual objects in rapid serial visual processing streams","volume":"188","author":"Grootswagers","year":"2019","journal-title":"Neuroimage"},{"key":"2024020716171334000_bib19","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"key":"2024020716171334000_bib20","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","article-title":"Deep residual learning for image recognition","volume-title":"2016 IEEE conference on Computer Vision and Pattern Recognition (CVPR)","author":"He","year":"2016"},{"key":"2024020716171334000_bib21","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1126\/science.1117593","article-title":"Fast readout of object identity from macaque inferior temporal cortex","volume":"310","author":"Hung","year":"2005","journal-title":"Science"},{"key":"2024020716171334000_bib22","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: A 2D graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Computing in Science & Engineering"},{"key":"2024020716171334000_bib23","doi-asserted-by":"publisher","first-page":"974","DOI":"10.1038\/s41593-019-0392-5","article-title":"Evidence that recurrent circuits are critical to the ventral stream\u2019s execution of core object recognition behavior","volume":"22","author":"Kar","year":"2019","journal-title":"Nature Neuroscience"},{"key":"2024020716171334000_bib24","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.neuroimage.2017.08.016","article-title":"Principles for models of neural information processing","volume":"180","author":"Kay","year":"2018","journal-title":"Neuroimage"},{"key":"2024020716171334000_bib25","doi-asserted-by":"publisher","first-page":"e1003915","DOI":"10.1371\/journal.pcbi.1003915","article-title":"Deep supervised, but not unsupervised, models may explain IT cortical representation","volume":"10","author":"Khaligh-Razavi","year":"2014","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib26","doi-asserted-by":"publisher","first-page":"eabe1833","DOI":"10.1126\/sciadv.abe1833","article-title":"The essential role of recurrent processing for figure-ground perception in mice","volume":"7","author":"Kirchberger","year":"2021","journal-title":"Science Advances"},{"key":"2024020716171334000_bib27","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational similarity analysis\u2014Connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Frontiers in Systems Neuroscience"},{"key":"2024020716171334000_bib28","first-page":"1","article-title":"ImageNet classification with deep convolutional neural networks","volume-title":"Advances in neural information processing systems","author":"Krizhevsky","year":"2012"},{"key":"2024020716171334000_bib29","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1016\/S0959-4388(98)80042-1","article-title":"Feedforward, horizontal, and feedback processing in the visual cortex","volume":"8","author":"Lamme","year":"1998","journal-title":"Current Opinion in Neurobiology"},{"key":"2024020716171334000_bib30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1907.06374","article-title":"What does it mean to understand a neural network?","author":"Lillicrap","year":"2019","journal-title":"ArXiv"},{"key":"2024020716171334000_bib31","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"Microsoft COCO: Common Objects in Context","volume":"8693","author":"Lin","year":"2014","journal-title":"Computer Vision \u2013 ECCV"},{"key":"2024020716171334000_bib32","doi-asserted-by":"publisher","first-page":"2390","DOI":"10.1162\/jocn_a_01914","article-title":"A critical test of deep convolutional neural networks\u2019 ability to capture recurrent processing in the brain using visual masking","volume":"34","author":"Loke","year":"2022","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2024020716171334000_bib33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1801.00631","article-title":"Deep learning: A critical appraisal","author":"Marcus","year":"2018","journal-title":"ArXiv"},{"key":"2024020716171334000_bib34","doi-asserted-by":"crossref","first-page":"51","DOI":"10.25080\/Majora-92bf1922-00a","article-title":"Data structures for statistical computing in Python","volume-title":"Proceedings of the 9th Python in Science Conference","author":"McKinney","year":"2010"},{"key":"2024020716171334000_bib35","doi-asserted-by":"publisher","first-page":"5725","DOI":"10.1038\/s41467-020-19632-w","article-title":"Individual differences among deep neural network models","volume":"11","author":"Mehrer","year":"2020","journal-title":"Nature Communications"},{"key":"2024020716171334000_bib36","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1016\/j.tics.2007.09.009","article-title":"The role of context in object recognition","volume":"11","author":"Oliva","year":"2007","journal-title":"Trends in Cognitive Sciences"},{"key":"2024020716171334000_bib37","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume-title":"Advances in neural information processing systems","author":"Paszke","year":"2019"},{"key":"2024020716171334000_bib38","doi-asserted-by":"publisher","first-page":"7255","DOI":"10.1523\/JNEUROSCI.0388-18.2018","article-title":"Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks","volume":"38","author":"Rajalingham","year":"2018","journal-title":"Journal of Neuroscience"},{"key":"2024020716171334000_bib39","doi-asserted-by":"publisher","first-page":"7103","DOI":"10.1523\/JNEUROSCI.2628-20.2021","article-title":"Untangling the animacy organization of occipitotemporal cortex","volume":"41","author":"Ritchie","year":"2021","journal-title":"Journal of Neuroscience"},{"key":"2024020716171334000_bib40","article-title":"Cognitive psychology for deep neural networks: A shape bias case study","volume-title":"International Conference on Machine Learning","author":"Ritter","year":"2017"},{"key":"2024020716171334000_bib41","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1146\/annurev.neuro.29.051605.112939","article-title":"Cortical algorithms for perceptual grouping","volume":"29","author":"Roelfsema","year":"2006","journal-title":"Annual Review of Neuroscience"},{"key":"2024020716171334000_bib42","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1162\/08989290260045756","article-title":"Figure\u2014ground segregation in a recurrent network architecture","volume":"14","author":"Roelfsema","year":"2002","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2024020716171334000_bib43","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.neuroimage.2017.12.077","article-title":"Fantastic DNimals and where to find them","volume":"180","author":"Scholte","year":"2018","journal-title":"Neuroimage"},{"key":"2024020716171334000_bib44","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1167\/9.4.29","article-title":"Brain responses strongly correlate with Weibull image statistics when processing natural images","volume":"9","author":"Scholte","year":"2009","journal-title":"Journal of Vision"},{"key":"2024020716171334000_bib45","doi-asserted-by":"publisher","first-page":"2097","DOI":"10.1162\/jocn.2008.20142","article-title":"Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging","volume":"20","author":"Scholte","year":"2008","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2024020716171334000_bib46","doi-asserted-by":"publisher","first-page":"407007","DOI":"10.1101\/407007","article-title":"Brain-score: Which artificial neural network for object recognition is most brain-like?","author":"Schrimpf","year":"2018","journal-title":"BioRxiv"},{"key":"2024020716171334000_bib47","doi-asserted-by":"crossref","first-page":"61","DOI":"10.25080\/Majora-92bf1922-011","article-title":"Statsmodels: Econometric and statistical modeling with python","volume-title":"Proceedings of the 9th Python in science conference","author":"Seabold","year":"2010"},{"key":"2024020716171334000_bib48","doi-asserted-by":"publisher","first-page":"6281","DOI":"10.1523\/JNEUROSCI.2851-20.2021","volume":"41","author":"Seijdel","year":"2021","journal-title":"Journal of Neuroscience"},{"key":"2024020716171334000_bib49","doi-asserted-by":"publisher","first-page":"e1008022","DOI":"10.1371\/journal.pcbi.1008022","article-title":"Depth in convolutional neural networks solves scene segmentation","volume":"16","author":"Seijdel","year":"2020","journal-title":"PLoS Computational Biology"},{"key":"2024020716171334000_bib50","doi-asserted-by":"publisher","first-page":"1019","DOI":"10.1016\/j.cub.2019.02.016","article-title":"The segmentation of proto-objects in the monkey primary visual cortex","volume":"29","author":"Self","year":"2019","journal-title":"Current Biology"},{"key":"2024020716171334000_bib51","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.1556","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014","journal-title":"ArXiv"},{"key":"2024020716171334000_bib52","doi-asserted-by":"publisher","first-page":"2044","DOI":"10.1162\/jocn_a_01755","article-title":"Diverse deep neural networks all predict human inferior temporal cortex well, after training and fitting","volume":"33","author":"Storrs","year":"2021","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2024020716171334000_bib53","doi-asserted-by":"publisher","first-page":"517004","DOI":"10.1101\/2022.11.18.517004","article-title":"High-resolution image reconstruction with latent diffusion models from human brain activity","volume":"11","author":"Takagi","year":"2022","journal-title":"BioRxiv"},{"key":"2024020716171334000_bib54","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2202.08340","article-title":"A developmentally-inspired examination of shape versus texture bias in machines","author":"Tartaglini","year":"2022","journal-title":"ArXiv"},{"key":"2024020716171334000_bib55","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1038\/381520a0","article-title":"Speed of processing in the human visual system","volume":"381","author":"Thorpe","year":"1996","journal-title":"Nature"},{"key":"2024020716171334000_bib56","article-title":"Understanding CNNs as a model of the inferior temporal cortex: Using mediation analysis to unpack the contribution of perceptual and semantic features in random and trained networks","author":"Truzzi","year":"2020","journal-title":"NeurIPS 2020 Workshop SVRHM"},{"key":"2024020716171334000_bib57","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"van der Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"2024020716171334000_bib58","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental algorithms for scientific computing in Python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nature Methods"},{"key":"2024020716171334000_bib59","doi-asserted-by":"publisher","first-page":"3021","DOI":"10.21105\/joss.03021","article-title":"seaborn: Statistical data visualization","volume":"6","author":"Waskom","year":"2021","journal-title":"Journal of Open Source Software"},{"key":"2024020716171334000_bib60","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1038\/nn.4244","article-title":"Using goal-driven deep learning models to understand sensory cortex","volume":"19","author":"Yamins","year":"2016","journal-title":"Nature Neuroscience"},{"key":"2024020716171334000_bib61","doi-asserted-by":"publisher","first-page":"8619","DOI":"10.1073\/pnas.1403112111","article-title":"Performance-optimized hierarchical models predict neural responses in higher visual cortex","volume":"111","author":"Yamins","year":"2014","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2024020716171334000_bib62","doi-asserted-by":"publisher","first-page":"7376","DOI":"10.1523\/JNEUROSCI.16-22-07376.1996","article-title":"Contextual modulation in primary visual cortex","volume":"16","author":"Zipser","year":"1996","journal-title":"Journal of Neuroscience"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/3\/551\/2329122\/jocn_a_02098.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/3\/551\/2329122\/jocn_a_02098.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,2,7]],"date-time":"2024-02-07T11:17:46Z","timestamp":1707304666000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/36\/3\/551\/118876\/Human-Visual-Cortex-and-Deep-Convolutional-Neural"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":62,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,3,1]]},"published-print":{"date-parts":[[2024,3,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_02098","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.04.14.536853","asserted-by":"object"}]},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}