{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T09:33:04Z","timestamp":1775986384585,"version":"3.50.1"},"reference-count":56,"publisher":"MIT Press","issue":"12","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Recurrent processing is a crucial feature in human visual processing supporting perceptual grouping, figure-ground segmentation, and recognition under challenging conditions. There is a clear need to incorporate recurrent processing in deep convolutional neural networks, but the computations underlying recurrent processing remain unclear. In this article, we tested a form of recurrence in deep residual networks (ResNets) to capture recurrent processing signals in the human brain. Although ResNets are feedforward networks, they approximate an excitatory additive form of recurrence. Essentially, this form of recurrence consists of repeating excitatory activations in response to a static stimulus. Here, we used ResNets of varying depths (reflecting varying levels of recurrent processing) to explain EEG activity within a visual masking paradigm. Sixty-two humans and 50 artificial agents (10 ResNet models of depths \u22124, 6, 10, 18, and 34) completed an object categorization task. We show that deeper networks explained more variance in brain activity compared with shallower networks. Furthermore, all ResNets captured differences in brain activity between unmasked and masked trials, with differences starting at \u223c98 msec (from stimulus onset). These early differences indicated that EEG activity reflected \u201cpure\u201d feedforward signals only briefly (up to \u223c98 msec). After \u223c98 msec, deeper networks showed a significant increase in explained variance, which peaks at \u223c200 msec, but only within unmasked trials, not masked trials. In summary, we provided clear evidence that excitatory additive recurrent processing in ResNets captures some of the recurrent processing in humans.<\/jats:p>","DOI":"10.1162\/jocn_a_01914","type":"journal-article","created":{"date-parts":[[2022,9,19]],"date-time":"2022-09-19T16:22:37Z","timestamp":1663604557000},"page":"2390-2405","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["A Critical Test of Deep Convolutional Neural Networks' Ability to Capture Recurrent Processing in the Brain Using Visual Masking"],"prefix":"10.1162","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8859-3802","authenticated-orcid":true,"given":"Jessica","family":"Loke","sequence":"first","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"Noor","family":"Seijdel","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"Lukas","family":"Snoek","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"Matthew","family":"van der Meer","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9851-0409","authenticated-orcid":true,"given":"Ron","family":"van de Klundert","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"Eva","family":"Quispel","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"Natalie","family":"Cappaert","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]},{"given":"H. Steven","family":"Scholte","sequence":"additional","affiliation":[{"name":"University of Amsterdam, the Netherlands"}]}],"member":"281","published-online":{"date-parts":[[2022,11,1]]},"reference":[{"key":"2022110114534266300_bib100","doi-asserted-by":"publisher","first-page":"e1005508","DOI":"10.1371\/journal.pcbi.1005508","article-title":"Representational models: A common framework for understanding encoding, pattern-component, and representational-similarity analysis","volume":"13","author":"Diedrichsen","year":"2017","journal-title":"PLoS Computational Biology"},{"key":"2022110114534266300_bib1","doi-asserted-by":"publisher","first-page":"1488","DOI":"10.1162\/jocn.2007.19.9.1488","article-title":"Masking disrupts reentrant processing in human visual cortex","volume":"19","author":"Fahrenfort","year":"2007","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022110114534266300_bib2","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory? Nature reviews","volume":"11","author":"Friston","year":"2010","journal-title":"Neuroscience"},{"key":"2022110114534266300_bib3","doi-asserted-by":"publisher","first-page":"267","DOI":"10.3389\/fnins.2013.00267","article-title":"MEG and EEG data analysis with MNE-python","volume":"7","author":"Gramfort","year":"2013","journal-title":"Frontiers in Neuroscience"},{"key":"2022110114534266300_bib4","first-page":"20","volume-title":"Caltech-256 object category dataset","author":"Griffin","year":"2007"},{"key":"2022110114534266300_bib5","doi-asserted-by":"publisher","first-page":"e1006690","DOI":"10.1371\/journal.pcbi.1006690","article-title":"Scene complexity modulates degree of feedback activity during object detection in natural scenes","volume":"14","author":"Groen","year":"2018","journal-title":"PLoS Computational Biology"},{"key":"2022110114534266300_bib6","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1016\/j.neuroimage.2018.02.044","article-title":"Multivariate pattern analysis for MEG: A comparison of dissimilarity measures","volume":"173","author":"Guggenmos","year":"2018","journal-title":"Neuroimage"},{"key":"2022110114534266300_bib7","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1038\/s41586-020-2649-2","article-title":"Array programming with NumPy","volume":"585","author":"Harris","year":"2020","journal-title":"Nature"},{"key":"2022110114534266300_bib8","first-page":"770","article-title":"Deep residual learning for image recognition","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"He","year":"2016"},{"key":"2022110114534266300_bib9","article-title":"Feature-reweighted RSA: A method for improving the fit between computational models, brains, and behavior","author":"Kaniuth","year":"2021","journal-title":"bioRxiv"},{"key":"2022110114534266300_bib10","doi-asserted-by":"publisher","first-page":"974","DOI":"10.1038\/s41593-019-0392-5","article-title":"Evidence that recurrent circuits are critical to the ventral stream's execution of core object recognition behavior","volume":"22","author":"Kar","year":"2019","journal-title":"Nature Neuroscience"},{"key":"2022110114534266300_bib11","doi-asserted-by":"publisher","first-page":"e1003915","DOI":"10.1371\/journal.pcbi.1003915","article-title":"Deep supervised, but not unsupervised, models may explain IT cortical representation","volume":"10","author":"Khaligh-Razavi","year":"2014","journal-title":"PLoS Computational Biology"},{"key":"2022110114534266300_bib12","doi-asserted-by":"publisher","DOI":"10.1093\/acrefore\/9780190264086.013.46","article-title":"Deep neural networks in computational neuroscience","author":"Kietzmann","year":"2019","journal-title":"Oxford Research Encyclopedia of Neuroscience"},{"key":"2022110114534266300_bib13","doi-asserted-by":"publisher","first-page":"21854","DOI":"10.1073\/pnas.1905544116","article-title":"Recurrence is required to capture the representational dynamics of the human visual system","volume":"116","author":"Kietzmann","year":"2019","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022110114534266300_bib14","doi-asserted-by":"publisher","first-page":"209","DOI":"10.1016\/j.neuron.2017.05.033","article-title":"Distinct feedforward and feedback effects of microstimulation in visual cortex reveal neural mechanisms of texture segregation","volume":"95","author":"Klink","year":"2017","journal-title":"Neuron"},{"key":"2022110114534266300_bib15","doi-asserted-by":"publisher","first-page":"222","DOI":"10.1111\/nyas.14320","article-title":"Beyond the feedforward sweep: Feedback computations in the visual cortex","volume":"1464","author":"Kreiman","year":"2020","journal-title":"Annals of the New York Academy of Sciences"},{"key":"2022110114534266300_bib16","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1146\/annurev-vision-082114-035447","article-title":"Deep neural networks: A new framework for modeling biological vision and brain information processing","volume":"1","author":"Kriegeskorte","year":"2015","journal-title":"Annual Review of Vision Science"},{"key":"2022110114534266300_bib17","doi-asserted-by":"publisher","first-page":"4","DOI":"10.3389\/neuro.06.004.2008","article-title":"Representational Similarity Analysis\u2014Connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Frontiers in Systems Neuroscience"},{"key":"2022110114534266300_bib18","article-title":"Brain-like object recognition with high-performing shallow recurrent ANNs","author":"Kubilius","year":"2019","journal-title":"arXiv Preprint"},{"key":"2022110114534266300_bib19","article-title":"The open images dataset V4: Unified image classification, object detection, and visual relationship detection at scale","author":"Kuznetsova","year":"2018","journal-title":"arXiv [cs.CV]"},{"key":"2022110114534266300_bib20","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1016\/s0166-2236(00)01657-x","article-title":"The distinct modes of vision offered by feedforward and recurrent processing","volume":"23","author":"Lamme","year":"2000","journal-title":"Trends in Neurosciences"},{"key":"2022110114534266300_bib21","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1016\/s0959-4388(98)80042-1","article-title":"Feedforward, horizontal, and feedback processing in the visual cortex","volume":"8","author":"Lamme","year":"1998","journal-title":"Current Opinion in Neurobiology"},{"key":"2022110114534266300_bib22","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1162\/089892902320474490","article-title":"Masking interrupts figure-ground signals in V1","volume":"14","author":"Lamme","year":"2002","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022110114534266300_bib23","article-title":"Bridging the gaps between residual learning, recurrent neural networks and visual cortex","author":"Liao","year":"2016","journal-title":"arXiv:1604.03640"},{"key":"2022110114534266300_bib24","doi-asserted-by":"publisher","first-page":"740","DOI":"10.1007\/978-3-319-10602-1_48","article-title":"Microsoft COCO: Common objects in context","volume":"2014","author":"Lin","year":"2014","journal-title":"Computer Vision\u2013ECCV"},{"key":"2022110114534266300_bib25","doi-asserted-by":"publisher","first-page":"51","DOI":"10.25080\/Majora-92bf1922-00a","article-title":"Data structures for statistical computing in Python","volume-title":"Proceedings of the 9th python in science conference","author":"McKinney","year":"2010"},{"key":"2022110114534266300_bib26","doi-asserted-by":"publisher","first-page":"5725","DOI":"10.1038\/s41467-020-19632-w","article-title":"Individual differences among deep neural network models","volume":"11","author":"Mehrer","year":"2020","journal-title":"Nature Communications"},{"key":"2022110114534266300_bib27","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1037\/rev0000109","article-title":"Complementary surrounds explain diverse contextual phenomena across visual modalities","volume":"125","author":"M\u00e9ly","year":"2018","journal-title":"Psychological Review"},{"key":"2022110114534266300_bib28","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780195050387.001.0001","volume-title":"Electric fields of the brain: The neurophysics of EEG","author":"Nunez","year":"2009","edition":"2nd ed."},{"key":"2022110114534266300_bib29","article-title":"PyTorch: An imperative style, high-performance deep learning library","volume-title":"Advances in neural information processing systems","author":"Paszke","year":"2019"},{"key":"2022110114534266300_bib30","doi-asserted-by":"publisher","first-page":"184","DOI":"10.1016\/0013-4694(89)90180-6","article-title":"Spherical splines for scalp potential and current density mapping","volume":"72","author":"Perrin","year":"1989","journal-title":"Electroencephalography and Clinical Neurophysiology"},{"key":"2022110114534266300_bib31","doi-asserted-by":"publisher","first-page":"e1007001","DOI":"10.1371\/journal.pcbi.1007001","article-title":"Beyond core object recognition: Recurrent processes account for object recognition under occlusion","volume":"15","author":"Rajaei","year":"2019","journal-title":"PLoS Computational Biology"},{"key":"2022110114534266300_bib32","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1093\/bjps\/axx023","article-title":"Decoding the brain: Neural representation and the limits of multivariate pattern analysis in cognitive neuroscience","volume":"70","author":"Ritchie","year":"2019","journal-title":"British Journal for the Philosophy of Science"},{"key":"2022110114534266300_bib33","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1146\/annurev.neuro.29.051605.112939","article-title":"Cortical algorithms for perceptual grouping","volume":"29","author":"Roelfsema","year":"2006","journal-title":"Annual Review of Neuroscience"},{"key":"2022110114534266300_bib34","doi-asserted-by":"publisher","first-page":"525","DOI":"10.1162\/08989290260045756","article-title":"Figure\u2014Ground segregation in a recurrent network architecture","volume":"14","author":"Roelfsema","year":"2002","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022110114534266300_bib35","doi-asserted-by":"publisher","first-page":"5917","DOI":"10.1007\/s11229-021-03052-4","article-title":"Representational Similarity Analysis in neuroimaging: Proxy vehicles and provisional representations","volume":"199","author":"Roskies","year":"2021","journal-title":"Synthese"},{"key":"2022110114534266300_bib36","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1007\/s11263-007-0090-8","article-title":"LabelMe: A database and web-based tool for image annotation","volume":"77","author":"Russell","year":"2008","journal-title":"International Journal of Computer Vision"},{"key":"2022110114534266300_bib37","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.neuroimage.2017.12.077","article-title":"Fantastic DNimals and where to find them","volume":"180","author":"Scholte","year":"2018","journal-title":"Neuroimage"},{"key":"2022110114534266300_bib38","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1167\/9.4.29","article-title":"Brain responses strongly correlate with Weibull image statistics when processing natural images","volume":"9","author":"Scholte","year":"2009","journal-title":"Journal of Vision"},{"key":"2022110114534266300_bib39","doi-asserted-by":"publisher","first-page":"2097","DOI":"10.1162\/jocn.2008.20142","article-title":"Feedforward and recurrent processing in scene segmentation: Electroencephalography and functional magnetic resonance imaging","volume":"20","author":"Scholte","year":"2008","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022110114534266300_bib40","doi-asserted-by":"publisher","DOI":"10.1101\/407007","article-title":"Brain-score: Which artificial neural network for object recognition is most brain-like?","author":"Schrimpf","year":"2018","journal-title":"bioRxiv"},{"key":"2022110114534266300_bib41","doi-asserted-by":"crossref","first-page":"61","DOI":"10.25080\/Majora-92bf1922-011","article-title":"Statsmodels: Econometric and statistical modeling with python","volume-title":"Proceedings of the 9th python in science conference","author":"Seabold","year":"2010"},{"key":"2022110114534266300_bib42","doi-asserted-by":"publisher","first-page":"6281","DOI":"10.1523\/JNEUROSCI.2851-20.2021","article-title":"On the necessity of recurrent processing during object recognition: It depends on the need for scene segmentation","volume":"41","author":"Seijdel","year":"2021","journal-title":"Journal of Neuroscience"},{"key":"2022110114534266300_bib43","doi-asserted-by":"publisher","first-page":"6424","DOI":"10.1073\/pnas.0700622104","article-title":"A feedforward architecture accounts for rapid categorization","volume":"104","author":"Serre","year":"2007","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022110114534266300_bib44","doi-asserted-by":"publisher","first-page":"1551","DOI":"10.3389\/fpsyg.2017.01551","article-title":"Recurrent convolutional neural networks: A better model of biological object recognition","volume":"8","author":"Spoerer","year":"2017","journal-title":"Frontiers in Psychology"},{"key":"2022110114534266300_bib45","doi-asserted-by":"publisher","first-page":"2044","DOI":"10.1162\/jocn_a_01755","article-title":"Diverse deep neural networks all predict human inferior temporal cortex well, after training and fitting","volume":"33","author":"Storrs","year":"2021","journal-title":"Journal of Cognitive Neuroscience"},{"key":"2022110114534266300_bib46","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1007\/978-981-10-0213-7_3","article-title":"Recognition of occluded objects","volume-title":"Computational and cognitive neuroscience of vision","author":"Tang","year":"2017"},{"key":"2022110114534266300_bib47","doi-asserted-by":"publisher","first-page":"8835","DOI":"10.1073\/pnas.1719397115","article-title":"Recurrent computations for visual pattern completion","volume":"115","author":"Tang","year":"2018","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."},{"key":"2022110114534266300_bib48","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1038\/381520a0","article-title":"Speed of processing in the human visual system","volume":"381","author":"Thorpe","year":"1996","journal-title":"Nature"},{"key":"2022110114534266300_bib49","article-title":"Going in circles is the way forward: The role of recurrence in visual inference","author":"van Bergen","year":"2020","journal-title":"arXiv [q-bio.NC]"},{"key":"2022110114534266300_bib50","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1109\/10.841330","article-title":"Independent component approach to the analysis of EEG and MEG recordings","volume":"47","author":"Vigario","year":"2000","journal-title":"IEEE Transactions on Biomedical Engineering"},{"key":"2022110114534266300_bib51","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1038\/s41592-019-0686-2","article-title":"SciPy 1.0: Fundamental algorithms for scientific computing in python","volume":"17","author":"Virtanen","year":"2020","journal-title":"Nature Methods"},{"key":"2022110114534266300_bib52","doi-asserted-by":"publisher","first-page":"3021","DOI":"10.21105\/joss.03021","article-title":"Seaborn: Statistical data visualization","volume":"6","author":"Waskom","year":"2021","journal-title":"Journal of Open Source Software"},{"key":"2022110114534266300_bib53","doi-asserted-by":"publisher","first-page":"3485","DOI":"10.1109\/CVPR.2010.5539970","article-title":"SUN database: Large-scale scene recognition from abbey to zoo","volume-title":"2010 IEEE computer society conference on computer vision and pattern recognition","author":"Xiao","year":"2010"},{"key":"2022110114534266300_bib54","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1038\/nn.4244","article-title":"Using goal-driven deep learning models to understand sensory cortex","volume":"19","author":"Yamins","year":"2016","journal-title":"Nature Neuroscience"},{"key":"2022110114534266300_bib55","doi-asserted-by":"publisher","first-page":"8619","DOI":"10.1073\/pnas.1403112111","article-title":"Performance-optimized hierarchical models predict neural responses in higher visual cortex","volume":"111","author":"Yamins","year":"2014","journal-title":"Proceedings of the National Academy of Sciences, U.S.A."}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/34\/12\/2390\/2056531\/jocn_a_01914.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/34\/12\/2390\/2056531\/jocn_a_01914.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,1]],"date-time":"2022-11-01T11:00:40Z","timestamp":1667300440000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/34\/12\/2390\/113004\/A-Critical-Test-of-Deep-Convolutional-Neural"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022]]},"references-count":56,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,11,1]]},"published-print":{"date-parts":[[2022,11,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_01914","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.01.30.478404","asserted-by":"object"}]},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022]]},"published":{"date-parts":[[2022]]}}}