{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T00:12:52Z","timestamp":1767139972261,"version":"build-2238731810"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"16","license":[{"start":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T00:00:00Z","timestamp":1738454400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T00:00:00Z","timestamp":1738454400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005416","name":"Norges Forskningsr\u00e5d","doi-asserted-by":"publisher","award":["300504"],"award-info":[{"award-number":["300504"]}],"id":[{"id":"10.13039\/501100005416","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005366","name":"University of Oslo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005366","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Backpropagation-optimized artificial neural networks, while precise, lack robustness, leading to unforeseen behaviors that affect their safety. Biological neural systems do solve some of these issues already. Unlike artificial models, biological neurons adjust connectivity based on neighboring cell activity. Understanding the biological mechanisms of robustness can pave the way toward building trustworthy and safe systems. Robustness in neural representations is hypothesized to correlate with the smoothness of the encoding manifold. Recent work suggests power law covariance spectra, which were observed studying the primary visual cortex of mice, to be indicative of a balanced trade-off between accuracy and robustness in representations. Here, we show that unsupervised local learning models with winner takes all dynamics learn such power law representations, providing upcoming studies a mechanistic model with that characteristic. Our research aims to understand the interplay between geometry, spectral properties, robustness, and expressivity in neural representations. Hence, we study the link between representation smoothness and spectrum by using weight, Jacobian and spectral regularization while assessing performance and adversarial robustness. Our work serves as a foundation for future research into the mechanisms underlying power law spectra and optimally smooth encodings in both biological and artificial systems. The insights gained may elucidate the mechanisms that realize robust neural networks in mammalian brains and inform the development of more stable and reliable artificial systems.<\/jats:p>","DOI":"10.1007\/s00521-025-11019-6","type":"journal-article","created":{"date-parts":[[2025,2,2]],"date-time":"2025-02-02T09:26:11Z","timestamp":1738488371000},"page":"9331-9342","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring biologically inspired mechanisms of adversarial robustness"],"prefix":"10.1007","volume":"37","author":[{"given":"Konstantin","family":"Holzhausen","sequence":"first","affiliation":[]},{"given":"Mia","family":"Merlid","sequence":"additional","affiliation":[]},{"given":"H\u00e5kon Olav","family":"Torvik","sequence":"additional","affiliation":[]},{"given":"Anders","family":"Malthe-S\u00f8renssen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4262-5549","authenticated-orcid":false,"given":"Mikkel Elle","family":"Lepper\u00f8d","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,2,2]]},"reference":[{"key":"11019_CR1","doi-asserted-by":"publisher","unstructured":"Holzhausen K, Merlid M, Torvik HO, Malthe-S\u00f8renssen A, Lepper\u00f8d M E (2025). Exploring Biologically Inspired Mechanisms of Adversarial Robustness (Version 1.0.1) [Computer software]. https:\/\/doi.org\/10.5281\/zenodo.14753528","DOI":"10.5281\/zenodo.14753528"},{"issue":"48","key":"11019_CR2","doi-asserted-by":"publisher","first-page":"30088","DOI":"10.1073\/pnas.1907377117","volume":"117","author":"V Antun","year":"2020","unstructured":"Antun V, Renna F, Poon C, Adcock B, Hansen AC (2020) On instabilities of deep learning in image reconstruction and the potential costs of AI. Proc Natl Acad Sci 117(48):30088\u201330095. https:\/\/doi.org\/10.1073\/pnas.1907377117","journal-title":"Proc Natl Acad Sci"},{"issue":"6433","key":"11019_CR3","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1126\/science.aaw4399","volume":"363","author":"SG Finlayson","year":"2019","unstructured":"Finlayson SG, Bowers JD, Ito J, Zittrain JL, Beam AL, Kohane IS (2019) Adversarial attacks on medical machine learning. Science 363(6433):1287\u20131289. https:\/\/doi.org\/10.1126\/science.aaw4399","journal-title":"Science"},{"issue":"11","key":"11019_CR4","doi-asserted-by":"publisher","first-page":"2017","DOI":"10.1038\/s41593-023-01442-0","volume":"26","author":"J Feather","year":"2023","unstructured":"Feather J, Leclerc G, Madry A, McDermott JH (2023) Model metamers reveal divergent invariances between biological and artificial neural networks. Nat Neurosci 26(11):2017\u20132034. https:\/\/doi.org\/10.1038\/s41593-023-01442-0","journal-title":"Nat Neurosci"},{"key":"11019_CR5","doi-asserted-by":"publisher","unstructured":"Grinberg L, Hopfield J, Krotov D (2019) Local unsupervised learning for image analysis. In: Real Neurons & Hidden Units: Future Directions at the Intersection of Neuroscience and Artificial intelligence @ NeurIPS 2019. https:\/\/doi.org\/10.48550\/arxiv.1908.08993. https:\/\/openreview.net\/forum?id=rylq7mF8IS","DOI":"10.48550\/arxiv.1908.08993"},{"key":"11019_CR6","doi-asserted-by":"publisher","unstructured":"Guo C, Lee MJ, Leclerc G, Dapello J, Rao Y, Madry A, DiCarlo JJ (2022) Adversarially trained neural representations are already as robust as corresponding biological neural representations. Proceedings of the 39th International Conference on Machine Learning, in Proceedings of Machine Learning Research 162:8072-8081. https:\/\/doi.org\/10.48550\/ARXIV.2206.11228","DOI":"10.48550\/ARXIV.2206.11228"},{"key":"11019_CR7","doi-asserted-by":"publisher","unstructured":"Geirhos R, Narayanappa K, Mitzkus B, Thieringer T, Bethge M, Wichmann FA, Brendel W (2021) Partial success in closing the gap between human and machine vision. Advances in Neural Information Processing Systems. https:\/\/doi.org\/10.48550\/ARXIV.2106.07411","DOI":"10.48550\/ARXIV.2106.07411"},{"key":"11019_CR8","doi-asserted-by":"publisher","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. International Conference on Learning Representations (ICLR). https:\/\/doi.org\/10.48550\/arXiv.1412.6572","DOI":"10.48550\/arXiv.1412.6572"},{"key":"11019_CR9","doi-asserted-by":"publisher","unstructured":"Geirhos R, Temme CRM, Rauber J, Sch\u00fctt HH, Bethge M, Wichmann FA (2018) Generalisation in humans and deep neural networks. Advances in Neural Information Processing Systems, 31. Curran Associates, Inc. https:\/\/doi.org\/10.48550\/ARXIV.1808.08750","DOI":"10.48550\/ARXIV.1808.08750"},{"key":"11019_CR10","unstructured":"Harrington A, DuTell V, Tewari A, Hamilton M, Stent S, Rosenholtz R, Freeman WT (2022) Exploring the perceptual straightness of adversarially robust and biologically-inspired visual representations. In: SVRHM 2022 Workshop @ NeurIPS. https:\/\/openreview.net\/forum?id=A8ucsSFEAqS"},{"key":"11019_CR11","doi-asserted-by":"publisher","unstructured":"Hebb DO (1949) The organization of behavior: a neuropsychological theory. Brain Research Bulletin. John Whiley & Sons Inc, New York. https:\/\/doi.org\/10.1016\/s0361-9230(99)00182-3","DOI":"10.1016\/s0361-9230(99)00182-3"},{"key":"11019_CR12","unstructured":"Hoffman J, Roberts DA, Yaida S (2019) Robust learning with Jacobian regularization. arxiv:1908.02729"},{"key":"11019_CR13","doi-asserted-by":"crossref","unstructured":"Kurakin A, Goodfellow I, Bengio S (2017) Adversarial examples in the physical world. Proceedings of the International Conference on Learning Representations (ICLR), Workshop. https:\/\/openreview.net\/forum?id=HJGU3Rodl","DOI":"10.1201\/9781351251389-8"},{"key":"11019_CR14","unstructured":"Krotov D, Hopfield JJ (2016) Dense associative memory for pattern recognition. Advances in Neural Information Processing Systems, 29. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2016\/file\/eaae339c4d89fc102edd9dbdb6a28915-Paper.pdf"},{"issue":"16","key":"11019_CR15","doi-asserted-by":"publisher","first-page":"201820458","DOI":"10.1073\/pnas.1820458116","volume":"116","author":"D Krotov","year":"2019","unstructured":"Krotov D, Hopfield JJ (2019) Unsupervised learning by competing hidden units. Proc Natl Acad Sci 116(16):201820458. https:\/\/doi.org\/10.1073\/pnas.1820458116","journal-title":"Proc Natl Acad Sci"},{"key":"11019_CR16","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems,  Curran Associates, Inc. https:\/\/papers.nips.cc\/paper\/2012\/hash\/c399862d3b9d6b76c8436e924a68c45b-Abstract.html"},{"key":"11019_CR17","doi-asserted-by":"publisher","unstructured":"Moosavi-Dezfooli S-M, Fawzi A, Fawzi O, Frossard P (2017) Universal adversarial perturbations. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp. 86\u201394. https:\/\/doi.org\/10.1109\/CVPR.2017.17","DOI":"10.1109\/CVPR.2017.17"},{"key":"11019_CR18","doi-asserted-by":"publisher","unstructured":"Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2018) Towards deep learning models resistant to adversarial attacks. International Conference on Learning Representations (ICLR). https:\/\/doi.org\/10.48550\/arXiv.1706.06083","DOI":"10.48550\/arXiv.1706.06083"},{"key":"11019_CR19","unstructured":"Nassar J, Sokol PA, Chung S, Harris KD, Park IM (2020) On 1\/n neural representation and robustness. Advances in Neural Information Processing Systems, 33, 6211\u20136222. Curran Associates, Inc. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/44bf89b63173d40fb39f9842e308b3f9-Paper.pdf"},{"issue":"3","key":"11019_CR20","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1007\/bf00275687","volume":"15","author":"E Oja","year":"1982","unstructured":"Oja E (1982) Simplified neuron model as a principal component analyzer. J Math Biol 15(3):267\u2013273. https:\/\/doi.org\/10.1007\/bf00275687","journal-title":"J Math Biol"},{"key":"11019_CR21","doi-asserted-by":"publisher","unstructured":"Patel D, Kozma R (2020) Unsupervised Features Extracted using Winner-Take-All Mechanism Lead to Robust Image Classification. In: 2020 International Joint Conference on Neural Networks (IJCNN) 00, 1\u20137. https:\/\/doi.org\/10.1109\/ijcnn48605.2020.9207242","DOI":"10.1109\/ijcnn48605.2020.9207242"},{"issue":"7765","key":"11019_CR22","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1038\/s41586-019-1346-5","volume":"571","author":"C Stringer","year":"2019","unstructured":"Stringer C, Pachitariu M, Steinmetz N, Carandini M, Harris KD (2019) High-dimensional geometry of population responses in visual cortex. Nature 571(7765):361\u2013365. https:\/\/doi.org\/10.1038\/s41586-019-1346-5","journal-title":"Nature"},{"key":"11019_CR23","doi-asserted-by":"publisher","unstructured":"Varga D, Csisz\u00e1rik A, Zombori Z (2018) Gradient regularization improves accuracy of discriminative models. International Conference on Learning Representations (ICLR) Workshop. https:\/\/doi.org\/10.48550\/arxiv.1712.09936","DOI":"10.48550\/arxiv.1712.09936"},{"key":"11019_CR24","doi-asserted-by":"publisher","unstructured":"Wang B, Ponce CR (2021) The geometry of deep generative image models and its applications. International Conference on Learning Representations (ICLR). https:\/\/doi.org\/10.48550\/ARXIV.2101.06006","DOI":"10.48550\/ARXIV.2101.06006"},{"key":"11019_CR25","doi-asserted-by":"publisher","unstructured":"Zavatone-Veth JA, Yang S, Rubinfien JA, Pehlevan C (2023) Neural networks learn to magnify areas near decision boundaries. Advances in Neural Information Processing Systems (NeurIPS). https:\/\/doi.org\/10.48550\/arxiv.2301.11375","DOI":"10.48550\/arxiv.2301.11375"}],"updated-by":[{"DOI":"10.1007\/s00521-025-11175-9","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2025,3,24]],"date-time":"2025-03-24T00:00:00Z","timestamp":1742774400000}}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-025-11019-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-025-11019-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-025-11019-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,27]],"date-time":"2025-05-27T03:07:31Z","timestamp":1748315251000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-025-11019-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,2]]},"references-count":25,"journal-issue":{"issue":"16","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["11019"],"URL":"https:\/\/doi.org\/10.1007\/s00521-025-11019-6","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,2]]},"assertion":[{"value":"6 June 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 March 2025","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1007\/s00521-025-11175-9","URL":"https:\/\/doi.org\/10.1007\/s00521-025-11175-9","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no Conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}]}}