{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T13:52:58Z","timestamp":1772200378694,"version":"3.50.1"},"reference-count":53,"publisher":"MIT Press","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:p>We develop theoretical foundations of resonator networks, a new type of recurrent neural network introduced in Frady, Kent, Olshausen, and Sommer ( 2020 ), a companion article in this issue, to solve a high-dimensional vector factorization problem arising in Vector Symbolic Architectures. Given a composite vector formed by the Hadamard product between a discrete set of high-dimensional vectors, a resonator network can efficiently decompose the composite into these factors. We compare the performance of resonator networks against optimization-based methods, including Alternating Least Squares and several gradient-based algorithms, showing that resonator networks are superior in several important ways. This advantage is achieved by leveraging a combination of nonlinear dynamics and searching in superposition, by which estimates of the correct solution are formed from a weighted superposition of all possible solutions. While the alternative methods also search in superposition, the dynamics of resonator networks allow them to strike a more effective balance between exploring the solution space and exploiting local information to drive the network toward probable solutions. Resonator networks are not guaranteed to converge, but within a particular regime they almost always do. In exchange for relaxing the guarantee of global convergence, resonator networks are dramatically more effective at finding factorizations than all alternative approaches considered.<\/jats:p>","DOI":"10.1162\/neco_a_01329","type":"journal-article","created":{"date-parts":[[2020,10,20]],"date-time":"2020-10-20T21:25:44Z","timestamp":1603229144000},"page":"2332-2388","source":"Crossref","is-referenced-by-count":31,"title":["Resonator Networks, 2: Factorization Performance and Capacity Compared to Optimization-Based Methods"],"prefix":"10.1162","volume":"32","author":[{"given":"Spencer J.","family":"Kent","sequence":"first","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience and Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, CA 94720, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"E. Paxon","family":"Frady","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, U.S.A., and Intel Laboratories, Neuromorphic Computing Lab, San Francisco, CA 94111, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Friedrich T.","family":"Sommer","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience and Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, CA 94720, U.S.A., and Intel Laboratories, Neuromorphic Computing Lab, San Francisco, CA 94111, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bruno A.","family":"Olshausen","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, Helen Wills Neuroscience Institute, and School of Optometry, University of California, Berkeley, Berkeley, CA 94720, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.55.1530"},{"key":"B4","first-page":"2773","volume":"15","author":"Anandkumar A.","year":"2014","journal-title":"Journal of Machine Learning Research"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1049\/el:20010123"},{"key":"B6","volume-title":"Map-seeking circuits in visual cognition: A computational mechanism for biological and machine vision","author":"Arathorn D. W.","year":"2002"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.4086\/toc.2012.v008a006"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1137\/080716542"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199535255.001.0001"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1007\/s00041-008-9041-1"},{"key":"B11","unstructured":"Cannon, L. E. (1969).A cellular computer to implement the Kalman filter algorithm. PhD diss. Montana State University\u2013Bozeman."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310791"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1007\/BF02293596"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015628"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1983.6313075"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1007\/s10107-015-0946-6"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1002\/cpa.20042"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1137\/S0895479896305696"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1137\/S0895479898346995"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390191"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1902653116"},{"key":"B23","volume-title":"Advances in analogy research: Integration of theory and data from the cognitive, computational, and neural sciences","author":"Gayler R. W.","year":"1998"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-007-0028-3"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1007\/s10851-007-0024-7"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1561\/2400000013"},{"key":"B29","volume-title":"The organization of behavior: A neuropsychological theory.","author":"Hebb D. O.","year":"1949"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1007\/BF01580223"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1063\/1.36259"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1145\/2512329"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1002\/sapm192761164"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.79.8.2554"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.81.10.3088"},{"issue":"3","key":"B36","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/BF00339943","volume":"52","author":"Hopfield J. J.","year":"1985","journal-title":"Biological Cybernetics"},{"key":"B37","first-page":"1457","volume":"5","author":"Hoyer P. O.","year":"2004","journal-title":"Journal of Machine Learning Research"},{"key":"B38","first-page":"194","volume-title":"Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics","volume":"1","author":"Huang J.","year":"1999"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-61510-5_146"},{"issue":"3","key":"B40","first-page":"1494","volume":"33","author":"Kobayashi M.","year":"2002","journal-title":"Systems and Computers in Japan"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1137\/07070111X"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/21.87054"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1016\/0024-3795(77)90069-6"},{"key":"B44","first-page":"556","volume-title":"Advances in neural information processing systems","volume":"13","author":"Lee D. D.","year":"2001"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2010.01-09-953"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.34.4217"},{"key":"B47","volume-title":"Holographic reduced representation: Distributed representation of cognitive structure","author":"Plate T. A.","year":"2003"},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.1109\/TCSI.2017.2705051"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1002\/1099-128X(200005\/06)14:3<229::AID-CEM587>3.0.CO;2-N"},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2017.2690524"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.52.5261"},{"key":"B52","volume-title":"Introduction to stochastic search and optimization: Estimation, simulation, and control","author":"Spall J. C.","year":"2005"},{"key":"B53","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300015349"},{"key":"B54","first-page":"122","volume-title":"Problems in measuring change","author":"Tucker L. R.","year":"1963"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289464"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1126\/science.274.5293.1724"},{"key":"B57","unstructured":"Vasilescu, M. A. O. & Terzopoulos, D. (2002). Multilinear analysis of image ensembles: Tensorfaces. InProceedings of the European Conference on Computer Vision(pp. 447\u2013460). Berlin: Springer."},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.1016\/0893-6080(95)00114-X"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01329","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,11,24]],"date-time":"2022-11-24T01:51:40Z","timestamp":1669254700000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/32\/12\/2332-2388\/95653"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12]]},"references-count":53,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["10.1162\/neco_a_01329"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01329","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12]]}}}