{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T15:55:25Z","timestamp":1787759725991,"version":"build-2784847793"},"reference-count":44,"publisher":"MIT Press","issue":"2","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,1,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Binding operation is fundamental to many cognitive processes, such as cognitive map formation, relational reasoning, and language comprehension. In these processes, two different modalities, such as location and objects, events and their contextual cues, and words and their roles, need to be bound together, but little is known about the underlying neural mechanisms. Previous work has introduced a binding model based on quadratic functions of bound pairs, followed by vector summation of multiple pairs. Based on this framework, we address the following questions: Which classes of quadratic matrices are optimal for decoding relational structures? And what is the resultant accuracy? We introduce a new class of binding matrices based on a matrix representation of octonion algebra, an eight-dimensional extension of complex numbers. We show that these matrices enable a more accurate unbinding than previously known methods when a small number of pairs are present. Moreover, numerical optimization of a binding operator converges to this octonion binding. We also show that when there are a large number of bound pairs, however, a random quadratic binding performs, as well as the octonion and previously proposed binding methods. This study thus provides new insight into potential neural mechanisms of binding operations in the brain.<\/jats:p>","DOI":"10.1162\/neco_a_01558","type":"journal-article","created":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T00:56:33Z","timestamp":1671670593000},"page":"105-155","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":5,"title":["Optimal Quadratic Binding for Relational Reasoning in Vector Symbolic Neural Architectures"],"prefix":"10.1162","volume":"35","author":[{"given":"Naoki","family":"Hiratani","sequence":"first","affiliation":[{"name":"Center for Brain Science, Harvard University, Cambridge MA 02138, U.S.A. n.hiratani@gmail.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haim","family":"Sompolinsky","sequence":"additional","affiliation":[{"name":"Center for Brain Science, Harvard University, Cambridge MA 02138, U.S.A."},{"name":"Edmond and Lily Safra Center for Brain Sciences, Hebrew University, Jerusalem 91904, Israel haim@fiz.huji.ac.il"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,1,20]]},"reference":[{"issue":"5","key":"2023012618411757100_B1","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1016\/j.jmp.2009.02.005","article-title":"Geometric analogue of holographic reduced representation","volume":"53","author":"Aerts","year":"2009","journal-title":"Journal of Mathematical Psychology"},{"key":"2023012618411757100_B2","first-page":"2425","article-title":"VQA: Visual question answering","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Antol","year":"2015"},{"issue":"2","key":"2023012618411757100_B3","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1090\/S0273-0979-01-00934-X","article-title":"The octonions","volume":"39","author":"Baez","year":"2002","journal-title":"Bulletin of the American Mathematical Society"},{"issue":"1","key":"2023012618411757100_B4","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1017\/S0140525X01003922","article-title":"The magical number 4 in short-term memory: A reconsideration of mental storage capacity","volume":"24","author":"Cowan","year":"2001","journal-title":"Behavioral and Brain Sciences"},{"issue":"6111","key":"2023012618411757100_B5","doi-asserted-by":"publisher","first-page":"1202","DOI":"10.1126\/science.1225266","article-title":"A large-scale model of the functioning brain","volume":"338","author":"Eliasmith","year":"2012","journal-title":"Science"},{"issue":"1","key":"2023012618411757100_B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11571-012-9219-8","article-title":"The neural binding problem(s)","volume":"7","author":"Feldman","year":"2013","journal-title":"Cognitive Neurodynamics"},{"issue":"12","key":"2023012618411757100_B7","doi-asserted-by":"publisher","first-page":"2311","DOI":"10.1162\/neco_a_01331","article-title":"Resonator networks, 1: An efficient solution for factoring high-dimensional, distributed representations of data structures","volume":"32","author":"Frady","year":"2020","journal-title":"Neural Computation"},{"issue":"6","key":"2023012618411757100_B8","doi-asserted-by":"publisher","first-page":"1449","DOI":"10.1162\/neco_a_01084","article-title":"A theory of sequence indexing and working memory in recurrent neural networks","volume":"30","author":"Frady","year":"2018","journal-title":"Neural Computation"},{"key":"2023012618411757100_B9","doi-asserted-by":"crossref","DOI":"10.1109\/TNNLS.2021.3105949","article-title":"Variable binding for sparse distributed representations theory and applications","volume-title":"IEEE Transactions on Neural Networks and Learning Systems","author":"Frady","year":"2021"},{"issue":"8","key":"2023012618411757100_B10","doi-asserted-by":"publisher","first-page":"2038","DOI":"10.1162\/NECO_a_00467","article-title":"Representing objects, relations, and sequences","volume":"25","author":"Gallant","year":"2013","journal-title":"Neural Computation"},{"key":"2023012618411757100_B11","article-title":"Learning with holographic reduced representations","volume-title":"Advances in neural information processing systems","author":"Ganesan","year":"2021"},{"key":"2023012618411757100_B12","first-page":"133","article-title":"Vector symbolic architectures answer Jackendoff's challenges for cognitive neuroscience","author":"Gayler","year":"2003","journal-title":"Proceedings of the ICCS\/ASCS International Conference on Cognitive Science"},{"key":"2023012618411757100_B13","article-title":"Connections, binding, unification, and analogical promiscuity","volume-title":"Advances in analogy research: Integration of theory and data from the cognitive, computational, and neural sciences","author":"Gayler","year":"1998"},{"issue":"5","key":"2023012618411757100_B14","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1162\/neco_a_01179","article-title":"Vector-derived transformation binding: An improved binding operation for deep symbol-like processing in neural networks","volume":"31","author":"Gosmann","year":"2019","journal-title":"Neural Computation"},{"key":"2023012618411757100_B15","author":"Greff","year":"2020","journal-title":"On the binding problem in artificial neural networks"},{"issue":"7787","key":"2023012618411757100_B16","doi-asserted-by":"publisher","first-page":"446","DOI":"10.1038\/s41586-019-1816-9","article-title":"Frontal cortex neuron types categorically encode single decision variables","volume":"576","author":"Hirokawa","year":"2019","journal-title":"Nature"},{"key":"2023012618411757100_B17","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780198270126.001.0001","volume-title":"Foundations of language: Brain, meaning, grammar, evolution","author":"Jackendoff","year":"2002"},{"key":"2023012618411757100_B18","first-page":"2901","article-title":"Clevr: A diagnostic dataset for compositional language and elementary visual reasoning","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Johnson","year":"2017"},{"issue":"2","key":"2023012618411757100_B19","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/s12559-009-9009-8","article-title":"Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors","volume":"1","author":"Kanerva","year":"2009","journal-title":"Cognitive Computation"},{"key":"2023012618411757100_B20","article-title":"Fully distributed representation","author":"Kanerva","year":"1997","journal-title":"Proceedings of the 1997 Real World Computing Symposium"},{"issue":"12","key":"2023012618411757100_B21","doi-asserted-by":"publisher","first-page":"2332","DOI":"10.1162\/neco_a_01329","article-title":"Resonator networks, 2: Factorization performance and capacity compared to optimization-based methods","volume":"32","author":"Kent","year":"2020","journal-title":"Neural Computation"},{"key":"2023012618411757100_B22","author":"Kleyko","year":"2021","journal-title":"Vector symbolic architectures as a computing framework for nanoscale hardware"},{"key":"2023012618411757100_B23","article-title":"A survey on hyperdimensional computing aka vector symbolic architectures, part I: Models and data transformations","author":"Kleyko","year":"2022","journal-title":"ACM Computing Surveys"},{"key":"2023012618411757100_B24","first-page":"403","article-title":"Holistic representations for memorization and inference","author":"Ma","year":"2018","journal-title":"Conference on Uncertainty in Artificial Intelligence"},{"issue":"2","key":"2023012618411757100_B25","doi-asserted-by":"publisher","DOI":"10.1037\/h0043158","article-title":"The magical number seven, plus or minus two: Some limits on our capacity for processing information","volume":"63","author":"Miller","year":"1956","journal-title":"Psychological Review"},{"issue":"6","key":"2023012618411757100_B26","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.89.6.609","article-title":"A theory for the storage and retrieval of item and associative information","volume":"89","author":"Murdock","year":"1982","journal-title":"Psychological Review"},{"issue":"1","key":"2023012618411757100_B27","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1109\/JPROC.2015.2483592","article-title":"A review of relational machine learning for knowledge graphs","volume":"104","author":"Nickel","year":"2015","journal-title":"Proceedings of the IEEE"},{"key":"2023012618411757100_B28","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10314","article-title":"Holographic embeddings of knowledge graphs","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Nickel","year":"2016"},{"issue":"7865","key":"2023012618411757100_B29","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1038\/s41586-021-03652-7","article-title":"Geometry of abstract learned knowledge in the hippocampus","volume":"595","author":"Nieh","year":"2021","journal-title":"Nature"},{"key":"2023012618411757100_B30","author":"Plate","year":"1994","journal-title":"Distributed representations and nested compositional structure"},{"issue":"3","key":"2023012618411757100_B31","doi-asserted-by":"publisher","first-page":"623","DOI":"10.1109\/72.377968","article-title":"Holographic reduced representations","volume":"6","author":"Plate","year":"1995","journal-title":"IEEE Transactions on Neural"},{"key":"2023012618411757100_B32","first-page":"15","article-title":"A common framework for distributed representation schemes for compositional structure","volume-title":"Connectionist systems for knowledge representation and deduction","author":"Plate","year":"1997"},{"issue":"2","key":"2023012618411757100_B33","doi-asserted-by":"publisher","first-page":"411","DOI":"10.1162\/089976601300014592","article-title":"Binding and normalization of binary sparse distributed representations by context-dependent thinning","volume":"13","author":"Rachkovskij","year":"2001","journal-title":"Neural Computation"},{"issue":"7451","key":"2023012618411757100_B34","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1038\/nature12160","article-title":"The importance of mixed selectivity in complex cognitive tasks","volume":"497","author":"Rigotti","year":"2013","journal-title":"Nature"},{"key":"2023012618411757100_B35","author":"Santoro","year":"2017","journal-title":"A simple neural network module for relational reasoning"},{"key":"2023012618411757100_B36","doi-asserted-by":"publisher","first-page":"4523","DOI":"10.1007\/s10462-021-10110-3","article-title":"A comparison of vector symbolic architectures","volume":"55","author":"Schlegel","year":"2021","journal-title":"Artificial Intelligence Review"},{"key":"2023012618411757100_B37","volume-title":"Compositions of quadratic forms","author":"Shapiro","year":"2011"},{"issue":"1\u20132","key":"2023012618411757100_B38","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/0004-3702(90)90007-M","article-title":"Tensor product variable binding and the representation of symbolic structures in connectionist systems","volume":"46","author":"Smolensky","year":"1990","journal-title":"Artificial Intelligence"},{"issue":"6","key":"2023012618411757100_B39","doi-asserted-by":"publisher","first-page":"1102","DOI":"10.1111\/cogs.12047","article-title":"Optimization and quantization in gradient symbol systems A framework for integrating the continuous and the discrete in cognition","volume":"38","author":"Smolensky","year":"2014","journal-title":"Cognitive Science"},{"key":"2023012618411757100_B40","first-page":"926","article-title":"Reasoning with neural tensor networks for knowledge base completion","volume-title":"Advances in neural information processing systems","author":"Socher","year":"2013"},{"key":"2023012618411757100_B41","author":"Steinberg","year":"2022","journal-title":"Associative memory of structured knowledge"},{"key":"2023012618411757100_B42","first-page":"4223","article-title":"Tips and tricks for visual question answering: Learnings from the 2017 challenge","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Teney","year":"2018"},{"key":"2023012618411757100_B43","author":"Tian","year":"2000","journal-title":"Matrix representations of octonions and their applications"},{"issue":"5","key":"2023012618411757100_B44","doi-asserted-by":"crossref","first-page":"1249","DOI":"10.1016\/j.cell.2020.10.024","article-title":"The Tolman-Eichenbaum machine: Unifying space and relational memory through generalization in the hippocampal formation","volume":"183","author":"Whittington","year":"2020","journal-title":"Cell"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/2\/105\/2068494\/neco_a_01558.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/2\/105\/2068494\/neco_a_01558.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,1,26]],"date-time":"2023-01-26T18:42:26Z","timestamp":1674758546000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/2\/105\/114138\/Optimal-Quadratic-Binding-for-Relational-Reasoning"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,20]]},"references-count":44,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1,20]]},"published-print":{"date-parts":[[2023,1,20]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01558","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,2]]},"published":{"date-parts":[[2023,1,20]]}}}