{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T04:55:13Z","timestamp":1771476913465,"version":"3.50.1"},"reference-count":70,"publisher":"MIT Press","issue":"1","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,12,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We introduce residue hyperdimensional computing, a computing framework that unifies residue number systems with an algebra defined over random, high-dimensional vectors. We show how residue numbers can be represented as high-dimensional vectors in a manner that allows algebraic operations to be performed with component-wise, parallelizable operations on the vector elements. The resulting framework, when combined with an efficient method for factorizing high-dimensional vectors, can represent and operate on numerical values over a large dynamic range using resources that scale only logarithmically with the range, a vast improvement over previous methods. It also exhibits impressive robustness to noise. We demonstrate the potential for this framework to solve computationally difficult problems in visual perception and combinatorial optimization, showing improvement over baseline methods. More broadly, the framework provides a possible account for the computational operations of grid cells in the brain, and it suggests new machine learning architectures for representing and manipulating numerical data.<\/jats:p>","DOI":"10.1162\/neco_a_01723","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T17:58:33Z","timestamp":1731952713000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":7,"title":["Computing With Residue Numbers in High-Dimensional Representation"],"prefix":"10.1162","volume":"37","author":[{"given":"Christopher J.","family":"Kymn","sequence":"first","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, University of California, Berkeley, CA 94720, U.S.A. cjkymn@berkeley.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Denis","family":"Kleyko","sequence":"additional","affiliation":[{"name":"Centre for Applied Autonomous Sensor Systems, Orebro University, Orebro SE-701 82, Sweden"},{"name":"Intelligent Systems Lab, Research Institutes of Sweden, 164 40 Kista, Sweden denis.kleyko@oru.se"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"E. Paxon","family":"Frady","sequence":"additional","affiliation":[{"name":"Neuromorphic Computing Lab, Intel, Santa Clara, CA 95054, U.S.A. e.paxon.frady@intel.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Connor","family":"Bybee","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, University of California, Berkeley, CA 94720, U.S.A. bybee@berkeley.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pentti","family":"Kanerva","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, University of California, Berkeley, CA 94720, U.S.A. pkanerva@csli.stanford.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Friedrich T.","family":"Sommer","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, University of California, Berkeley, CA 94720, U.S.A."},{"name":"Neuromorphic Computing Lab, Intel, Santa Clara, CA 95054, U.S.A. fsommer@berkeley.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bruno A.","family":"Olshausen","sequence":"additional","affiliation":[{"name":"Redwood Center for Theoretical Neuroscience, University of California, Berkeley, CA 94720, U.S.A. baolshausen@berkeley.edu"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,12,12]]},"reference":[{"key":"2024121121100472500_bib1","article-title":"A theory of joint attractor dynamics in the hippocampus and the entorhinal cortex accounts for artificial remapping and grid cell field-to-field variability","volume":"9","author":"Agmon","year":"2020","journal-title":"Proceedings of the eLife"},{"key":"2024121121100472500_bib2","author":"Babai","year":"1979","journal-title":"Monte-Carlo algorithms in graph isomorphism testing"},{"key":"2024121121100472500_bib3","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.78606","article-title":"Population codes enable learning from few examples by shaping inductive bias","volume":"11","author":"Bordelon","year":"2022","journal-title":"eLife"},{"key":"2024121121100472500_bib4","first-page":"1","article-title":"Thermometer encoding: One hot way to resist adversarial examples","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Buckman","year":"2018"},{"key":"2024121121100472500_bib5","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1145\/3517343.3517369","article-title":"Optimal oscillator memory networks","volume-title":"Proceedings of the Neuro-Inspired Computational Elements Conference","author":"Bybee","year":"2022"},{"key":"2024121121100472500_bib6","first-page":"1","article-title":"Generalized learning vector quantization for classification in randomized neural networks and hyperdimensional computing","volume-title":"Proceedings of the International Joint Conference on Neural Networks","author":"Diao","year":"2021"},{"issue":"5","key":"2024121121100472500_bib7","doi-asserted-by":"publisher","first-page":"608","DOI":"10.1109\/12.223680","article-title":"A new technique for fast number comparison in the residue number system","volume":"42","author":"Dimauro","year":"1993","journal-title":"IEEE Transactions on Computers"},{"key":"2024121121100472500_bib8","first-page":"2367","article-title":"Accurate representation for spatial cognition using grid cells","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society","author":"Dumont","year":"2020"},{"issue":"27","key":"2024121121100472500_bib9","doi-asserted-by":"publisher","first-page":"6858","DOI":"10.1523\/JNEUROSCI.5684-07.2008","article-title":"What grid cells convey about rat location","volume":"28","author":"Fiete","year":"2008","journal-title":"Journal of Neuroscience"},{"issue":"12","key":"2024121121100472500_bib10","doi-asserted-by":"publisher","first-page":"2311","DOI":"10.1162\/neco_a_01331","article-title":"Resonator networks, 1: An efficient solution for factoring high-dimensional, distributed representations of data structures","volume":"32","author":"Frady","year":"2020","journal-title":"Neural Computation"},{"key":"2024121121100472500_bib11","author":"Frady","year":"2021","journal-title":"Computing on functions using randomized vector representations."},{"key":"2024121121100472500_bib12","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1145\/3517343.3522597","article-title":"Computing on functions using randomized vector representations (in brief)","volume-title":"Proceedings of the Neuro-Inspired Computational Elements Conference","author":"Frady","year":"2022"},{"key":"2024121121100472500_bib13","doi-asserted-by":"publisher","first-page":"1449","DOI":"10.1162\/neco_a_01084","article-title":"A theory of sequence indexing and working memory in recurrent neural networks","volume-title":"Neural Computation","author":"Frady","year":"2018"},{"issue":"36","key":"2024121121100472500_bib14","doi-asserted-by":"publisher","first-page":"18050","DOI":"10.1073\/pnas.1902653116","article-title":"Robust computation with rhythmic spike patterns","volume":"116","author":"Frady","year":"2019","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"7895","key":"2024121121100472500_bib15","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1038\/s41586-021-04268-7","article-title":"Toroidal topology of population activity in grid cells","volume":"602","author":"Gardner","year":"2022","journal-title":"Nature"},{"key":"2024121121100472500_bib16","first-page":"46","article-title":"The residue number system","volume-title":"Proceedings of the Western Joint Computer Conference","author":"Garner","year":"1959"},{"key":"2024121121100472500_bib17","first-page":"133","article-title":"Vector symbolic architectures answer Jackendoff\u2019s challenges for cognitive neuroscience","volume-title":"Proceedings of the Joint International Conference on Cognitive Science","author":"Gayler","year":"2003"},{"key":"2024121121100472500_bib18","first-page":"225","article-title":"Chinese remaindering with errors","volume-title":"Proceedings of the 31st Annual ACM Symposium on Theory of Computing","author":"Goldreich","year":"1999"},{"issue":"9","key":"2024121121100472500_bib19","doi-asserted-by":"publisher","first-page":"643","DOI":"10.1016\/0893-6080(95)00136-0","article-title":"An assembly neural network for texture segmentation","volume":"4","author":"Goltsev","year":"1996","journal-title":"Neural Networks"},{"issue":"7052","key":"2024121121100472500_bib20","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1038\/nature03721","article-title":"Microstructure of a spatial map in the entorhinal cortex","volume":"436","author":"Hafting","year":"2005","journal-title":"Nature"},{"key":"2024121121100472500_bib21","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/s12559-009-9009-8","article-title":"Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors","volume":"1","author":"Kanerva","year":"2009","journal-title":"Cognitive Computation"},{"key":"2024121121100472500_bib22","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4684-2001-2_9","article-title":"Chapter reducibility among combinatorial problems","volume-title":"Complexity of computer computations","author":"Karp","year":"1972"},{"issue":"12","key":"2024121121100472500_bib23","doi-asserted-by":"publisher","first-page":"2332","DOI":"10.1162\/neco_a_01329","article-title":"Resonator networks, 2: Factorization performance and capacity compared to optimization-based methods","volume":"32","author":"Kent","year":"2020","journal-title":"Neural Computation"},{"key":"2024121121100472500_bib24","first-page":"1","article-title":"HDM: Hyper-dimensional modulation for robust low-power communications","volume-title":"Proceedings of the 2018 IEEE International Conference on Communications","author":"Kim","year":"2018"},{"key":"2024121121100472500_bib25","volume-title":"Algorithm design","author":"Kleinberg","year":"2006"},{"issue":"7","key":"2024121121100472500_bib26","doi-asserted-by":"publisher","first-page":"1159","DOI":"10.1162\/neco_a_01590","article-title":"Efficient decoding of compositional structure in holistic representations","volume":"35","author":"Kleyko","year":"2023","journal-title":"Neural Computation"},{"key":"2024121121100472500_bib27","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1145\/3517343.3517368","article-title":"Integer factorization with compositional distributed representations","volume-title":"Proceedings of the Neuro-Inspired Computational Elements Conference","author":"Kleyko","year":"2022"},{"issue":"10","key":"2024121121100472500_bib28","doi-asserted-by":"publisher","first-page":"1538","DOI":"10.1109\/JPROC.2022.3209104","article-title":"Vector symbolic architectures as a computing framework for emerging hardware","volume":"110","author":"Kleyko","year":"2022","journal-title":"Proceedings of the IEEE"},{"issue":"8","key":"2024121121100472500_bib29","doi-asserted-by":"publisher","first-page":"3777","DOI":"10.1109\/TNNLS.2020.3015971","article-title":"Density encoding enables resource-efficient randomly connected neural networks","volume":"32","author":"Kleyko","year":"2020","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2024121121100472500_bib30","doi-asserted-by":"publisher","first-page":"30766","DOI":"10.1109\/ACCESS.2018.2840128","article-title":"Hyperdimensional computing in industrial systems: The use-case of distributed fault isolation in a power plant","volume":"6","author":"Kleyko","year":"2018","journal-title":"IEEE Access"},{"issue":"6","key":"2024121121100472500_bib31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3538531","article-title":"A survey on hyperdimensional computing aka vector symbolic architectures, Part I: Models and data transformations","volume":"55","author":"Kleyko","year":"2022","journal-title":"ACM Computing Surveys"},{"issue":"9","key":"2024121121100472500_bib32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3558000","article-title":"A survey on hyperdimensional computing aka vector symbolic architectures, Part II: Applications, cognitive models, and challenges","volume":"55","author":"Kleyko","year":"2023","journal-title":"ACM Computing Surveys"},{"issue":"12","key":"2024121121100472500_bib33","doi-asserted-by":"publisher","first-page":"5880","DOI":"10.1109\/TNNLS.2018.2814400","article-title":"Classification and recall with binary hyperdimensional computing: Trade-offs in choice of density and mapping characteristics","volume":"29","author":"Kleyko","year":"2018","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2024121121100472500_bib34","author":"Komer","year":"2020","journal-title":"Biologically inspired spatial representation"},{"key":"2024121121100472500_bib35","doi-asserted-by":"publisher","DOI":"10.3389\/neuro.01.016.2008","article-title":"Representational similarity analysis: Connecting the branches of systems neuroscience","volume":"2","author":"Kriegeskorte","year":"2008","journal-title":"Frontiers in Systems Neuroscience"},{"issue":"6096","key":"2024121121100472500_bib36","doi-asserted-by":"publisher","first-page":"853","DOI":"10.1126\/science.1222403","article-title":"Neural representations of location composed of spatially periodic bands","volume":"337","author":"Krupic","year":"2012","journal-title":"Science"},{"key":"2024121121100472500_bib37","first-page":"1","article-title":"Compositional factorization of visual scenes with convolutional sparse coding and resonator networks","volume-title":"Proceedings of the 2024 Neuro-Inspired Computational Elements Conference","author":"Kymn","year":"2024"},{"key":"2024121121100472500_bib38","first-page":"92","article-title":"\u00dcber die maximalordnung der permutationen gegebenen grades","volume":"3","author":"Landau","year":"1903","journal-title":"Archiv der Math. und Phys"},{"issue":"5","key":"2024121121100472500_bib39","doi-asserted-by":"publisher","first-page":"479","DOI":"10.1038\/s41565-023-01357-8","article-title":"In-memory factorization of holographic perceptual representations","volume":"18","author":"Langenegger","year":"2023","journal-title":"Nature Nanotechnology"},{"issue":"6","key":"2024121121100472500_bib40","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1007\/s10958-009-9366-6","article-title":"Equiangular tight frames","volume":"157","author":"Malozemov","year":"2009","journal-title":"Journal of Mathematical Sciences"},{"issue":"1","key":"2024121121100472500_bib41","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.109.018103","article-title":"Resolution of nested neuronal representations can be exponential in the number of neurons","volume":"109","author":"Mathis","year":"2012","journal-title":"Physical Review Letters"},{"key":"2024121121100472500_bib42","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.05979","article-title":"Probable nature of higher-dimensional symmetries underlying mammalian grid-cell activity patterns","volume":"4","author":"Mathis","year":"2015","journal-title":"eLife"},{"key":"2024121121100472500_bib43","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-41385-3","volume-title":"Residue number systems","author":"Mohan","year":"2016"},{"key":"2024121121100472500_bib44","author":"Nanda","year":"2005","journal-title":"Subset sum problem"},{"issue":"4","key":"2024121121100472500_bib45","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.38.2196","article-title":"Discrete-state phasor neural networks","volume":"38","author":"Noest","year":"1988","journal-title":"Physical Review A"},{"key":"2024121121100472500_bib46","doi-asserted-by":"crossref","first-page":"295","DOI":"10.7551\/mitpress\/9504.003.0037","article-title":"Perception as an inference problem","volume-title":"The cognitive neurosciences","author":"Olshausen","year":"2014","edition":"5th ed."},{"issue":"6583","key":"2024121121100472500_bib47","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"key":"2024121121100472500_bib48","doi-asserted-by":"publisher","DOI":"10.1142\/p523","volume-title":"Residue number systems: Theory and implementation","author":"Omondi","year":"2007"},{"key":"2024121121100472500_bib49","first-page":"515","article-title":"The closeness code: An integer to binary vector transformation suitable for neural network algorithms","volume-title":"Proceedings of the IEEE First Annual International Conference on Neural Networks","author":"Penz","year":"1987"},{"key":"2024121121100472500_bib50","first-page":"34","article-title":"Holographic recurrent networks","volume-title":"Advances in neural information processing systems, 5","author":"Plate","year":"1992"},{"key":"2024121121100472500_bib51","author":"Plate","year":"1994","journal-title":"Distributed representations and nested compositional structure"},{"issue":"3","key":"2024121121100472500_bib52","doi-asserted-by":"publisher","first-page":"623","DOI":"10.1109\/72.377968","article-title":"Holographic reduced representations","volume":"6","author":"Plate","year":"1995","journal-title":"IEEE Transactions on Neural Networks"},{"key":"2024121121100472500_bib53","volume-title":"Holographic reduced representation: Distributed representation for cognitive structures","author":"Plate","year":"2003"},{"issue":"2","key":"2024121121100472500_bib54","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1038\/35039062","article-title":"Information processing with population codes","volume":"1","author":"Pouget","year":"2000","journal-title":"Nature Reviews Neuroscience"},{"issue":"3","key":"2024121121100472500_bib55","first-page":"270","article-title":"Linear classifiers based on binary distributed representations","volume":"14","author":"Rachkovskij","year":"2007","journal-title":"Information Theories and Applications"},{"issue":"6","key":"2024121121100472500_bib56","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1615\/J Automat Inf Scien.v37.i6.20","article-title":"Sparse binary distributed encoding of scalars","volume":"37","author":"Rachkovskij","year":"2005","journal-title":"Journal of Automation and Information Sciences"},{"issue":"1","key":"2024121121100472500_bib57","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1109\/JPROC.2018.2871163","article-title":"Efficient biosignal processing using hyperdimensional computing: Network templates for combined learning and classification of ExG signals","volume":"107","author":"Rahimi","year":"2019","journal-title":"Proceedings of the IEEE"},{"key":"2024121121100472500_bib58","first-page":"1","article-title":"Random features for large-scale kernel machines","volume-title":"Advances in neural information processing systems, 20","author":"Rahimi","year":"2007"},{"issue":"9","key":"2024121121100472500_bib59","doi-asserted-by":"crossref","first-page":"1878","DOI":"10.1109\/TNNLS.2015.2462721","article-title":"Sequence prediction with sparse distributed hyperdimensional coding applied to the analysis of mobile phone use patterns","volume":"27","author":"R\u00e4s\u00e4nen","year":"2016","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"issue":"6","key":"2024121121100472500_bib60","first-page":"641","volume":"6","author":"Renner","year":"2024","journal-title":"Neuromorphic visual scene understanding with resonator networks"},{"key":"2024121121100472500_bib61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fneur.2021.701791","article-title":"A primer on hyperdimensional computing for iEEG seizure detection","volume":"12","author":"Schindler","year":"2021","journal-title":"Frontiers in Neurology"},{"key":"2024121121100472500_bib62","first-page":"7462","article-title":"Implicit neural representations with periodic activation functions","volume-title":"Advances in neural information processing systems","author":"Sitzmann","year":"2020"},{"key":"2024121121100472500_bib63","first-page":"465","article-title":"A random walk in Hamming space","volume-title":"Proceedings of the International Joint Conference on Neural Networks","author":"Smith","year":"1990"},{"key":"2024121121100472500_bib64","doi-asserted-by":"publisher","first-page":"510","DOI":"10.1007\/s12559-013-9243-y","article-title":"Modular composite representation","volume":"6","author":"Snaider","year":"2014","journal-title":"Cognitive Computation"},{"issue":"10","key":"2024121121100472500_bib65","doi-asserted-by":"publisher","first-page":"1330","DOI":"10.1038\/nn.2901","article-title":"Grid cells generate an analog error-correcting code for singularly precise neural computation","volume":"14","author":"Srinivasan","year":"2011","journal-title":"Nature Neuroscience"},{"key":"2024121121100472500_bib66","first-page":"7537","article-title":"Fourier features let networks learn high frequency functions in low dimensional domains","volume-title":"Advances in neural information processing systems","author":"Tancik","year":"2020"},{"key":"2024121121100472500_bib67","article-title":"Attention is all you need","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"2024121121100472500_bib68","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1007\/978-3-030-19311-9_19","article-title":"OIM: Oscillator-based Ising machines for solving combinatorial optimisation problems","volume-title":"Proceedings of the 18th International Conference on Unconventional Computation and Natural Computation","author":"Wang","year":"2019"},{"key":"2024121121100472500_bib69","first-page":"1","article-title":"SPORCO: A Python package for standard and convolutional sparse representations","volume":"15","author":"Wohlberg","year":"2017","journal-title":"Proceedings of the Python in Science Conference"},{"key":"2024121121100472500_bib70","first-page":"1157","article-title":"Understanding hyperdimensional computing for parallel single-pass learning","volume-title":"Advances in neural information processing systems","author":"Yu","year":"2022"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/37\/1\/1\/2483438\/neco_a_01723.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/37\/1\/1\/2483438\/neco_a_01723.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,11]],"date-time":"2024-12-11T21:10:30Z","timestamp":1733951430000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/37\/1\/1\/125267\/Computing-With-Residue-Numbers-in-High-Dimensional"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,12]]},"references-count":70,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,12,12]]},"published-print":{"date-parts":[[2024,12,12]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01723","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,12,12]]}}}