{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T20:26:15Z","timestamp":1747254375315},"reference-count":38,"publisher":"MIT Press - Journals","issue":"9","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Associative memories enjoy many interesting properties in terms of error correction capabilities, robustness to noise, storage capacity, and retrieval performance, and their usage spans over a large set of applications. In this letter, we investigate and extend tournament-based neural networks, originally proposed by Jiang, Gripon, Berrou, and Rabbat (2016), a novel sequence storage associative memory architecture with high memory efficiency and accurate sequence retrieval. We propose a more general method for learning the sequences, which we call feedback tournament-based neural networks. The retrieval process is also extended to both directions: forward and backward\u2014in other words, any large-enough segment of a sequence can produce the whole sequence. Furthermore, two retrieval algorithms, cache-winner and explore-winner, are introduced to increase the retrieval performance. Through simulation results, we shed light on the strengths and weaknesses of each algorithm.<\/jats:p>","DOI":"10.1162\/neco_a_01417","type":"journal-article","created":{"date-parts":[[2021,6,30]],"date-time":"2021-06-30T21:34:31Z","timestamp":1625088871000},"page":"2550-2577","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["On Neural Associative Memory Structures: Storage and Retrieval of Sequences in a Chain of Tournaments"],"prefix":"10.1162","volume":"33","author":[{"given":"Asieh Abolpour","family":"Mofrad","sequence":"first","affiliation":[{"name":"Selmer Center, Department of Informatics, University of Bergen, 5020 Bergen, Norway asieh.abolpour-mofrad@oslomet.no"}]},{"given":"Samaneh Abolpour","family":"Mofrad","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical Engineering, and Mathematical Sciences, Western Norway University of Applied Sciences, 5063 Bergen, Norway, and Mohn Medical Imaging and Visualization Center, Haukeland University Hospital, 5021 Bergen, Norway Samaneh.Abolpour.Mofrad@hvl.no"}]},{"given":"Anis","family":"Yazidi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, OsloMet, Oslo Metropolitan University, 0130 Oslo, Norway and Department of Plastic and Reconstructive Surgery, Oslo University Hospital, 0318 Oslo, Norway Anis.Yazidi@oslomet.no"}]},{"given":"Matthew Geoffrey","family":"Parker","sequence":"additional","affiliation":[{"name":"Selmer Center, Department of Informatics, University of Bergen, 5020 Bergen, Norway Matthew.Parker@ii.uib.no"}]}],"member":"281","published-online":{"date-parts":[[2021,8,19]]},"reference":[{"key":"2021121723254111400_B1","first-page":"439","article-title":"A neural network model for solving the feature correspondence problem","author":"Aboudib","year":"2016","journal-title":"Proceedings of the International Conference on Artificial Neural Networks"},{"key":"2021121723254111400_B2","first-page":"140","article-title":"A study of retrieval algorithms of sparse messages in networks of neural cliques","author":"Aboudib","year":"2014","journal-title":"Proceedings of COGNITIVE 2014: The 6th International Conference on Advanced Cognitive Technologies and Applications"},{"issue":"5","key":"2021121723254111400_B3","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1109\/TNNLS.2013.2285253","article-title":"Storing sparse messages in networks of neural cliques","volume":"25","author":"Aliabadi","year":"2014","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2021121723254111400_B4","first-page":"167","article-title":"Information, noise, coding, modulation: What about the brain?","author":"Berrou","year":"2014","journal-title":"Proceedings of the 8th International Symposium on Turbo Codes and Iterative Information Processing"},{"key":"2021121723254111400_B5","first-page":"1","article-title":"Coded Hopfield networks","author":"Berrou","year":"2010","journal-title":"Proceedings of the 6th International Symposium on Turbo Codes and Iterative Information Processing"},{"key":"2021121723254111400_B6","article-title":"A connectionist model of reading with error correction properties.","author":"Berrou","year":"2018","journal-title":"Proceedings of the 7th Language and Technology Conference on Human Language Technology. Challenges for Computer Science and Linguistics"},{"key":"2021121723254111400_B7","first-page":"262","article-title":"Huffman coding for storing non-uniformly distributed messages in networks of neural cliques","author":"Boguslawski","year":"2014","journal-title":"Proceedings of the 28th AAAI Conference on Artificial Intelligence"},{"key":"2021121723254111400_B8","first-page":"1422","article-title":"Sequence learning with hidden units in spiking neural networks.","volume":"31","author":"Brea","year":"2011","journal-title":"Advances in neural information processing systems"},{"key":"2021121723254111400_B9","doi-asserted-by":"crossref","first-page":"2501","DOI":"10.1109\/ISCAS.2015.7169193","article-title":"Algorithm and implementation of an associative memory for oriented edge detection using improved clustered neural networks.","author":"Danilo","year":"2015","journal-title":"Proceedings of the 2015 IEEE International Symposium on Circuits and Systems"},{"issue":"7","key":"2021121723254111400_B10","doi-asserted-by":"crossref","first-page":"1087","DOI":"10.1109\/TNN.2011.2146789","article-title":"Sparse neural networks with large learning diversity","volume":"22","author":"Gripon","year":"2011","journal-title":"IEEE Transactions on Neural Networks"},{"key":"2021121723254111400_B11","first-page":"269","article-title":"Nearly-optimal associative memories based on distributed constant weight codes","author":"Gripon","year":"2012","journal-title":"Proceedings of the Information Theory and Applications Workshop"},{"issue":"1","key":"2021121723254111400_B12","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/s10955-016-1530-z","article-title":"A comparative study of sparse associative memories","volume":"164","author":"Gripon","year":"2016","journal-title":"Journal of Statistical Physics"},{"key":"2021121723254111400_B13","article-title":"Finding all matches in a database using binary neural networks","author":"Hacene","year":"2017","journal-title":"Proceedings of COGNITIVE 2017: International Conference on Advanced Cognitive Technologies and Applications"},{"issue":"9","key":"2021121723254111400_B14","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1007\/s11265-019-01450-z","article-title":"Budget restricted incremental learning with pre-trained convolutional neural networks and binary associative memories","volume":"91","author":"Hacene","year":"2019","journal-title":"Journal of Signal Processing Systems"},{"key":"2021121723254111400_B15","doi-asserted-by":"crossref","DOI":"10.3389\/fncir.2016.00023","article-title":"Why neurons have thousands of synapses, a theory of sequence memory in neocortex","volume":"10","author":"Hawkins","year":"2016","journal-title":"Frontiers in Neural Circuits"},{"key":"2021121723254111400_B16","author":"Hawkins","year":"2007","journal-title":"On intelligence: How a new understanding of the brain will lead to the creation of truly intelligent machines"},{"issue":"1521","key":"2021121723254111400_B17","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.1098\/rstb.2008.0322","article-title":"Sequence memory for prediction, inference and behaviour","volume":"364","author":"Hawkins","year":"2009","journal-title":"Philosophical Transactions of the Royal Society of London B: Biological Sciences"},{"issue":"5","key":"2021121723254111400_B18","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1162\/neco_a_01181","article-title":"Sparse associative memory","volume":"31","author":"Hoffmann","year":"2019","journal-title":"Neural Computation"},{"issue":"8","key":"2021121723254111400_B19","doi-asserted-by":"crossref","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"Hopfield","year":"1982","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"5","key":"2021121723254111400_B20","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1162\/neco.2007.09-06-345","article-title":"Searching for memories, sudoku, implicit check bits, and the iterative use of not-always-correct rapid neural computation","volume":"20","author":"Hopfield","year":"2008","journal-title":"Neural Computation"},{"issue":"4","key":"2021121723254111400_B21","doi-asserted-by":"crossref","first-page":"642","DOI":"10.1109\/TVLSI.2014.2316733","article-title":"Algorithm and architecture for a low-power content-addressable memory based on sparse clustered networks","volume":"23","author":"Jarollahi","year":"2015","journal-title":"IEEE Transactions on Very Large Scale Integration Systems"},{"issue":"3","key":"2021121723254111400_B22","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1007\/s11265-014-0886-z","article-title":"Algorithm and architecture of fully-parallel associative memories based on sparse clustered networks","volume":"76","author":"Jarollahi","year":"2014","journal-title":"Journal of Signal Processing Systems"},{"key":"2021121723254111400_B23","author":"Jiang","year":"2014","journal-title":"Storing sequences in binary neural networks with high efficiency"},{"issue":"5","key":"2021121723254111400_B24","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1109\/TNNLS.2015.2431319","article-title":"Storing sequences in binary tournament-based neural networks","volume":"27","author":"Jiang","year":"2016","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2021121723254111400_B25","first-page":"400","article-title":"Improved retrieval for challenging scenarios in clique-based neural networks","author":"Jiang","year":"2015","journal-title":"Proceedings of the International Work Conference on Artificial Neural Networks"},{"issue":"10","key":"2021121723254111400_B26","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0184683","article-title":"Enhanced storage capacity with errors in scale-free Hopfield neural networks: An analytical study","volume":"12","author":"Kim","year":"2017","journal-title":"PLOS One"},{"key":"2021121723254111400_B27","first-page":"1172","volume-title":"Advances in neural information processing systems, 29","author":"Krotov","year":"2016"},{"issue":"5","key":"2021121723254111400_B28","doi-asserted-by":"crossref","first-page":"1704","DOI":"10.1109\/TCSI.2018.2881508","article-title":"A fully flexible circuit implementation of clique-based neural networks in 65-nm CMOS","volume":"66","author":"Larras","year":"2018","journal-title":"IEEE Transactions on Circuits and Systems I: Regular Papers"},{"key":"2021121723254111400_B29","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1109\/JETCAS.2020.3023481","article-title":"On the distribution of clique-based neural networks for edge AI","volume":"10","author":"Larras","year":"2020","journal-title":"IEEE Journal on Emerging and Selected Topics in Circuits and Systems"},{"key":"2021121723254111400_B30","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1109\/HPCS.2017.88","article-title":"Large-scale memory of sequences using binary sparse neural networks on GPU.","author":"Marques","year":"2017","journal-title":"Proceedings of the 2017 International Conference on High Performance Computing and Simulation"},{"key":"2021121723254111400_B31","first-page":"493","article-title":"Extended Hopfield network for sequence learning: Application to gesture recognition.","author":"Maurer","year":"2005","journal-title":"Proceedings of the International Conference on Artificial Neural Networks"},{"key":"2021121723254111400_B32","first-page":"178","article-title":"Neural network associative memories with local coding.","author":"Mofrad","year":"2015","journal-title":"Proceedings of the 2015 IEEE 14th Canadian Workshop on Information Theory"},{"key":"2021121723254111400_B33","doi-asserted-by":"crossref","first-page":"1681","DOI":"10.1162\/NECO_a_00964","article-title":"Nested-clique network model of neural associative memory","volume":"29","author":"Mofrad","year":"2017","journal-title":"Neural Computation"},{"key":"2021121723254111400_B34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/NECO_a_00856","article-title":"Clique based neural associative memories with local coding and pre-coding","volume":"28","author":"Mofrad","year":"2016","journal-title":"Neural Computation"},{"issue":"4","key":"2021121723254111400_B35","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.conb.2004.07.007","article-title":"Sparse coding of sensory inputs","volume":"14","author":"Olshausen","year":"2004","journal-title":"Current Opinion in Neurobiology"},{"key":"2021121723254111400_B36","article-title":"A cortical sparse distributed coding model linking mini- and macrocolumn-scale functionality","volume":"4","author":"Rinkus","year":"2010","journal-title":"Frontiers in Neuroanatomy"},{"key":"2021121723254111400_B37","doi-asserted-by":"crossref","first-page":"960","DOI":"10.1038\/222960a0","article-title":"Non-holographic associative memory","volume":"222","author":"Willshaw","year":"1969","journal-title":"Nature"},{"key":"2021121723254111400_B38","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1109\/HPCSim.2014.6903755","article-title":"A GPU-based associative memory using sparse neural networks","author":"Yao","year":"2014","journal-title":"Proceedings of the 2014 International Conference on High Performance Computing and Simulation"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/33\/9\/2550\/1978112\/neco_a_01417.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/33\/9\/2550\/1978112\/neco_a_01417.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,17]],"date-time":"2021-12-17T23:26:41Z","timestamp":1639783601000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/33\/9\/2550\/102626\/On-Neural-Associative-Memory-Structures-Storage"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,19]]},"references-count":38,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,8,19]]},"published-print":{"date-parts":[[2021,8,19]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01417","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,9]]},"published":{"date-parts":[[2021,8,19]]}}}