{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,9]],"date-time":"2025-12-09T08:29:51Z","timestamp":1765268991241},"reference-count":46,"publisher":"MIT Press","issue":"11","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,10,11]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We discuss prototype formation in the Hopfield network. Typically, Hebbian learning with highly correlated states leads to degraded memory performance. We show that this type of learning can lead to prototype formation, where unlearned states emerge as representatives of large correlated subsets of states, alleviating capacity woes. This process has similarities to prototype learning in human cognition. We provide a substantial literature review of prototype learning in associative memories, covering contributions from psychology, statistical physics, and computer science. We analyze prototype formation from a theoretical perspective and derive a stability condition for these states based on the number of examples of the prototype presented for learning, the noise in those examples, and the number of nonexample states presented. The stability condition is used to construct a probability of stability for a prototype state as the factors of stability change. We also note similarities to traditional network analysis, allowing us to find a prototype capacity. We corroborate these expectations of prototype formation with experiments using a simple Hopfield network with standard Hebbian learning. We extend our experiments to a Hopfield network trained on data with multiple prototypes and find the network is capable of stabilizing multiple prototypes concurrently. We measure the basins of attraction of the multiple prototype states, finding attractor strength grows with the number of examples and the agreement of examples. We link the stability and dominance of prototype states to the energy profile of these states, particularly when comparing the profile shape to target states or other spurious states.<\/jats:p>","DOI":"10.1162\/neco_a_01704","type":"journal-article","created":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T15:23:43Z","timestamp":1725031423000},"page":"2322-2364","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["Prototype Analysis in Hopfield Networks With Hebbian Learning"],"prefix":"10.1162","volume":"36","author":[{"given":"Hayden","family":"McAlister","sequence":"first","affiliation":[{"name":"School of Computing, University of Otago, Dunedin 9016, New Zealand mcaha814@student.otago.ac.nz"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anthony","family":"Robins","sequence":"additional","affiliation":[{"name":"School of Computing, University of Otago, Dunedin 9016, New Zealand anthony.robins@otago.ac.nz"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lech","family":"Szymanski","sequence":"additional","affiliation":[{"name":"School of Computing, University of Otago, Dunedin 9016, New Zealand lech.szymanski@otago.ac.nz"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2024,10,11]]},"reference":[{"issue":"2","key":"2024100916001038000_bib1","doi-asserted-by":"publisher","first-page":"1007","DOI":"10.1103\/PhysRevA.32.1007","article-title":"Spin-glass models of neural networks","volume":"32","author":"Amit","year":"1985","journal-title":"Physical Review A"},{"issue":"14","key":"2024100916001038000_bib2","doi-asserted-by":"publisher","first-page":"1530","DOI":"10.1103\/PhysRevLett.55.1530","article-title":"Storing infinite numbers of patterns in a spin-glass model of neural networks","volume":"55","author":"Amit","year":"1985","journal-title":"Physical Review Letters"},{"issue":"1","key":"2024100916001038000_bib3","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1016\/0003-4916(87)90092-3","article-title":"Statistical mechanics of neural networks near saturation","volume":"173","author":"Amit","year":"1987","journal-title":"Annals of Physics"},{"key":"2024100916001038000_bib4","doi-asserted-by":"publisher","first-page":"1483","DOI":"10.1109\/72.641470","article-title":"On the problem of spurious patterns in neural associative memory models","volume-title":"IEEE Transactions on Neural Networks","author":"Athithan","year":"1997"},{"year":"2014","author":"Bahdanau","journal-title":"Neural machine translation by jointly learning to align and translate.","key":"2024100916001038000_bib5"},{"key":"2024100916001038000_bib6","first-page":"192","article-title":"Hopfield networks for vector quantization","volume-title":"Artificial neural networks and machine learning","author":"Bauckhage","year":"2020"},{"issue":"1","key":"2024100916001038000_bib7","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1162\/neco_a_01452","article-title":"Confidence-controlled Hebbian learning efficiently extracts category membership from stimuli encoded in view of a categorization task","volume":"34","author":"Berlemont","year":"2022","journal-title":"Neural Computation"},{"key":"2024100916001038000_bib8","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.4310\/ATMP.2001.v5.n6.a2","article-title":"The spin-glass phase-transition in the Hopfield model with p-spin interactions","volume":"5","author":"Bovier","year":"2001","journal-title":"Advances in Theoretical and Mathematical Physics"},{"key":"2024100916001038000_bib9","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1109\/18.52486","article-title":"On the number of spurious memories in the Hopfield model (neural network)","volume-title":"IEEE Transactions on Information Theory","author":"Bruck","year":"1990"},{"issue":"1","key":"2024100916001038000_bib10","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1002\/int.4550030106","article-title":"A study on neural networks","volume":"3","author":"Bruck","year":"1988","journal-title":"International Journal of Intelligent Systems"},{"issue":"2","key":"2024100916001038000_bib11","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1007\/s10955-017-1806-y","article-title":"On a model of associative memory with huge storage capacity","volume":"168","author":"Demircigil","year":"2017","journal-title":"Journal of Statistical Physics"},{"issue":"6","key":"2024100916001038000_bib12","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST database of handwritten digit images for machine learning research","volume":"29","author":"Deng","year":"2012","journal-title":"IEEE Signal Processing Magazine"},{"issue":"9","key":"2024100916001038000_bib13","doi-asserted-by":"publisher","first-page":"949","DOI":"10.1103\/PhysRevLett.58.949","article-title":"Learning of correlated patterns in spin-glass networks by local learning rules","volume":"58","author":"Diederich","year":"1987","journal-title":"Physical Review Letters"},{"issue":"5","key":"2024100916001038000_bib14","doi-asserted-by":"publisher","first-page":"902","DOI":"10.1162\/neco.1994.6.5.902","article-title":"An internal mechanism for detecting parasite attractors in a Hopfield network","volume":"6","author":"Gascuel","year":"1994","journal-title":"Neural Computation"},{"issue":"3","key":"2024100916001038000_bib15","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1037\/0096-3445.117.3.227","article-title":"From conditioning to category learning: An adaptive network model","volume":"117","author":"Gluck","year":"1988","journal-title":"Journal of Experimental Psychology: General"},{"key":"2024100916001038000_bib16","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1016\/j.neunet.2017.04.007","article-title":"Hopfield networks as a model of prototype-based category learning: A method to distinguish trained, spurious, and prototypical attractors","volume":"91","author":"Gorman","year":"2017","journal-title":"Neural Networks"},{"key":"2024100916001038000_bib17","volume-title":"Neural networks and learning machines","author":"Haykin","year":"2009","edition":"3rd"},{"volume-title":"The organization of behavior: A neuropsychological theory","year":"1949","author":"Hebb","key":"2024100916001038000_bib18"},{"volume-title":"Introduction to the theory of neural computation","year":"1991","author":"Hertz","key":"2024100916001038000_bib19"},{"issue":"8","key":"2024100916001038000_bib20","doi-asserted-by":"publisher","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"Hopfield","year":"1982","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"issue":"10","key":"2024100916001038000_bib21","doi-asserted-by":"publisher","first-page":"3088","DOI":"10.1073\/pnas.81.10.3088","article-title":"Neurons with graded response have collective computational properties like those of two-state neurons","volume":"81","author":"Hopfield","year":"1984","journal-title":"Proceedings of the National Academy of Sciences of the United States of America"},{"issue":"3","key":"2024100916001038000_bib22","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1007\/BF00339943","article-title":"\u201cNeural\u201d computation of decisions in optimization problems","volume":"52","author":"Hopfield","year":"1985","journal-title":"Biological Cybernetics"},{"issue":"11","key":"2024100916001038000_bib23","doi-asserted-by":"publisher","first-page":"4384","DOI":"10.1103\/PhysRevB.17.4384","article-title":"Infinite-ranged models of spin-glasses","volume":"17","author":"Kirkpatrick","year":"1978","journal-title":"Physical Review B"},{"issue":"4","key":"2024100916001038000_bib24","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1109\/TC.1972.5008975","article-title":"Correlation matrix memories","volume":"C-21","author":"Kohonen","year":"1972","journal-title":"IEEE Transactions on Computers"},{"volume-title":"Associative memory: A system-theoretical approach","year":"1978","author":"Kohonen","key":"2024100916001038000_bib25"},{"key":"2024100916001038000_bib26","article-title":"ImageNet classification with deep convolutional neural networks","volume-title":"Advances in neural information processing systems","author":"Krizhevsky","year":"2012"},{"year":"2016","author":"Krotov","journal-title":"Dense associative memory for pattern recognition","key":"2024100916001038000_bib27"},{"issue":"12","key":"2024100916001038000_bib28","doi-asserted-by":"publisher","first-page":"3151","DOI":"10.1162\/neco_a_01143","article-title":"Dense associative memory is robust to adversarial inputs","volume":"30","author":"Krotov","year":"2018","journal-title":"Neural Computation"},{"key":"2024100916001038000_bib29","first-page":"267","article-title":"Models of categorization","volume-title":"The Cambridge handbook of computational psychology","author":"Kruschke","year":"2008"},{"issue":"1","key":"2024100916001038000_bib30","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/0025-5564(74)90031-5","article-title":"The existence of persistent states in the brain","volume":"19","author":"Little","year":"1974","journal-title":"Mathematical Biosciences"},{"issue":"9","key":"2024100916001038000_bib31","doi-asserted-by":"publisher","first-page":"1207","DOI":"10.1016\/S0893-6080(99)00042-8","article-title":"The asymptotic memory capacity of the generalized Hopfield network","volume":"12","author":"Ma","year":"1999","journal-title":"Neural Networks"},{"key":"2024100916001038000_bib32","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/978-3-642-82405-0_3","article-title":"The Ising model","volume-title":"The theory of magnetism II: Thermodynamics and statistical mechanics","author":"Mattis","year":"1985"},{"key":"2024100916001038000_bib33","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1109\/TIT.1987.1057328","article-title":"The capacity of the Hopfield associative memory","volume-title":"IEEE Transactions on Information Theory","author":"McEliece","year":"1987"},{"issue":"3","key":"2024100916001038000_bib34","doi-asserted-by":"publisher","first-page":"352","DOI":"10.3758\/BF03200862","article-title":"Comparing models of rule-based classification learning: A replication and extension of Shepard, Hovland, and Jenkins (1961)","volume":"22","author":"Nosofsky","year":"1994","journal-title":"Memory and Cognition"},{"issue":"3","key":"2024100916001038000_bib35","doi-asserted-by":"publisher","first-page":"345","DOI":"10.3758\/BF03208813","article-title":"A rule-plus-exception model for classifying objects in continuous-dimension spaces","volume":"5","author":"Nosofsky","year":"1998","journal-title":"Psychonomic Bulletin and Review"},{"year":"2021","author":"Ramsauer","journal-title":"Hopfield networks is all you need.","key":"2024100916001038000_bib36"},{"issue":"2","key":"2024100916001038000_bib37","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1080\/095400998116530","article-title":"Catastrophic forgetting and the pseudorehearsal solution in Hopfield-type networks","volume":"10","author":"Robins","year":"1998","journal-title":"Connection Science"},{"issue":"3","key":"2024100916001038000_bib38","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1016\/j.neunet.2003.11.007","article-title":"A robust method for distinguishing between learned and spurious attractors","volume":"17","author":"Robins","year":"2004","journal-title":"Neural Networks"},{"issue":"3","key":"2024100916001038000_bib39","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1016\/0010-0285(73)90017-0","article-title":"Natural categories","volume":"4","author":"Rosch","year":"1973","journal-title":"Cognitive Psychology"},{"issue":"3","key":"2024100916001038000_bib40","doi-asserted-by":"publisher","first-page":"2571","DOI":"10.1103\/PhysRevA.34.2571","article-title":"Neural networks with nonlinear synapses and a static noise","volume":"34","author":"Sompolinsky","year":"1986","journal-title":"Physical Review A"},{"issue":"4","key":"2024100916001038000_bib41","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1007\/BF00272311","article-title":"Adaptive networks using learning matrices","volume":"2","author":"Steinbuch","year":"1965","journal-title":"Kybernetik"},{"issue":"6","key":"2024100916001038000_bib42","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1109\/PGEC.1963.263588","article-title":"Learning matrices and their applications","volume":"EC-12","author":"Steinbuch","year":"1963","journal-title":"IEEE Transactions on Electronic Computers"},{"key":"2024100916001038000_bib43","first-page":"451","article-title":"Increasing the capacity of a Hopfield network without sacrificing functionality","volume-title":"Artificial neural networks\u2014ICANN\u201997","author":"Storkey","year":"1997"},{"year":"2017","author":"Vaswani","journal-title":"Attention is all you need","key":"2024100916001038000_bib44"},{"key":"2024100916001038000_bib45","first-page":"96","article-title":"Adaptive switching circuits","author":"Widrow","year":"1960","journal-title":"1980 IRE WESCON Convention Record"},{"key":"2024100916001038000_bib46","doi-asserted-by":"publisher","first-page":"1058","DOI":"10.1109\/72.870039","article-title":"An efficient learning algorithm for associative memories","volume-title":"IEEE Transactions on Neural Networks","author":"Wu","year":"2000"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/11\/2322\/2474365\/neco_a_01704.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/11\/2322\/2474365\/neco_a_01704.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,9]],"date-time":"2024-10-09T16:00:54Z","timestamp":1728489654000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/36\/11\/2322\/124220\/Prototype-Analysis-in-Hopfield-Networks-With"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,11]]},"references-count":46,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,10,11]]},"published-print":{"date-parts":[[2024,10,11]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01704","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"type":"print","value":"0899-7667"},{"type":"electronic","value":"1530-888X"}],"subject":[],"published-other":{"date-parts":[[2024,11]]},"published":{"date-parts":[[2024,10,11]]}}}