{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T18:26:31Z","timestamp":1787336791924,"version":"build-2736575974"},"reference-count":60,"publisher":"MIT Press","issue":"9","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,8,21]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>A cornerstone of our understanding of both biological and artificial neural networks is that they store information in the strengths of synaptic connections among the neurons. However, in contrast to the well-established theory for quantifying information encoded by the firing activity of neural networks, there does not exist a framework for quantifying information stored in the distribution of synaptic connection weights. Here, we develop a theoretical framework for synaptic information, by using densely connected Hebbian networks performing autoassociative memory tasks as the substrate, and by modeling data patterns that need to be stored as log-normal distributions. Specifically, we derive analytical approximations for Shannon mutual information between the data patterns and singletons, pairs, and arbitrary n-tuples of synaptic connections within the network. Our framework corroborates well-established insights regarding pattern storage capacity and the principle of distributed coding in neural firing activities. Notably, it formalizes the heterogeneity inherent in information encoding across synapses in a network, and discovers synergistic interactions among synapses, revealing that the information encoded jointly by all the synapses exceeds the sum of its parts. Taken together, this study introduces a powerful, interpretable framework for quantitatively understanding information storage in the synapses of neural networks, one that illustrates the duality of synaptic connectivity and neural population activity in learning and memory.<\/jats:p>","DOI":"10.1162\/neco.a.1566","type":"journal-article","created":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:35:57Z","timestamp":1785537357000},"page":"1545-1578","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Quantifying Information Stored in Synaptic Connections Rather Than in Firing Activities of Neural Networks"],"prefix":"10.1162","volume":"38","author":[{"given":"Xinhao","family":"Fan","sequence":"first","affiliation":[{"name":"Solomon H. Snyder Department of Neuroscience, Johns Hopkins University, Baltimore, MD 21218, U.S.A. xfan20@jhu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shreesh P.","family":"Mysore","sequence":"additional","affiliation":[{"name":"Department of Psychological and Brain Sciences, Johns Hopkins University; Solomon H. Snyder Department of Neuroscience, Johns Hopkins University; and Kavli Neuroscience Discovery Institute, Johns Hopkins University, Baltimore, MD 21218, U.S.A. mysore@jhu.edu"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2026,8,21]]},"reference":[{"issue":"4","key":"2026082113305390300_bib1","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1109\/TIT.1985.1057069","article-title":"Information capacity of the Hopfield model","volume":"31","author":"Abu-Mostafa","year":"1985","journal-title":"IEEE Transactions on Information Theory"},{"key":"2026082113305390300_bib2","author":"Achille","year":"2017","journal-title":"On the emergence of invariance and disentangling in deep representations"},{"key":"2026082113305390300_bib3","article-title":"Deep learning without weight transport","volume-title":"Advances in neural information processing systems","author":"Akrout","year":"2019"},{"issue":"14","key":"2026082113305390300_bib4","doi-asserted-by":"publisher","first-page":"1530","DOI":"10.1103\/PhysRevLett.55.1530","article-title":"Storing infinite numbers of patterns in a spin-glass model of neural networks","volume":"55","author":"Amit","year":"1985","journal-title":"Physical Review Letters"},{"issue":"4","key":"2026082113305390300_bib5","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1523\/JNEUROSCI.4759-06.2007","article-title":"Learning-induced enduring changes in functional connectivity among prefrontal cortical neurons","volume":"27","author":"Baeg","year":"2007","journal-title":"Journal of Neuroscience"},{"issue":"18","key":"2026082113305390300_bib6","doi-asserted-by":"publisher","first-page":"7641","DOI":"10.1073\/pnas.1018985108","article-title":"Dynamic reconfiguration of human brain networks during learning","volume":"108","author":"Bassett","year":"2011","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"4","key":"2026082113305390300_bib7","doi-asserted-by":"publisher","DOI":"10.1063\/1.4979282","article-title":"Multilayer motif analysis of brain networks","volume":"27","author":"Battiston","year":"2017","journal-title":"Chaos"},{"issue":"11","key":"2026082113305390300_bib8","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1016\/0166-2236(92)90005-S","article-title":"Reliability and information transmission in spiking neurons","volume":"15","author":"Bialek","year":"1992","journal-title":"Trends in Neurosciences"},{"issue":"11","key":"2026082113305390300_bib9","doi-asserted-by":"publisher","first-page":"947","DOI":"10.1038\/14731","article-title":"Information theory and neural coding","volume":"2","author":"Borst","year":"1999","journal-title":"Nature Neuroscience"},{"issue":"5","key":"2026082113305390300_bib10","doi-asserted-by":"publisher","first-page":"869","DOI":"10.1016\/S0893-6080(98)00035-5","article-title":"Information storage capacity of incompletely connected associative memories","volume":"11","author":"Bosch","year":"1998","journal-title":"Neural Networks"},{"issue":"7","key":"2026082113305390300_bib11","doi-asserted-by":"publisher","first-page":"1531","DOI":"10.1162\/089976600300015259","article-title":"Synergy in a neural code","volume":"12","author":"Brenner","year":"2000","journal-title":"Neural Computation"},{"issue":"4","key":"2026082113305390300_bib12","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1038\/nrn3687","article-title":"The log-dynamic brain: How skewed distributions affect network operations","volume":"15","author":"Buzs\u00e0ki","year":"2014","journal-title":"Nature Reviews Neuroscience"},{"key":"2026082113305390300_bib13","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3389\/fncom.2015.00057","article-title":"Triplet correlations among similarly tuned cells impact population coding","volume":"9","author":"Cayco-Gajic","year":"2015","journal-title":"Frontiers in Computational Neuroscience"},{"key":"2026082113305390300_bib14","volume-title":"Elements of information theory","author":"Cover","year":"1999"},{"key":"2026082113305390300_bib15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10827-011-0314-3","article-title":"Information theory in neuroscience","volume":"30","author":"Dimitrov","year":"2011","journal-title":"Journal of Computational Neuroscience"},{"issue":"3","key":"2026082113305390300_bib16","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1088\/0954-898X_3_3_002","article-title":"Dynamic properties of neural networks with adapting synapses","volume":"3","author":"Dong","year":"1992","journal-title":"Network: Computation in Neural Systems"},{"issue":"11","key":"2026082113305390300_bib17","doi-asserted-by":"publisher","first-page":"1184","DOI":"10.1038\/81460","article-title":"Neurocomputational models of working memory","volume":"3","author":"Durstewitz","year":"2000","journal-title":"Nature Neuroscience"},{"key":"2026082113305390300_bib18","author":"Fan","year":"2025","journal-title":"Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule"},{"issue":"1","key":"2026082113305390300_bib19","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1109\/TCOM.1960.1097606","article-title":"The sum of log-normal probability distributions in scatter transmission systems","volume":"8","author":"Fenton","year":"1960","journal-title":"IRE Transactions on Communications Systems"},{"issue":"7","key":"2026082113305390300_bib20","doi-asserted-by":"publisher","first-page":"2758","DOI":"10.1523\/JNEUROSCI.13-07-02758.1993","article-title":"How independent are the messages carried by adjacent inferior temporal cortical neurons?","volume":"13","author":"Gawne","year":"1993","journal-title":"Journal of Neuroscience"},{"issue":"3","key":"2026082113305390300_bib21","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1016\/j.tins.2007.12.002","article-title":"Stability of the fittest: Organizing learning through retroaxonal signals","volume":"31","author":"Harris","year":"2008","journal-title":"Trends in Neurosciences"},{"key":"2026082113305390300_bib22","volume-title":"The organization of behavior: A neuropsychological theory","author":"Hebb","year":"1949"},{"key":"2026082113305390300_bib23","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/168304.168306","article-title":"Keeping the neural networks simple by minimizing the description length of the weights","volume-title":"Proceedings of the Sixth Annual Conference on Computational Learning Theory","author":"Hinton","year":"1993"},{"issue":"15","key":"2026082113305390300_bib24","doi-asserted-by":"publisher","first-page":"6462","DOI":"10.1073\/pnas.88.15.6462","article-title":"Olfactory computation and object perception","volume":"88","author":"Hopfield","year":"1991","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2026082113305390300_bib25","first-page":"541","article-title":"Statistics of natural images and models","volume-title":"Proceedings of the 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition","author":"Huang","year":"1999"},{"issue":"2","key":"2026082113305390300_bib26","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1093\/cercor\/bhs006","article-title":"Interpyramid spike trans-mission stabilizes the sparseness of recurrent network activity","volume":"23","author":"Ikegaya","year":"2013","journal-title":"Cerebral Cortex"},{"issue":"1","key":"2026082113305390300_bib27","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1038\/s41467-020-20722-y","article-title":"Scaling of sensory information in large neural populations shows signatures of information-limiting correlations","volume":"12","author":"Kafashan","year":"2021","journal-title":"Nature Communications"},{"issue":"1","key":"2026082113305390300_bib28","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1038\/s41467-024-44877-0","article-title":"Distinguishing examples while building concepts in hippocampal and artificial networks","volume":"15","author":"Kang","year":"2024","journal-title":"Nature Communications"},{"issue":"2","key":"2026082113305390300_bib29","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1162\/neco.2009.08-07-588","article-title":"Memory capacities for synaptic and structural plasticity","volume":"22","author":"Knoblauch","year":"2010","journal-title":"Neural Computation"},{"issue":"6","key":"2026082113305390300_bib30","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Physical Review E\u2014Statistical, Nonlinear, and Soft Matter Physics"},{"issue":"1","key":"2026082113305390300_bib31","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1038\/nrn1301","article-title":"Structural plasticity and memory","volume":"5","author":"Lamprecht","year":"2004","journal-title":"Nature Reviews Neuroscience"},{"issue":"1","key":"2026082113305390300_bib32","doi-asserted-by":"publisher","first-page":"13276","DOI":"10.1038\/ncomms13276","article-title":"Random synaptic feedback weights support error backpropagation for deep learning","volume":"7","author":"Lillicrap","year":"2016","journal-title":"Nature Communications"},{"issue":"3","key":"2026082113305390300_bib33","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1109\/2.36","article-title":"Self-organization in a perceptual network","volume":"21","author":"Linsker","year":"1988","journal-title":"Computer"},{"issue":"6","key":"2026082113305390300_bib34","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1038\/s41593-022-01070-0","article-title":"A synergistic core for human brain evolution and cognition","volume":"25","author":"Luppi","year":"2022","journal-title":"Nature Neuroscience"},{"issue":"5869","key":"2026082113305390300_bib35","doi-asserted-by":"publisher","first-page":"1543","DOI":"10.1126\/science.1150769","article-title":"Synaptic theory of working memory","volume":"319","author":"Mongillo","year":"2008","journal-title":"Science"},{"issue":"5","key":"2026082113305390300_bib36","doi-asserted-by":"publisher","first-page":"1093","DOI":"10.1088\/0305-4470\/24\/5\/023","article-title":"Associative memory: On the (puzzling) sparse coding limit","volume":"24","author":"Nadal","year":"1991","journal-title":"Journal of Physics A: Mathematical and General"},{"issue":"1","key":"2026082113305390300_bib37","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1007\/BF00337019","article-title":"On associative memory","volume":"36","author":"Palm","year":"1980","journal-title":"Biological Cybernetics"},{"issue":"5","key":"2026082113305390300_bib37a","doi-asserted-by":"publisher","first-page":"703","DOI":"10.1162\/neco.1992.4.5.703","article-title":"On the information storage capacity of local learning rules","volume":"4","author":"Palm","year":"1992","journal-title":"Neural Computation"},{"key":"2026082113305390300_bib38","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1007\/978-1-4612-0723-8_3","article-title":"Associative data storage and retrieval in neural networks","volume-title":"Models of neural networks III: Association, generalization, and representation","author":"Palm","year":"1996"},{"issue":"22","key":"2026082113305390300_bib39","doi-asserted-by":"publisher","first-page":"6908","DOI":"10.1073\/pnas.1506855112","article-title":"Predictive in-formation in a sensory population","volume":"112","author":"Palmer","year":"2015","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"8042","key":"2026082113305390300_bib40","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-08139-9","article-title":"Intermittent rate coding and cue-specific ensembles support working memory","volume":"636","author":"Panichello","year":"2024","journal-title":"Nature"},{"issue":"1423","key":"2026082113305390300_bib41","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.1098\/rspb.1999.0736","article-title":"Correlations and the encoding of information in the nervous system","volume":"266","author":"Panzeri","year":"1999","journal-title":"Proceedings of the Royal Society of London. Series B: Biological Sciences"},{"key":"2026082113305390300_bib42","doi-asserted-by":"publisher","first-page":"1112","DOI":"10.3758\/s13423-014-0585-6","article-title":"Zipf\u2019s word frequency law in natural language: A critical review and future directions","volume":"21","author":"Piantadosi","year":"2014","journal-title":"Psychonomic Bulletin and Review"},{"issue":"3","key":"2026082113305390300_bib43","doi-asserted-by":"publisher","first-page":"173","DOI":"10.1038\/nrn2578","article-title":"Extracting information from neuronal populations: Information theory and decoding approaches","volume":"10","author":"Quian Quiroga","year":"2009","journal-title":"Nature Reviews Neuroscience"},{"issue":"1","key":"2026082113305390300_bib44","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1146\/annurev-psych-120710-100344","article-title":"Distributed representations in memory: Insights from functional brain imaging","volume":"63","author":"Rissman","year":"2012","journal-title":"Annual Review of Psychology"},{"issue":"45","key":"2026082113305390300_bib45","doi-asserted-by":"publisher","first-page":"16217","DOI":"10.1523\/JNEUROSCI.1677-11.2011","article-title":"On the distribution of firing rates in networks of cortical neurons","volume":"31","author":"Roxin","year":"2011","journal-title":"Journal of Neuroscience"},{"issue":"37","key":"2026082113305390300_bib46","doi-asserted-by":"publisher","first-page":"11539","DOI":"10.1523\/JNEUROSCI.23-37-11539.2003","article-title":"Synergy, redundancy, and independence in population codes","volume":"23","author":"Schneidman","year":"2003","journal-title":"Journal of Neuroscience"},{"issue":"3","key":"2026082113305390300_bib47","doi-asserted-by":"publisher","first-page":"e68","DOI":"10.1371\/journal.pbio.0030068","article-title":"Highly nonrandom features of synaptic connectivity in local cortical circuits","volume":"3","author":"Song","year":"2005","journal-title":"PLOS Biology"},{"issue":"11","key":"2026082113305390300_bib48","doi-asserted-by":"publisher","first-page":"e369","DOI":"10.1371\/journal.pbio.0020369","article-title":"Motifs in brain networks","volume":"2","author":"Sporns","year":"2004","journal-title":"PLOS Biology"},{"issue":"5","key":"2026082113305390300_bib49","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1007\/s00422-003-0439-1","article-title":"Distributed coding of sound locations in the auditory cortex","volume":"89","author":"Stecker","year":"2003","journal-title":"Biological Cybernetics"},{"issue":"7786","key":"2026082113305390300_bib50","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1038\/s41586-019-1787-x","article-title":"Distributed coding of choice, action and engagement across the mouse brain","volume":"576","author":"Steinmetz","year":"2019","journal-title":"Nature"},{"issue":"7","key":"2026082113305390300_bib51","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1016\/j.tics.2015.05.004","article-title":"\u201cActivity-silent\u201d working memory in prefrontal cortex: A dynamic coding framework","volume":"19","author":"Stokes","year":"2015","journal-title":"Trends in Cognitive Sciences"},{"issue":"8","key":"2026082113305390300_bib52","doi-asserted-by":"publisher","first-page":"1438","DOI":"10.1038\/s41593-023-01382-9","article-title":"Organizing memories for generalization in complementary learning systems","volume":"26","author":"Sun","year":"2023","journal-title":"Nature Neuroscience"},{"issue":"3","key":"2026082113305390300_bib53","doi-asserted-by":"publisher","DOI":"10.1523\/ENEURO.0052-18.2018","article-title":"A tutorial for information theory in neuroscience","volume":"5","author":"Timme","year":"2018","journal-title":"eNeuro"},{"key":"2026082113305390300_bib54","first-page":"1","article-title":"Deep learning and the information bottle-neck principle","volume-title":"Proceedings of the 2015 IEEE Information Theory Workshop","author":"Tishby","year":"2015"},{"key":"2026082113305390300_bib55","doi-asserted-by":"publisher","first-page":"317318","DOI":"10.1038\/258317a0","article-title":"\u201c1\/f noise\u201d in music and speech","volume":"258","author":"Voss","year":"1975","journal-title":"Nature"},{"key":"2026082113305390300_bib56","author":"Williams","year":"2010","journal-title":"Nonnegative decomposition of multivariate information"},{"issue":"5197","key":"2026082113305390300_bib57","doi-asserted-by":"publisher","first-page":"960","DOI":"10.1038\/222960a0","article-title":"Nonholographic associative memory","volume":"222","author":"Willshaw","year":"1969","journal-title":"Nature"},{"key":"2026082113305390300_bib58","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1016\/j.pneurobio.2012.09.004","article-title":"Population-wide distributions of neural activity during perceptual decision-making","volume":"103","author":"Wohrer","year":"2013","journal-title":"Progress in Neurobiology"},{"issue":"2","key":"2026082113305390300_bib59","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1162\/089976603762552988","article-title":"Equivalence of backpropagation and contrastive Hebbian learning in a layered network","volume":"15","author":"Xie","year":"2003","journal-title":"Neural Computation"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/38\/9\/1545\/2618238\/neco.a.1566.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/38\/9\/1545\/2618238\/neco.a.1566.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T17:30:58Z","timestamp":1787333458000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/38\/9\/1545\/137979\/Quantifying-Information-Stored-in-Synaptic"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,21]]},"references-count":60,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,8,21]]},"published-print":{"date-parts":[[2026,8,21]]}},"URL":"https:\/\/doi.org\/10.1162\/neco.a.1566","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,9]]},"published":{"date-parts":[[2026,8,21]]}}}