{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T13:04:47Z","timestamp":1786539887707,"version":"3.56.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1012553","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T00:00:00Z","timestamp":1730937600000}}],"reference-count":43,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T00:00:00Z","timestamp":1730073600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>\n                    The brain is not only constrained by energy needed to fuel computation, but it is also constrained by energy needed to form memories. Experiments have shown that learning simple conditioning tasks which might require only a few synaptic updates, already carries a significant metabolic cost. Yet, learning a task like MNIST to 95% accuracy appears to require at least 10\n                    <jats:sup>8<\/jats:sup>\n                    synaptic updates. Therefore the brain has likely evolved to be able to learn using as little energy as possible. We explored the energy required for learning in feedforward neural networks. Based on a parsimonious energy model, we propose two plasticity restricting algorithms that save energy: 1) only modify synapses with large updates, and 2) restrict plasticity to subsets of synapses that form a path through the network. In biology networks are often much larger than the task requires, yet vanilla backprop prescribes to update all synapses. In particular in this case, large savings can be achieved while only incurring a slightly worse learning time. Thus competitively restricting plasticity helps to save metabolic energy associated to synaptic plasticity. The results might lead to a better understanding of biological plasticity and a better match between artificial and biological learning. Moreover, the algorithms might benefit hardware because also electronic memory storage is energetically costly.\n                  <\/jats:p>","DOI":"10.1371\/journal.pcbi.1012553","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T13:47:03Z","timestamp":1730123223000},"page":"e1012553","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":4,"title":["Competitive plasticity to reduce the energetic costs of learning"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6525-6814","authenticated-orcid":true,"given":"Mark C. W.","family":"van Rossum","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron","family":"Pache","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,10,28]]},"reference":[{"issue":"10","key":"pcbi.1012553.ref001","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1097\/00004647-200110000-00001","article-title":"An Energy Budget for Signaling in the Grey Matter of the Brain","volume":"21","author":"D Attwell","year":"2001","journal-title":"J of Cerebral Blood Flow & Metabolism"},{"issue":"5","key":"pcbi.1012553.ref002","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1016\/j.neuron.2012.08.019","article-title":"Synaptic energy use and supply","volume":"75","author":"JJ Harris","year":"2012","journal-title":"Neuron"},{"issue":"3","key":"pcbi.1012553.ref003","doi-asserted-by":"crossref","first-page":"531","DOI":"10.1162\/neco.1996.8.3.531","article-title":"Energy efficient neural codes","volume":"8","author":"WB Levy","year":"1996","journal-title":"Neural Computation"},{"issue":"6","key":"pcbi.1012553.ref004","doi-asserted-by":"crossref","first-page":"e1004265","DOI":"10.1371\/journal.pcbi.1004265","article-title":"Energy efficient sparse connectivity from imbalanced synaptic plasticity rules","volume":"11","author":"J Sacramento","year":"2015","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1012553.ref005","unstructured":"Grytskyy D, Jolivet RB. A learning rule balancing energy consumption and information maximization in a feed-forward neuronal network; 2021."},{"issue":"5725","key":"pcbi.1012553.ref006","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1126\/science.1111331","article-title":"A cost of long-term memory in Drosophila","volume":"308","author":"F Mery","year":"2005","journal-title":"Science"},{"issue":"6118","key":"pcbi.1012553.ref007","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1126\/science.1226018","article-title":"To favor survival under food shortage, the brain disables costly memory","volume":"339","author":"PY Pla\u00e7ais","year":"2013","journal-title":"Science"},{"key":"pcbi.1012553.ref008","doi-asserted-by":"crossref","unstructured":"Girard M, Jiang J, van Rossum MCW. Estimating the energy requirements for long term memory formation. arxiv. 2023; p. 2301.09565.","DOI":"10.1101\/2023.01.16.524203"},{"issue":"2","key":"pcbi.1012553.ref009","doi-asserted-by":"crossref","first-page":"e8996","DOI":"10.1371\/journal.pone.0008996","article-title":"Metabolic regulation of neuronal plasticity by the energy sensor AMPK","volume":"5","author":"WB Potter","year":"2010","journal-title":"PloS one"},{"issue":"3","key":"pcbi.1012553.ref010","doi-asserted-by":"crossref","first-page":"770","DOI":"10.1016\/j.neubiorev.2010.09.008","article-title":"Glucose enhancement of human memory: a comprehensive research review of the glucose memory facilitation effect","volume":"35","author":"MA Smith","year":"2011","journal-title":"Neuroscience & Biobehavioral Reviews"},{"key":"pcbi.1012553.ref011","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1038\/s42003-022-03362-4","article-title":"Learning induces coordinated neuronal plasticity of metabolic demands and functional brain networks","volume":"5","author":"S Klug","year":"2022","journal-title":"Communications biology"},{"key":"pcbi.1012553.ref012","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1038\/nature14251","article-title":"Branch-specific dendritic Ca-2+ spikes cause persistent synaptic plasticity","volume":"520","author":"J Cichon","year":"2015","journal-title":"Nature"},{"issue":"5491","key":"pcbi.1012553.ref013","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1126\/science.290.5491.533","article-title":"Learning-induced LTP in neocortex","volume":"290","author":"MS Rioult-Pedotti","year":"2000","journal-title":"Science"},{"issue":"13","key":"pcbi.1012553.ref014","doi-asserted-by":"crossref","first-page":"5121","DOI":"10.1073\/pnas.1120700109","article-title":"Synaptic evidence for the efficacy of spaced learning","volume":"109","author":"EA Kram\u00e1r","year":"2012","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1012553.ref015","unstructured":"Sacramento J, Costa RP, Bengio Y, Senn W. Dendritic cortical microcircuits approximate the backpropagation algorithm. In: Advances in neural information processing systems; 2018. p. 8721\u20138732."},{"key":"pcbi.1012553.ref016","first-page":"2021","article-title":"A rapid and efficient learning rule for biological neural circuits","author":"E Sezener","year":"2021","journal-title":"BioRxiv"},{"key":"pcbi.1012553.ref017","unstructured":"Salvetti A, Wilamowski BM, Dagli C. Introducing Stochastic Process within the Backpropagation Algorithm for Improved Convergence. In: Proc. of Artifiical Neural Network in Engineering. ANNIE; 1994."},{"issue":"3","key":"pcbi.1012553.ref018","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1145\/3007787.3001163","article-title":"EIE: Efficient inference engine on compressed deep neural network","volume":"44","author":"S Han","year":"2016","journal-title":"ACM SIGARCH Computer Architecture News"},{"key":"pcbi.1012553.ref019","unstructured":"Golub M, Lemieux G, Lis M. Full deep neural network training on a pruned weight budget. arXiv preprint arXiv:180606949. 2018;."},{"issue":"4","key":"pcbi.1012553.ref020","doi-asserted-by":"crossref","first-page":"1473","DOI":"10.1152\/jn.00092.2019","article-title":"Metabolic constraints on synaptic learning and memory","volume":"122","author":"J Karbowski","year":"2019","journal-title":"J of neurophysiology"},{"issue":"7173","key":"pcbi.1012553.ref021","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1038\/nature06416","article-title":"Locally dynamic synaptic learning rules in pyramidal neuron dendrites","volume":"450","author":"CD Harvey","year":"2007","journal-title":"Nature"},{"key":"pcbi.1012553.ref022","doi-asserted-by":"crossref","first-page":"e39151","DOI":"10.7554\/eLife.39151","article-title":"The fate of hippocampal synapses depends on the sequence of plasticity-inducing events","volume":"7","author":"JS Wiegert","year":"2018","journal-title":"Elife"},{"issue":"6616","key":"pcbi.1012553.ref023","doi-asserted-by":"crossref","first-page":"533","DOI":"10.1038\/385533a0","article-title":"Synaptic tagging and long-term potentiation","volume":"385","author":"U Frey","year":"1997","journal-title":"Nature"},{"issue":"1","key":"pcbi.1012553.ref024","doi-asserted-by":"crossref","first-page":"e1000259","DOI":"10.1371\/journal.pcbi.1000259","article-title":"State based model of long-term potentiation and synaptic tagging and capture","volume":"5","author":"AB Barrett","year":"2009","journal-title":"PLoS Comput Biol"},{"issue":"1","key":"pcbi.1012553.ref025","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1038\/nrn2963","article-title":"Making memories last: the synaptic tagging and capture hypothesis","volume":"12","author":"RL Redondo","year":"2011","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1012553.ref026","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1016\/S0896-6273(04)00713-5","article-title":"Competing for Memory Hippocampal LTP under Regimes of Reduced Protein Synthesis","volume":"44","author":"R Fonseca","year":"2004","journal-title":"Neuron"},{"issue":"33","key":"pcbi.1012553.ref027","doi-asserted-by":"crossref","first-page":"12217","DOI":"10.1073\/pnas.1403643111","article-title":"Competition between recently potentiated synaptic inputs reveals a winner-take-all phase of synaptic tagging and capture","volume":"111","author":"S Sajikumar","year":"2014","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1012553.ref028","doi-asserted-by":"crossref","first-page":"3915","DOI":"10.1038\/s41467-021-24269-4","article-title":"Synaptic plasticity-dependent competition rule influences memory formation","volume":"12","author":"Y Jeong","year":"2021","journal-title":"Nature communications"},{"key":"pcbi.1012553.ref029","doi-asserted-by":"crossref","first-page":"e37836","DOI":"10.7554\/eLife.37836","article-title":"Competition for synaptic building blocks shapes synaptic plasticity","volume":"7","author":"J Triesch","year":"2018","journal-title":"eLife"},{"key":"pcbi.1012553.ref030","doi-asserted-by":"crossref","first-page":"4732","DOI":"10.1073\/pnas.95.8.4732","article-title":"All-or-none potentiation at CA3-CA1 synapses","volume":"95","author":"CCH Petersen","year":"1998","journal-title":"Proc Natl Acad Sci"},{"key":"pcbi.1012553.ref031","doi-asserted-by":"crossref","first-page":"9626","DOI":"10.1523\/JNEUROSCI.22-21-09626.2002","article-title":"Induction and Experience-Dependent Consolidation of Stable Long-Term Potentiation Lasting Months in the Hippocampus","volume":"22","author":"WC Abraham","year":"2002","journal-title":"J Neurosci"},{"key":"pcbi.1012553.ref032","article-title":"Wide neural networks of any depth evolve as linear models under gradient descent","volume":"32","author":"J Lee","year":"2019","journal-title":"Advances in neural information processing systems"},{"key":"pcbi.1012553.ref033","doi-asserted-by":"crossref","unstructured":"Schmidt WF, Kraaijveld MA, Duin RP, et al. Feed forward neural networks with random weights. In: International conference on pattern recognition. IEEE Computer Society Press; 1992. p. 1\u20131.","DOI":"10.1109\/ICPR.1992.201708"},{"key":"pcbi.1012553.ref034","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme learning machine: Theory and applications","volume":"70","author":"G Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"pcbi.1012553.ref035","doi-asserted-by":"crossref","first-page":"427","DOI":"10.2307\/1934352","article-title":"Diversity and Evenness: A Unifying Notation and Its Consequences","volume":"54","author":"MO Hill","year":"1973","journal-title":"Ecology"},{"key":"pcbi.1012553.ref036","doi-asserted-by":"crossref","first-page":"e50804","DOI":"10.7554\/eLife.50804","article-title":"Energy efficient synaptic plasticity","volume":"9","author":"H Li","year":"2020","journal-title":"Elife"},{"key":"pcbi.1012553.ref037","first-page":"3483","article-title":"Anatomy and Physiology of Macaque Visual Cortical Areas V1, V2, and V5\/MT: Bases for Biologically Realistic Models","volume":"30","author":"S Vanni","year":"2020","journal-title":"Cerebral cortex (New York, NY: 1991)"},{"key":"pcbi.1012553.ref038","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1038\/41267","article-title":"Propagation of activity-dependent synaptic depression in simple neural networks","volume":"388","author":"RM Fitzsimonds","year":"1997","journal-title":"Nature"},{"key":"pcbi.1012553.ref039","doi-asserted-by":"crossref","first-page":"3233","DOI":"10.1523\/JNEUROSCI.20-09-03233.2000","article-title":"Selective presynaptic propagation of long-term potentiation in defined neural networks","volume":"20","author":"H Tao","year":"2000","journal-title":"Journal of neuroscience"},{"key":"pcbi.1012553.ref040","doi-asserted-by":"crossref","first-page":"eabh0146","DOI":"10.1126\/sciadv.abh0146","article-title":"Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks","volume":"7","author":"T Zhang","year":"2021","journal-title":"Science advances"},{"key":"pcbi.1012553.ref041","doi-asserted-by":"crossref","unstructured":"Pache A, van Rossum MCW. Lazy learning: a biologically-inspired plasticity rule for fast and energy efficient synaptic plasticity. eprint. 2023;arXiv:2303.16067.","DOI":"10.32470\/CCN.2023.1379-0"},{"key":"pcbi.1012553.ref042","doi-asserted-by":"crossref","first-page":"102779","DOI":"10.1016\/j.conb.2023.102779","article-title":"Energetically efficient learning in neuronal networks","volume":"83","author":"A Pache","year":"2023","journal-title":"Current Opinion in Neurobiology"},{"key":"pcbi.1012553.ref043","unstructured":"von Oswald J, Zhao D, Kobayashi S, Schug S, Caccia M, Zucchet N, et al. Learning where to learn: Gradient sparsity in meta and continual learning. Arxiv. 2021; p. 2110.14402."}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1012553","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T00:00:00Z","timestamp":1730937600000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012553","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T13:46:48Z","timestamp":1730987208000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1012553"}},"subtitle":[],"editor":[{"given":"Daniel","family":"Bush","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,10,28]]},"references-count":43,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10,28]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1012553","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2023.04.04.535544","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,28]]}}}