{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,27]],"date-time":"2025-08-27T00:14:57Z","timestamp":1756253697095,"version":"3.44.0"},"reference-count":60,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T00:00:00Z","timestamp":1756166400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>From a neuroscience perspective, artificial neural networks are regarded as abstract models of biological neurons, yet they rely on biologically implausible backpropagation for training. Energy-based models represent a class of brain-inspired learning frameworks that adjust system states by minimizing an energy function. Predictive coding (PC), a theoretical model within energy-based models, constructs its energy function from forward prediction errors, with optimization achieved by minimizing local layered errors. Owing to its local plasticity, PC emerges as the most promising alternative to backpropagation. However, PC face gradient explosion and vanishing challenges in deep networks with multiple layers. Gradient explosion occurs when layer-wise prediction errors are excessively large, while gradient vanishing arises when they are excessively small. To address these challenges, we propose bidirectional energy to stabilize prediction errors and mitigate gradient explosion, while using skip connections to resolve gradient vanishing problems. We also introduce a layer-adaptive learning rate (LALR) to enhance training efficiency. Our model achieves accuracies of 99.22% on MNIST, 93.78% on CIFAR-10, 83.96% on CIFAR-100, and 73.35% on Tiny ImageNet, comparable to the performance of identically structed networks trained with backprop. Finally, we developed a Jax-based framework for efficient training of energy-based models, reducing training time by half compared to PyTorch.<\/jats:p>","DOI":"10.3389\/frai.2025.1605706","type":"journal-article","created":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T12:48:21Z","timestamp":1756212501000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Effective methods and framework for energy-based local learning of deep neural networks"],"prefix":"10.3389","volume":"8","author":[{"given":"Haibo","family":"Chen","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bangcheng","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fucun","family":"He","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fei","family":"Zhou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuai","family":"Chen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chunpeng","family":"Wu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fan","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yansong","family":"Chua","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,8,26]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"18","DOI":"10.3389\/fncom.2019.00018","article-title":"Deep learning with asymmetric connections and hebbian updates","volume":"13","author":"Amit","year":"2019","journal-title":"Front. Comput. Neurosci"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1093\/med\/9780197676158.003.0038","article-title":"\u201cFeedforward and feedback connections: functional connectivity, synaptic physiology, and function,\u201d","author":"Angelucci","year":"2023","journal-title":"The Cerebral Cortex and Thalamus"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1407.7906","article-title":"How auto-encoders could provide credit assignment in deep networks via target propagation","author":"Bengio","year":"2014","journal-title":"arXiv"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1510.02777","article-title":"Early inference in energy-based models approximates back-propagation","author":"Bengio","year":"2015","journal-title":"arXiv"},{"key":"B5","doi-asserted-by":"publisher","first-page":"215","DOI":"10.1016\/j.neunet.2023.12.012","article-title":"Beyond multilayer perceptrons: Investigating complex topologies in neural networks","volume":"171","author":"Boccato","year":"2024","journal-title":"Neural Netw"},{"key":"B6","unstructured":"Bradbury\n              J.\n            \n            \n              Frostig\n              R.\n            \n            \n              Hawkins\n              P.\n            \n            \n              Johnson\n              M. J.\n            \n            \n              Leary\n              C.\n            \n            \n              Maclaurin\n              D.\n            \n          \n          JAX: Composable Transformations of Python\n          \n          2018"},{"key":"B7","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: a mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol"},{"key":"B8","doi-asserted-by":"crossref","first-page":"4235","DOI":"10.1109\/ICASSP48485.2024.10447377","article-title":"\u201cUnlocking deep learning: a bp-free approach for parallel block-wise training of neural networks,\u201d","volume-title":"ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP","author":"Cheng","year":"2024"},{"key":"B9","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1017\/S0140525X12000477","article-title":"Whatever next? Predictive brains, situated agents, and the future of cognitive science","volume":"36","author":"Clark","year":"2013","journal-title":"Behav. Brain Sci"},{"key":"B10","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1038\/337129a0","article-title":"The recent excitement about neural networks","volume":"337","author":"Crick","year":"1989","journal-title":"Nature"},{"key":"B11","first-page":"2","article-title":"Lagrangian dynamics of dendritic microcircuits enables real-time backpropagation of errors","volume":"100","author":"Dold","year":"2019","journal-title":"Target"},{"key":"B12","doi-asserted-by":"publisher","first-page":"2121","DOI":"10.5555\/1953048.2021068","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1016\/j.neunet.2003.06.005","article-title":"Learning and inference in the brain","volume":"16","author":"Friston","year":"2003","journal-title":"Neural Netw"},{"key":"B14","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1098\/rstb.2005.1622","article-title":"A theory of cortical responses","volume":"360","author":"Friston","year":"2005","journal-title":"Philos. Trans. R. Soc. B: Biol. Sci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: a unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nat. Rev. Neurosci"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1211","DOI":"10.1098\/rstb.2008.0300","article-title":"Predictive coding under the free-energy principle","volume":"364","author":"Friston","year":"2009","journal-title":"Philos. Trans. R. Soc. B: Biol. Sci"},{"key":"B17","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.jphysparis.2006.10.001","article-title":"A free energy principle for the brain","volume":"100","author":"Friston","year":"2006","journal-title":"J. Physiol"},{"key":"B18","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1007\/s11229-007-9237-y","article-title":"Free-energy and the brain","volume":"159","author":"Friston","year":"2007","journal-title":"Synthese"},{"key":"B19","first-page":"1","article-title":"Compiling machine learning programs via high-level tracing","volume":"4","author":"Frostig","year":"2018","journal-title":"Syst. Mach. Learn"},{"key":"B20","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1037\/h0085812","article-title":"The ups and downs of hebb synapses","volume":"44","author":"Hinton","year":"2003","journal-title":"Can. Psychol.\/Psychol. Canad"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.13345","article-title":"The forward-forward algorithm: some preliminary investigations","author":"Hinton","year":"2022","journal-title":"arXiv"},{"key":"B22","doi-asserted-by":"publisher","first-page":"955","DOI":"10.1109\/TAI.2024.3504479","article-title":"A scalable unsupervised and back propagation free learning with sacsom: a novel approach to SOM-based architectures","volume":"6","author":"Hirani","year":"2024","journal-title":"IEEE Trans. Artif. Intell"},{"key":"B23","doi-asserted-by":"publisher","first-page":"571","DOI":"10.1038\/s42003-024-06264-9","article-title":"From pixels to connections: exploring in vitro neuron reconstruction software for network graph generation","volume":"7","author":"Hoffmann","year":"2024","journal-title":"Commun. Biol"},{"key":"B24","doi-asserted-by":"publisher","first-page":"2554","DOI":"10.1073\/pnas.79.8.2554","article-title":"Neural networks and physical systems with emergent collective computational abilities","volume":"79","author":"Hopfield","year":"1982","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IJCNN.2019.8852098","article-title":"\u201cSelf-organizing neurons: toward brain-inspired unsupervised learning,\u201d","volume-title":"2019 International Joint Conference on Neural Networks (IJCNN)","author":"Khacef","year":"2019"},{"key":"B26","first-page":"1","article-title":"\u201cPreventing deterioration of classification accuracy in predictive coding networks,\u201d","volume-title":"International Workshop on Active Inference","author":"Kinghorn","year":"2022"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6980","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv"},{"key":"B28","doi-asserted-by":"publisher","first-page":"7723","DOI":"10.1073\/pnas.1820458116","article-title":"Unsupervised learning by competing hidden units","volume":"116","author":"Krotov","year":"2019","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1101\/408385","article-title":"Cornet: modeling the neural mechanisms of core object recognition","author":"Kubilius","year":"2018","journal-title":"bioRxiv"},{"key":"B30","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1007\/978-3-319-23528-8_31","article-title":"\u201cDifference target propagation,\u201d","volume-title":"Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part I 15","author":"Lee","year":"2015"},{"key":"B31","doi-asserted-by":"publisher","first-page":"13276","DOI":"10.1038\/ncomms13276","article-title":"Random synaptic feedback weights support error backpropagation for deep learning","volume":"7","author":"Lillicrap","year":"2016","journal-title":"Nat. Commun"},{"key":"B32","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1038\/s41583-020-0277-3","article-title":"Backpropagation and the brain","volume":"21","author":"Lillicrap","year":"2020","journal-title":"Nat. Rev. Neurosci"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1702.07097","article-title":"Adaptive bidirectional backpropagation: towards biologically plausible error signal transmission in neural networks","author":"Luo","year":"2017","journal-title":"arXiv"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2107.12979","article-title":"Predictive coding: a theoretical and experimental review","author":"Millidge","year":"2021","journal-title":"arXiv"},{"key":"B35","unstructured":"\u201cBackpropagation at the infinitesimal inference limit of energy-based models: unifying predictive coding, equilibrium propagation, and contrastive Hebbian learning,\u201d\n          \n          \n            \n              Millidge\n              B.\n            \n            \n              Song\n              Y.\n            \n            \n              Salvatori\n              T.\n            \n            \n              Lukasiewicz\n              T.\n            \n            \n              Bogacz\n              R.\n            \n          \n          Kigali\n          The Eleventh International Conference on Learning Representations\n          \n          2023"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1329","DOI":"10.1162\/neco_a_01497","article-title":"Predictive coding approximates backprop along arbitrary computation graphs","volume":"34","author":"Millidge","year":"2022","journal-title":"Neural Comput"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1297","DOI":"10.1126\/science.adi8474","article-title":"Backpropagation-free training of deep physical neural networks","volume":"382","author":"Momeni","year":"2023","journal-title":"Science"},{"key":"B38","doi-asserted-by":"publisher","first-page":"044017","DOI":"10.1088\/2634-4386\/aca710","article-title":"Softhebb: Bayesian inference in unsupervised hebbian soft winner-take-all networks","volume":"2","author":"Moraitis","year":"2022","journal-title":"Neuromorphic Comput. Eng"},{"key":"B39","unstructured":"\u201cDirect feedback alignment provides learning in deep neural networks,\u201d\n          \n          \n            \n              N\u00f8kland\n              A.\n            \n          \n          Advances in Neural Information Processing Systems\n          \n          2016"},{"key":"B40","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1007\/s11229-023-04292-2","article-title":"Incorporating (variational) free energy models into mechanisms: the case of predictive processing under the free energy principle","volume":"202","author":"Piekarski","year":"2023","journal-title":"Synthese"},{"key":"B41","unstructured":"\u201cPredictive coding beyond Gaussian distributions\u201d\n          \n          \n            \n              Pinchetti\n              L.\n            \n            \n              Salvatori\n              T.\n            \n            \n              Yordanov\n              Y.\n            \n            \n              Millidge\n              B.\n            \n            \n              Song\n              Y.\n            \n            \n              Lukasiewicz\n              T.\n            \n          \n          Advances in Neural Information Processing Systems\n          \n          2022"},{"key":"B42","first-page":"7296","article-title":"Kernelized information bottleneck leads to biologically plausible 3-factor hebbian learning in deep networks","volume":"33","author":"Pogodin","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B43","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1093\/cercor\/bhs358","article-title":"The cell-type specific cortical microcircuit: relating structure and activity in a full-scale spiking network model","volume":"24","author":"Potjans","year":"2014","journal-title":"Cereb. Cortex"},{"key":"B44","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nat. Neurosci"},{"key":"B45","doi-asserted-by":"publisher","first-page":"784310","DOI":"10.3389\/fnsys.2022.784310","article-title":"Notes on visual cortical feedback and feedforward connections","volume":"16","author":"Rockland","year":"2022","journal-title":"Front. Syst. Neurosci"},{"key":"B46","doi-asserted-by":"publisher","first-page":"e0266102","DOI":"10.1371\/journal.pone.0266102","article-title":"On the relationship between predictive coding and backpropagation","volume":"17","author":"Rosenbaum","year":"2022","journal-title":"PLoS ONE"},{"key":"B47","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1007\/s00422-023-00963-y","article-title":"Self-organizing maps on \u201cwhat-where\u201d codes towards fully unsupervised classification","volume":"117","author":"Sa-Couto","year":"2023","journal-title":"Biol. Cybern"},{"key":"B48","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1162\/netn_a_00428","article-title":"Combined topological and spatial constraints are required to capture the structure of neural connectomes","volume":"9","author":"Salova","year":"2025","journal-title":"Netw. Neurosci"},{"key":"B49","doi-asserted-by":"publisher","first-page":"24","DOI":"10.3389\/fncom.2017.00024","article-title":"Equilibrium propagation: bridging the gap between energy-based models and backpropagation","volume":"11","author":"Scellier","year":"2017","journal-title":"Front. Comput. Neurosci"},{"key":"B50","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1016\/j.bandc.2015.11.003","article-title":"A review of predictive coding algorithms","volume":"112","author":"Spratling","year":"2017","journal-title":"Brain Cogn"},{"key":"B51","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1109\/IJCNN.1989.118705","article-title":"\u201cIs backpropagation biologically plausible,\u201d","volume-title":"International Joint Conference on Neural Networks, Volume 2","author":"Stork","year":"1989"},{"key":"B52","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1162\/NECO_a_00949","article-title":"An approximation of the error backpropagation algorithm in a predictive coding network with local hebbian synaptic plasticity","volume":"29","author":"Whittington","year":"2017","journal-title":"Neural Comput"},{"key":"B53","doi-asserted-by":"publisher","first-page":"22964","DOI":"10.1038\/s41598-021-02203-4","article-title":"Deciphering the generating rules and functionalities of complex networks","volume":"11","author":"Xiao","year":"2021","journal-title":"Sci. Rep"},{"key":"B54","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1162\/089976603762552988","article-title":"Equivalence of backpropagation and contrastive hebbian learning in a layered network","volume":"15","author":"Xie","year":"2003","journal-title":"Neural Comput"},{"key":"B55","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1038\/s42005-021-00701-5","article-title":"Hidden network generating rules from partially observed complex networks","volume":"4","author":"Yang","year":"2021","journal-title":"Commun. Phys"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2310.07885","article-title":"Leader-follower neural networks with local error signals inspired by complex collectives","author":"Yin","year":"2023","journal-title":"arXiv"},{"key":"B57","doi-asserted-by":"publisher","first-page":"15078","DOI":"10.1038\/s41598-020-72013-7","article-title":"Network science characteristics of brain-derived neuronal cultures deciphered from quantitative phase imaging data","volume":"10","author":"Yin","year":"2020","journal-title":"Sci. Rep"},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1708.03888","article-title":"Large batch training of convolutional networks","author":"You","year":"2017","journal-title":"arXiv"},{"key":"B59","unstructured":"\u201cLarge batch optimization for deep learning: training BERT in 76 minutes,\u201d\n          \n          \n            \n              You\n              Y.\n            \n            \n              Li\n              J.\n            \n            \n              Reddi\n              S.\n            \n            \n              Hseu\n              J.\n            \n            \n              Kumar\n              S.\n            \n            \n              Bhojanapalli\n              S.\n            \n          \n          Addis Ababa\n          International Conference on Learning Representations\n          \n          2020"},{"key":"B60","doi-asserted-by":"publisher","first-page":"17948","DOI":"10.1038\/s41598-023-44791-3","article-title":"A unified approach of detecting phase transition in time-varying complex networks","volume":"13","author":"Znaidi","year":"2023","journal-title":"Sci. Rep"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1605706\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,26]],"date-time":"2025-08-26T12:48:25Z","timestamp":1756212505000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1605706\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,26]]},"references-count":60,"alternative-id":["10.3389\/frai.2025.1605706"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1605706","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,26]]},"article-number":"1605706"}}