{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:55:53Z","timestamp":1787018153101,"version":"build-2736575974"},"reference-count":55,"publisher":"MIT Press","issue":"12","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,11,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Backpropagation has rapidly become the workhorse credit assignment algorithm for modern deep learning methods. Recently, modified forms of predictive coding (PC), an algorithm with origins in computational neuroscience, have been shown to result in approximately or exactly equal parameter updates to those under backpropagation. Due to this connection, it has been suggested that PC can act as an alternative to backpropagation with desirable properties that may facilitate implementation in neuromorphic systems. Here, we explore these claims using the different contemporary PC variants proposed in the literature. We obtain time complexity bounds for these PC variants, which we show are lower bounded by backpropagation. We also present key properties of these variants that have implications for neurobiological plausibility and their interpretations, particularly from the perspective of standard PC as a variational Bayes algorithm for latent probabilistic models. Our findings shed new light on the connection between the two learning frameworks and suggest that in its current forms, PC may have more limited potential as a direct replacement of backpropagation than previously envisioned.<\/jats:p>","DOI":"10.1162\/neco_a_01620","type":"journal-article","created":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T17:31:40Z","timestamp":1697477500000},"page":"1881-1909","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":10,"title":["Predictive Coding as a Neuromorphic Alternative to Backpropagation: A Critical Evaluation"],"prefix":"10.1162","volume":"35","author":[{"given":"Umais","family":"Zahid","sequence":"first","affiliation":[{"name":"Huawei Technologies R&D, London N19 3HT, U.K. umais.zahid@huawei.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qinghai","family":"Guo","sequence":"additional","affiliation":[{"name":"Huawei Technologies R&D, Shenzhen 518129, China guoqinghai@huawei.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zafeirios","family":"Fountas","sequence":"additional","affiliation":[{"name":"Huawei Technologies R&D, London N19 3HT, U.K. zafeirios.fountas@huawei.com"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2023,11,7]]},"reference":[{"key":"2023110721555388900_bib1","article-title":"Deep learning without weight transport","volume-title":"Advances in neural information processing systems","author":"Akrout","year":"2019"},{"key":"2023110721555388900_bib2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2206.00164","author":"Alonso","year":"2022","journal-title":"A theoretical framework for inference learning."},{"key":"2023110721555388900_bib3","doi-asserted-by":"publisher","DOI":"10.3389\/fncom.2019.00018","article-title":"Deep learning with asymmetric connections and Hebbian updates","volume":"13","author":"Amit","year":"2019","journal-title":"Frontiers in Computational Neuroscience"},{"issue":"4","key":"2023110721555388900_bib4","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1016\/j.neuron.2012.10.038","article-title":"Canonical microcircuits for predictive coding","volume":"76","author":"Bastos","year":"2012","journal-title":"Neuron"},{"issue":"3","key":"2023110721555388900_bib5","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1016\/0304-3975(83)90110-X","article-title":"The complexity of partial derivatives","volume":"22","author":"Baur","year":"1983","journal-title":"Theoretical Computer Science"},{"key":"2023110721555388900_bib6","doi-asserted-by":"publisher","first-page":"198","DOI":"10.1016\/j.jmp.2015.11.003","article-title":"A tutorial on the free-energy framework for modelling perception and learning","volume":"76","author":"Bogacz","year":"2017","journal-title":"Journal of Mathematical Psychology"},{"key":"2023110721555388900_bib7","author":"Bradbury","year":"2018","journal-title":"JAX: Composable transformations of Python+NumPy programs."},{"key":"2023110721555388900_bib8","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: A mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"Journal of Mathematical Psychology"},{"issue":"6203","key":"2023110721555388900_bib9","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1038\/337129a0","article-title":"The recent excitement about neural networks","volume":"337","author":"Crick","year":"1989","journal-title":"Nature"},{"key":"2023110721555388900_bib10","doi-asserted-by":"publisher","first-page":"215","DOI":"10.3389\/fnhum.2010.00215","article-title":"Attention, uncertainty, and free-energy","volume":"4","author":"Feldman","year":"2010","journal-title":"Frontiers in Human Neuroscience"},{"issue":"1","key":"2023110721555388900_bib11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/cercor\/1.1.1-a","article-title":"Distributed hierarchical processing in the primate cerebral cortex","volume":"1","author":"Felleman","year":"1991","journal-title":"Cerebral Cortex"},{"key":"2023110721555388900_bib12","volume-title":"Instruction tables: Lists of instruction latencies, throughputs and micro-operation breakdowns for Intel, AMD and VIA CPUs","author":"Fog","year":"2011"},{"issue":"7","key":"2023110721555388900_bib13","doi-asserted-by":"publisher","first-page":"1501","DOI":"10.1162\/neco_a_01514","article-title":"A predictive processing model of episodic memory and time perception","volume":"34","author":"Fountas","year":"2022","journal-title":"Neural Computation"},{"issue":"9","key":"2023110721555388900_bib14","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1016\/j.neunet.2003.06.005","article-title":"Learning and inference in the brain","volume":"16","author":"Friston","year":"2003","journal-title":"Neural Networks"},{"issue":"1456","key":"2023110721555388900_bib15","doi-asserted-by":"publisher","first-page":"815","DOI":"10.1098\/rstb.2005.1622","article-title":"A theory of cortical responses","volume":"360","author":"Friston","year":"2005","journal-title":"Philosophical Transactions of the Royal Society B: Biological Sciences"},{"issue":"11","key":"2023110721555388900_bib16","doi-asserted-by":"publisher","first-page":"e1000211","DOI":"10.1371\/journal.pcbi.1000211","article-title":"Hierarchical models in the brain","volume":"4","author":"Friston","year":"2008","journal-title":"PLOS Computational Biology"},{"issue":"1521","key":"2023110721555388900_bib17","doi-asserted-by":"publisher","first-page":"1211","DOI":"10.1098\/rstb.2008.0300","article-title":"Predictive coding under the free-energy principle","volume":"364","author":"Friston","year":"2009","journal-title":"Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences"},{"key":"2023110721555388900_bib18","article-title":"On automatic differentiation","author":"Griewank","year":"1997"},{"key":"2023110721555388900_bib19","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1007\/978-0-387-74759-0_78","article-title":"Complexity of gradients, Jacobians, and Hessians","author":"Griewank","year":"2009","journal-title":"Encyclopedia of optimization"},{"key":"2023110721555388900_bib20","doi-asserted-by":"crossref","DOI":"10.1137\/1.9780898717761","volume-title":"Evaluating derivatives","author":"Griewank","year":"2008"},{"issue":"1","key":"2023110721555388900_bib21","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/S0364-0213(87)80025-3","article-title":"Competitive learning: From interactive activation to adaptive resonance","volume":"11","author":"Grossberg","year":"1987","journal-title":"Cognitive Science"},{"key":"2023110721555388900_bib22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.15556","author":"Hoffmann","year":"2022","journal-title":"Training compute-optimal large language models."},{"issue":"3","key":"2023110721555388900_bib23","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1016\/j.cognition.2008.05.010","article-title":"Predictive coding explains binocular rivalry: An epistemological review","volume":"108","author":"Hohwy","year":"2008","journal-title":"Cognition"},{"issue":"1668","key":"2023110721555388900_bib24","doi-asserted-by":"publisher","first-page":"20140169","DOI":"10.1098\/rstb.2014.0169","article-title":"Cerebral hierarchies: Predictive processing, precision and the pulvinar","volume":"370","author":"Kanai","year":"2015","journal-title":"Philosophical Transactions of the Royal Society B: Biological Sciences"},{"key":"2023110721555388900_bib25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2001.08361","author":"Kaplan","year":"2020","journal-title":"Scaling laws for neural language models."},{"key":"2023110721555388900_bib26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2208.07114","author":"Kinghorn","year":"2022","journal-title":"Preventing deterioration of classification accuracy in predictive coding networks."},{"key":"2023110721555388900_bib27","doi-asserted-by":"publisher","first-page":"1375","DOI":"10.1109\/ICNN.1994.374486","article-title":"Backpropagation without weight transport","volume-title":"Proceedings of 1994 IEEE International Conference on Neural Networks","author":"Kolen","year":"1994"},{"issue":"1","key":"2023110721555388900_bib28","doi-asserted-by":"publisher","first-page":"13276","DOI":"10.1038\/ncomms13276","article-title":"Random synaptic feedback weights support error backpropagation for deep learning","volume":"7","author":"Lillicrap","year":"2016","journal-title":"Nature Communications"},{"issue":"2","key":"2023110721555388900_bib29","doi-asserted-by":"publisher","first-page":"146","DOI":"10.1007\/BF01931367","article-title":"Taylor expansion of the accumulated rounding error","volume":"16","author":"Linnainmaa","year":"1976","journal-title":"BIT Numerical Mathematics"},{"issue":"1","key":"2023110721555388900_bib30","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1002\/cne.23458","article-title":"Anatomy of hierarchy: Feedforward and feedback pathways in macaque visual cortex","volume":"522","author":"Markov","year":"2014","journal-title":"Journal of Comparative Neurology"},{"key":"2023110721555388900_bib31","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1804.07612","author":"Masters","year":"2018","journal-title":"Revisiting small batch training for deep neural networks."},{"key":"2023110721555388900_bib32","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2202.09467","author":"Millidge","year":"2022","journal-title":"Predictive coding: Towards a future of deep learning beyond backpropagation?"},{"key":"2023110721555388900_bib33","author":"Millidge","year":"2020","journal-title":"Predictive coding approximates backprop along arbitrary computation graphs"},{"key":"2023110721555388900_bib34","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.01047","author":"Millidge","year":"2020","journal-title":"Relaxing the constraints on predictive coding models."},{"key":"2023110721555388900_bib35","author":"Millidge","year":"2020","journal-title":"Activation relaxation: A local dynamical approximation to backpropagation in the brain."},{"key":"2023110721555388900_bib36","author":"Paszke","year":"2017","journal-title":"Automatic differentiation in PyTorch."},{"key":"2023110721555388900_bib37","doi-asserted-by":"publisher","first-page":"146","DOI":"10.3389\/fnsyn.2010.00146","article-title":"Timing is not everything: Neuromodulation opens the STDP gate","volume":"2","author":"Pawlak","year":"2010","journal-title":"Frontiers in Synaptic Neuroscience"},{"issue":"7","key":"2023110721555388900_bib38","doi-asserted-by":"publisher","first-page":"1010","DOI":"10.1038\/s41593-021-00857-x","article-title":"Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits","volume":"24","author":"Payeur","year":"2021","journal-title":"Nature Neuroscience"},{"issue":"1","key":"2023110721555388900_bib39","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nature Neuroscience"},{"issue":"3","key":"2023110721555388900_bib40","doi-asserted-by":"publisher","first-page":"e0266102","DOI":"10.1371\/journal.pone.0266102","article-title":"On the relationship between predictive coding and backpropagation","volume":"17","author":"Rosenbaum","year":"2022","journal-title":"PLOS One"},{"issue":"6088","key":"2023110721555388900_bib41","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back-propagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"Nature"},{"key":"2023110721555388900_bib42","author":"Sacramento","year":"2018","journal-title":"Dendritic cortical microcircuits approximate the backpropagation algorithm."},{"key":"2023110721555388900_bib43","author":"Salvatori","year":"2021","journal-title":"Predictive coding can do exact backpropagation on convolutional and recurrent neural networks."},{"key":"2023110721555388900_bib44","author":"Shallue","year":"2019","journal-title":"Measuring the effects of data parallelism on neural network training."},{"issue":"1","key":"2023110721555388900_bib45","doi-asserted-by":"publisher","first-page":"1805","DOI":"10.1038\/s41467-023-37562-1","article-title":"Meta-learning biologically plausible plasticity rules with random feedback pathways","volume":"14","author":"Shervani-Tabar","year":"2023","journal-title":"Nature Communications"},{"key":"2023110721555388900_bib46","doi-asserted-by":"publisher","first-page":"1792","DOI":"10.3389\/fpsyg.2016.01792","article-title":"Neural elements for predictive coding","volume":"7","author":"Shipp","year":"2016","journal-title":"Frontiers in Psychology"},{"issue":"2","key":"2023110721555388900_bib47","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1016\/j.neuron.2006.06.017","article-title":"A cooperative switch determines the sign of synaptic plasticity in distal dendrites of neocortical pyramidal neurons","volume":"51","author":"Sj\u00f6str\u00f6m","year":"2006","journal-title":"Neuron"},{"key":"2023110721555388900_bib48","first-page":"22566","article-title":"Can the brain do backpropagation? Exact implementation of backpropagation in predictive coding networks","volume-title":"Advances in neural information processing systems","author":"Song","year":"2020"},{"key":"2023110721555388900_bib49","doi-asserted-by":"publisher","DOI":"10.1101\/2022.05.17.492325","author":"Song","year":"2022","journal-title":"Inferring neural activity before plasticity: A foundation for learning beyond backpropagation"},{"issue":"3","key":"2023110721555388900_bib50","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.neuron.2013.11.030","article-title":"Learning by the dendritic prediction of somatic spiking","volume":"81","author":"Urbanczik","year":"2014","journal-title":"Neuron"},{"issue":"5","key":"2023110721555388900_bib51","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1162\/NECO_a_00949","article-title":"An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity","volume":"29","author":"Whittington","year":"2017","journal-title":"Neural Computation"},{"issue":"3","key":"2023110721555388900_bib52","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1016\/j.tics.2018.12.005","article-title":"Theories of error back-propagation in the brain","volume":"23","author":"Whittington","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"key":"2023110721555388900_bib53","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1109\/ISPASS.2010.5452013","article-title":"Demystifying GPU microarchitecture through microbenchmarking","volume-title":"Proceedings of the 2010 IEEE International Symposium on Performance Analysis of Systems and Software","author":"Wong","year":"2010"},{"key":"2023110721555388900_bib54","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.04976","author":"Zahid","year":"2023","journal-title":"Curvature-sensitive predictive coding with approximate Laplace Monte Carlo."},{"key":"2023110721555388900_bib55","first-page":"192","article-title":"The neurobiological significance of the new learning models","volume-title":"Computational neuroscience","author":"Zipser","year":"1993"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/12\/1881\/2168815\/neco_a_01620.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/12\/1881\/2168815\/neco_a_01620.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,7]],"date-time":"2023-11-07T16:56:11Z","timestamp":1699376171000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/12\/1881\/117833\/Predictive-Coding-as-a-Neuromorphic-Alternative-to"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,7]]},"references-count":55,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2023,11,7]]},"published-print":{"date-parts":[[2023,11,7]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01620","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,12]]},"published":{"date-parts":[[2023,11,7]]}}}