{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T21:52:52Z","timestamp":1773957172763,"version":"3.50.1"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1011839","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000}}],"reference-count":137,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T00:00:00Z","timestamp":1708387200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["200020_184615"],"award-info":[{"award-number":["200020_184615"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001711","name":"Schweizerischer Nationalfonds zur F\u00f6rderung der Wissenschaftlichen Forschung","doi-asserted-by":"publisher","award":["200020_207426"],"award-info":[{"award-number":["200020_207426"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>In humans and animals, surprise is a physiological reaction to an unexpected event, but how surprise can be linked to plausible models of neuronal activity is an open problem. We propose a self-supervised spiking neural network model where a surprise signal is extracted from an increase in neural activity after an imbalance of excitation and inhibition. The surprise signal modulates synaptic plasticity via a three-factor learning rule which increases plasticity at moments of surprise. The surprise signal remains small when transitions between sensory events follow a previously learned rule but increases immediately after rule switching. In a spiking network with several modules, previously learned rules are protected against overwriting, as long as the number of modules is larger than the total number of rules\u2014making a step towards solving the stability-plasticity dilemma in neuroscience. Our model relates the subjective notion of surprise to specific predictions on the circuit level.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1011839","type":"journal-article","created":{"date-parts":[[2024,2,20]],"date-time":"2024-02-20T13:27:40Z","timestamp":1708435660000},"page":"e1011839","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":13,"title":["Fast adaptation to rule switching using neuronal surprise"],"prefix":"10.1371","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4189-5026","authenticated-orcid":true,"given":"Martin L. L. R.","family":"Barry","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4344-2189","authenticated-orcid":true,"given":"Wulfram","family":"Gerstner","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"340","published-online":{"date-parts":[[2024,2,20]]},"reference":[{"key":"pcbi.1011839.ref001","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1126\/science.959831","article-title":"The effect of stimulus sequence on the waveform of the cortical event-related potential","volume":"193","author":"KC Squires","year":"1976","journal-title":"Science"},{"issue":"4","key":"pcbi.1011839.ref002","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1080\/02699939108411042","article-title":"An experimental analysis of surprise","volume":"5","author":"WU Meyer","year":"1991","journal-title":"Cognition & Emotion"},{"key":"pcbi.1011839.ref003","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/9027.001.0001","volume-title":"Inside jokes: Using humor to reverse-engineer the mind","author":"MM Hurley","year":"2011"},{"key":"pcbi.1011839.ref004","doi-asserted-by":"crossref","first-page":"102712","DOI":"10.1016\/j.jmp.2022.102712","article-title":"A taxonomy of surprise definitions","volume":"110","author":"A Modirshanechi","year":"2022","journal-title":"J Mathem Psychol"},{"key":"pcbi.1011839.ref005","volume-title":"Auditory Neuroscience: Making Sense of Sound","author":"J Schnupp","year":"2011"},{"key":"pcbi.1011839.ref006","volume-title":"Sweet anticipation: music and the psychology of expectation","author":"D Huron","year":"2008"},{"key":"pcbi.1011839.ref007","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1038\/nrn3666","article-title":"Brain correlates of music-evoked emotions","volume":"15","author":"S Koelsch","year":"2014","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011839.ref008","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/S0926-6410(02)00137-4","article-title":"Functional magnetic resonance imaging of brain activity in the visual oddball task","volume":"14","author":"BA Ardekani","year":"2002","journal-title":"Cognitive Brain Research"},{"issue":"49","key":"pcbi.1011839.ref009","doi-asserted-by":"crossref","first-page":"16601","DOI":"10.1523\/JNEUROSCI.2770-10.2010","article-title":"Expectation and surprise determine neural population responses in the ventral visual stream","volume":"30","author":"T Egner","year":"2010","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1011839.ref010","doi-asserted-by":"crossref","first-page":"e2108882119","DOI":"10.1073\/pnas.2108882119","article-title":"Novel stimuli evoke excess activity in the mouse primary visual cortex","volume":"119","author":"J Homann","year":"2022","journal-title":"Proc Natl Acad Sci (USA)"},{"key":"pcbi.1011839.ref011","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/(SICI)1098-1063(2000)10:1<1::AID-HIPO1>3.0.CO;2-1","article-title":"Models of hippocampally dependent navigation using the temporal difference learning rule","volume":"10","author":"D Foster","year":"2000","journal-title":"Hippocampus"},{"key":"pcbi.1011839.ref012","doi-asserted-by":"crossref","first-page":"2745","DOI":"10.1523\/JNEUROSCI.5238-05.2006","article-title":"Prefrontal cell activities related to monkeys\u2019 success and failure in adapting to rule changes in a Wisconsin card sorting test analog","volume":"26","author":"FA Mansouri","year":"2002","journal-title":"J Neurosci"},{"key":"pcbi.1011839.ref013","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1016\/j.neuron.2007.03.017","article-title":"A neural circuit model of flexible sensorimotor mapping: learning and forgetting on multiple timescales","volume":"54","author":"S Fusi","year":"2007","journal-title":"Neuron"},{"key":"pcbi.1011839.ref014","doi-asserted-by":"crossref","first-page":"e20552","DOI":"10.7554\/eLife.20552","article-title":"Activity patterns of serotonin neurons underlying cognitive flexibility","volume":"6","author":"S Matias","year":"2017","journal-title":"eLife"},{"issue":"12","key":"pcbi.1011839.ref015","doi-asserted-by":"crossref","first-page":"1753","DOI":"10.1038\/s41593-018-0269-z","article-title":"Thalamic regulation of switching between cortical representations enables cognitive flexibility","volume":"21","author":"RV Rikhye","year":"2018","journal-title":"Nature Neuroscience"},{"issue":"6","key":"pcbi.1011839.ref016","doi-asserted-by":"crossref","first-page":"e1009070","DOI":"10.1371\/journal.pcbi.1009070","article-title":"Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making","volume":"17","author":"HA Xu","year":"2021","journal-title":"PLOS Computational Biology"},{"key":"pcbi.1011839.ref017","doi-asserted-by":"crossref","first-page":"907","DOI":"10.3389\/fpsyg.2013.00907","article-title":"Novelty or surprise?","volume":"4","author":"A Barto","year":"2013","journal-title":"Frontiers in Psychology"},{"key":"pcbi.1011839.ref018","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/978-1-4899-2286-1_10","volume-title":"Neural mechanisms of startle behavior","author":"M Davis","year":"1984"},{"key":"pcbi.1011839.ref019","doi-asserted-by":"crossref","first-page":"218","DOI":"10.3389\/fnbeh.2013.00218","article-title":"Pupil dilation signals uncertainty and surprise in a learning gambling task","volume":"7","author":"C Lav\u00edn","year":"2014","journal-title":"Frontiers in Behavioral Neuroscience"},{"key":"pcbi.1011839.ref020","doi-asserted-by":"crossref","first-page":"115","DOI":"10.3389\/fnins.2011.00115","article-title":"Pupil dilation signals surprise: Evidence for noradrenaline\u2019s role in decision making","volume":"5","author":"K Preuschoff","year":"2011","journal-title":"Frontiers in Neuroscience"},{"key":"pcbi.1011839.ref021","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.neuropsychologia.2016.02.003","article-title":"The contribution of surprise to the prediction based modulation of fMRI responses","volume":"84","author":"C Amado","year":"2016","journal-title":"Neuropsychologia"},{"key":"pcbi.1011839.ref022","doi-asserted-by":"crossref","first-page":"e41541","DOI":"10.7554\/eLife.41541","article-title":"Brain signatures of a multiscale process of sequence learning in humans","volume":"8","author":"M Maheu","year":"2019","journal-title":"eLife"},{"key":"pcbi.1011839.ref023","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1146\/annurev.neuro.28.061604.135709","article-title":"An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance","volume":"28","author":"G Aston-Jones","year":"2005","journal-title":"Annu Rev Neurosci"},{"issue":"2","key":"pcbi.1011839.ref024","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.neuron.2018.10.003","article-title":"Predictive processing: a canonical cortical computation","volume":"100","author":"GB Keller","year":"2018","journal-title":"Neuron"},{"issue":"2","key":"pcbi.1011839.ref025","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.neuron.2020.10.029","article-title":"Behavioral, physiological, and neural signatures of surprise during naturalistic sports viewing","volume":"109","author":"JW Antony","year":"2021","journal-title":"Neuron"},{"key":"pcbi.1011839.ref026","volume-title":"Advances in Neural Information Processing Systems","author":"N Chentanez","year":"2004"},{"issue":"3","key":"pcbi.1011839.ref027","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1038\/nrn1052","article-title":"Neural mechanisms for detecting and remembering novel events","volume":"4","author":"C Ranganath","year":"2003","journal-title":"Nature Reviews Neuroscience"},{"issue":"1-2","key":"pcbi.1011839.ref028","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1385\/JMN:30:1:133","article-title":"Cholinergic modulation of cortical function","volume":"30","author":"M Hasselmo","year":"2006","journal-title":"Journal of Molecular Neuroscience"},{"issue":"4","key":"pcbi.1011839.ref029","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1523\/JNEUROSCI.2068-12.2013","article-title":"Covariance-based synaptic plasticity in an attractor network model accounts for fast adaptation in free operant learning","volume":"33","author":"T Neiman","year":"2013","journal-title":"Journal of Neuroscience"},{"issue":"8","key":"pcbi.1011839.ref030","doi-asserted-by":"crossref","first-page":"1075","DOI":"10.1016\/j.neunet.2006.05.044","article-title":"Neural mechanism for stochastic behaviour during a competitive game","volume":"19","author":"A Soltani","year":"2006","journal-title":"Neural Networks"},{"issue":"7","key":"pcbi.1011839.ref031","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1038\/nn.3130","article-title":"Rational regulation of learning dynamics by pupil-linked arousal systems","volume":"15","author":"MR Nassar","year":"2012","journal-title":"Nature Neuroscience"},{"issue":"9","key":"pcbi.1011839.ref032","doi-asserted-by":"crossref","first-page":"1214","DOI":"10.1038\/nn1954","article-title":"Learning the value of information in an uncertain world","volume":"10","author":"TE Behrens","year":"2007","journal-title":"Nature Neuroscience"},{"key":"pcbi.1011839.ref033","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1016\/j.neuron.2005.04.026","article-title":"Uncertainty, neuromodulation, and attention","volume":"46","author":"A Yu","year":"2005","journal-title":"Neuron"},{"key":"pcbi.1011839.ref034","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.nlm.2014.03.006","article-title":"Locus coeruleus neuromodulation of memories encoded during negative or unexpected action outcomes","volume":"111","author":"D Clewett","year":"2014","journal-title":"Neurobiology of Learning and Memory"},{"key":"pcbi.1011839.ref035","doi-asserted-by":"crossref","first-page":"e23763","DOI":"10.7554\/eLife.23763","article-title":"The computational nature of memory modification","volume":"6","author":"SJ Gershman","year":"2017","journal-title":"Elife"},{"issue":"3","key":"pcbi.1011839.ref036","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1037\/0033-295X.96.3.433","article-title":"Visual search and stimulus similarity","volume":"96","author":"J Duncan","year":"1989","journal-title":"Psychological Review"},{"key":"pcbi.1011839.ref037","doi-asserted-by":"crossref","unstructured":"Schmidhuber J. Curious model-building control systems. In: Proceedings of the International Joint Conference on Neural Networks, Singapore. vol. 2. IEEE press; 1991. p. 1458\u20131463.","DOI":"10.1109\/IJCNN.1991.170605"},{"key":"pcbi.1011839.ref038","doi-asserted-by":"crossref","unstructured":"Itti L, Baldi P. A principled approach to detecting surprising events in video. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905). vol. 1. IEEE; 2005. p. 631\u2013637.","DOI":"10.1109\/CVPR.2005.40"},{"issue":"1","key":"pcbi.1011839.ref039","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1162\/neco_a_01025","article-title":"Balancing new against old information: the role of puzzlement surprise in learning","volume":"30","author":"M Faraji","year":"2018","journal-title":"Neural Computation"},{"issue":"2","key":"pcbi.1011839.ref040","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: a unified brain theory?","volume":"11","author":"K Friston","year":"2010","journal-title":"Nature Reviews Neuroscience"},{"issue":"3","key":"pcbi.1011839.ref041","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1109\/TAMD.2010.2056368","article-title":"Formal theory of creativity, fun, and intrinsic motivation (1990\u20132010)","volume":"2","author":"J Schmidhuber","year":"2010","journal-title":"IEEE Transactions on Autonomous Mental Development"},{"key":"pcbi.1011839.ref042","unstructured":"Adams RP, MacKay DJ. Bayesian online changepoint detection. arXiv preprint arXiv:07103742. 2007;."},{"issue":"7","key":"pcbi.1011839.ref043","doi-asserted-by":"crossref","first-page":"1706","DOI":"10.1162\/neco.2008.10-06-351","article-title":"Online learning with hidden Markov models","volume":"20","author":"G Mongillo","year":"2008","journal-title":"Neural computation"},{"issue":"7626","key":"pcbi.1011839.ref044","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1038\/nature20101","article-title":"Hybrid computing using a neural network with dynamic external memory","volume":"538","author":"A Graves","year":"2016","journal-title":"Nature"},{"key":"pcbi.1011839.ref045","unstructured":"Ramalho T, Garnelo M. Adaptive posterior learning: few-shot learning with a surprise-based memory module. ICLR 2019 and arXiv preprint arXiv:190202527. 2019;."},{"issue":"2","key":"pcbi.1011839.ref046","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1162\/neco_a_01352","article-title":"Learning in volatile environments with the Bayes factor surprise","volume":"33","author":"V Liakoni","year":"2021","journal-title":"Neural Computation"},{"key":"pcbi.1011839.ref047","unstructured":"Rezende DJ, Mohamed S, Wierstra D. Stochastic backpropagation and approximate inference in deep generative models. In: International conference on machine learning. PMLR; 2014. p. 1278\u20131286."},{"issue":"23","key":"pcbi.1011839.ref048","doi-asserted-by":"crossref","first-page":"9565","DOI":"10.1523\/JNEUROSCI.4098-12.2013","article-title":"Matching recall and storage in sequence learning with spiking neural networks","volume":"33","author":"J Brea","year":"2013","journal-title":"Journal of neuroscience"},{"key":"pcbi.1011839.ref049","first-page":"8721","volume-title":"Advances in neural information processing systems","author":"J Sacramento","year":"2018"},{"key":"pcbi.1011839.ref050","doi-asserted-by":"crossref","first-page":"146","DOI":"10.3389\/fnsyn.2010.00146","article-title":"Timing is not everything: neuromodulation opens the STDP gate","volume":"2","author":"V Pawlak","year":"2010","journal-title":"Front Synaptic Neurosci"},{"key":"pcbi.1011839.ref051","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1016\/j.tins.2011.07.006","article-title":"A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP","volume":"34","author":"J Lisman","year":"2011","journal-title":"Trends Neurosci"},{"key":"pcbi.1011839.ref052","doi-asserted-by":"crossref","first-page":"85","DOI":"10.3389\/fncir.2015.00085","article-title":"Neuromodulated spike-timing-dependent plasticity, and theory of three-factor learning rules","volume":"9","author":"N Fr\u00e9maux","year":"2016","journal-title":"Frontiers in Neural Circuits"},{"key":"pcbi.1011839.ref053","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.conb.2017.08.020","article-title":"Learning with three factors: modulating Hebbian plasticity with errors","volume":"46","author":"L Kusmierz","year":"2017","journal-title":"Curr Opin Neurobiol"},{"key":"pcbi.1011839.ref054","doi-asserted-by":"crossref","first-page":"53","DOI":"10.3389\/fncir.2018.00053","article-title":"Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules","volume":"12","author":"W Gerstner","year":"2018","journal-title":"Frontiers in Neural Circuits"},{"issue":"3","key":"pcbi.1011839.ref055","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1038\/nrn.2018.6","article-title":"Control of synaptic plasticity in deep cortical networks","volume":"19","author":"PR Roelfsema","year":"2018","journal-title":"Nature Reviews Neuroscience"},{"key":"pcbi.1011839.ref056","volume-title":"Deep Learning","author":"I Goodfellow","year":"2016"},{"key":"pcbi.1011839.ref057","doi-asserted-by":"crossref","first-page":"13276","DOI":"10.1038\/ncomms13276","article-title":"Random synaptic feedback weights support error backpropagation for deep learning","volume":"7","author":"TP Lillicrap","year":"2016","journal-title":"Nature Communications"},{"key":"pcbi.1011839.ref058","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1109\/MSP.2019.2931595","article-title":"Surrogate gradient learning in spiking neural networks","volume":"36","author":"EO Neftci","year":"2019","journal-title":"IEEE Signal Processing Magazine"},{"issue":"1","key":"pcbi.1011839.ref059","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-020-17236-y","article-title":"A solution to the learning dilemma for recurrent networks of spiking neurons","volume":"11","author":"G Bellec","year":"2020","journal-title":"Nature Communications"},{"key":"pcbi.1011839.ref060","first-page":"2516","volume-title":"Adv. Neur. Inform. Proc. Systems 33 (NeurIPS 2020)","author":"I Pozzi","year":"2020"},{"issue":"8","key":"pcbi.1011839.ref061","doi-asserted-by":"crossref","first-page":"7467","DOI":"10.4249\/scholarpedia.7467","article-title":"Balance of excitation and inhibition","volume":"4","author":"M Okun","year":"2009","journal-title":"Scholarpedia"},{"key":"pcbi.1011839.ref062","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1146\/annurev.neuro.28.061604.135637","article-title":"Neural network dynamics","volume":"28","author":"TP Vogels","year":"2005","journal-title":"Annu Rev Neurosci"},{"key":"pcbi.1011839.ref063","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1126\/science.1211095","article-title":"Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks","volume":"334","author":"TP Vogels","year":"2011","journal-title":"Science"},{"issue":"1","key":"pcbi.1011839.ref064","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"RP Rao","year":"1999","journal-title":"Nature Neuroscience"},{"issue":"1456","key":"pcbi.1011839.ref065","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1098\/rstb.2005.1622","article-title":"A theory of cortical responses","volume":"360","author":"K Friston","year":"2005","journal-title":"Philosophical Transactions of the Royal Society B: Biological sciences"},{"issue":"12","key":"pcbi.1011839.ref066","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1016\/j.visres.2008.03.009","article-title":"Predictive coding as a model of biased competition in visual attention","volume":"48","author":"MW Spratling","year":"2008","journal-title":"Vision Research"},{"key":"pcbi.1011839.ref067","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.conb.2017.08.010","article-title":"With or without you: predictive coding and Bayesian inference in the brain","volume":"46","author":"L Aitchison","year":"2017","journal-title":"Current Opinion Neurobiol"},{"key":"pcbi.1011839.ref068","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1016\/j.tins.2019.07.005","article-title":"Towards a unified view on pathways and functions of neural recurrent processing","volume":"42","author":"CMA Pennartz","year":"2019","journal-title":"Trends Neursci"},{"key":"pcbi.1011839.ref069","doi-asserted-by":"crossref","first-page":"e57541","DOI":"10.7554\/eLife.57541","article-title":"Learning prediction error neurons in a canonical inteneuron circuit","volume":"9","author":"L Hertag","year":"2020","journal-title":"eLife"},{"key":"pcbi.1011839.ref070","article-title":"Associative memories via predictive coding","volume":"34","author":"T Salvatori","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pcbi.1011839.ref071","unstructured":"Lotter W, Kreiman G, Cox D. Deep predictive coding networks for video prediction and unsupervised learning. ICLR 2017 and arXiv preprint arXiv:160508104. 2016;."},{"key":"pcbi.1011839.ref072","unstructured":"Oord Avd, Li Y, Vinyals O. Representation learning with contrastive predictive coding. arXiv preprint arXiv:180703748. 2018;."},{"key":"pcbi.1011839.ref073","article-title":"Local plasticity rules can learn deep representations using self-supervised contrastive predictions","volume":"34","author":"B Illing","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pcbi.1011839.ref074","unstructured":"Salvatori T, Song Y, Lukasiewicz T, Bogacz R, Xu Z. Predictive coding can do exact backpropagation on convolutional and recurrent neural networks. arXiv preprint arXiv:210303725. 2021;."},{"key":"pcbi.1011839.ref075","first-page":"217","volume-title":"Sensory Communication","author":"HB Barlow","year":"1961"},{"key":"pcbi.1011839.ref076","unstructured":"van de Ven G, Tolias AS. Three scenarios for continual learning. arXiv. 2019;(arxiv):1904.07734."},{"key":"pcbi.1011839.ref077","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1109\/2.33","article-title":"The ART of adpative pattern recognition by a self-organizing neural network","volume":"21","author":"G Carpenter","year":"1988","journal-title":"Computer"},{"key":"pcbi.1011839.ref078","doi-asserted-by":"crossref","first-page":"599","DOI":"10.1016\/j.neuron.2005.02.001","article-title":"Cascade models of synaptically stored memories","volume":"45","author":"S Fusi","year":"2005","journal-title":"Neuron"},{"key":"pcbi.1011839.ref079","unstructured":"Zenke F, Poole B, Ganguli S. Continual learning through synaptic intelligence. In: International Conference on Machine Learning. PMLR; 2017. p. 3987\u20133995."},{"issue":"13","key":"pcbi.1011839.ref080","doi-asserted-by":"crossref","first-page":"3521","DOI":"10.1073\/pnas.1611835114","article-title":"Overcoming catastrophic forgetting in neural networks","volume":"114","author":"J Kirkpatrick","year":"2017","journal-title":"Proc Natl Academy Sci (USA)"},{"issue":"7889","key":"pcbi.1011839.ref081","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1038\/s41586-021-04129-3","article-title":"Contextual inference underlies the learning of sensorimotor repertoires","volume":"600","author":"JB Heald","year":"2021","journal-title":"Nature"},{"issue":"7-8","key":"pcbi.1011839.ref082","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1016\/S0893-6080(98)00066-5","article-title":"Multiple paired forward and inverse models for motor control","volume":"11","author":"DM Wolpert","year":"1998","journal-title":"Neural networks"},{"issue":"46","key":"pcbi.1011839.ref083","doi-asserted-by":"crossref","first-page":"9237","DOI":"10.1523\/JNEUROSCI.3250-18.2019","article-title":"Minimizing precision-weighted sensory prediction errors via memory formation and switching in motor adaptation","volume":"39","author":"Y Oh","year":"2019","journal-title":"Journal of Neuroscience"},{"issue":"6","key":"pcbi.1011839.ref084","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pcbi.1007935","article-title":"Brain dynamics for confidence-weighted learning","volume":"16","author":"F Meyniel","year":"2020","journal-title":"PLOS Computational Biology"},{"key":"pcbi.1011839.ref085","doi-asserted-by":"crossref","first-page":"4030","DOI":"10.1038\/s41467-019-12048-1","article-title":"Pupil-linked phasic arousal evoked by violation but not emergence of regularity within rapid sound sequences","volume":"10","author":"S Zhao","year":"2019","journal-title":"Nature Comm"},{"issue":"2","key":"pcbi.1011839.ref086","doi-asserted-by":"crossref","first-page":"e1008068","DOI":"10.1371\/journal.pcbi.1008068","article-title":"Neural surprise in somatosensory Bayesian learning","volume":"17","author":"S Gijsen","year":"2021","journal-title":"PLoS Computational Biology"},{"issue":"12","key":"pcbi.1011839.ref087","doi-asserted-by":"crossref","first-page":"e1005260","DOI":"10.1371\/journal.pcbi.1005260","article-title":"Human inferences about sequences: A minimal transition probability model","volume":"12","author":"F Meyniel","year":"2016","journal-title":"PLoS Computational Biology"},{"issue":"1","key":"pcbi.1011839.ref088","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1146\/annurev.ps.40.020189.001203","article-title":"Brain dopamine and reward","volume":"40","author":"RA Wise","year":"1989","journal-title":"Annual Review of Psychology"},{"key":"pcbi.1011839.ref089","doi-asserted-by":"crossref","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate for prediction and reward","volume":"275","author":"W Schultz","year":"1997","journal-title":"Science"},{"key":"pcbi.1011839.ref090","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","edition":"2"},{"issue":"12","key":"pcbi.1011839.ref091","doi-asserted-by":"crossref","first-page":"758","DOI":"10.1038\/s41583-018-0078-0","article-title":"Towards a neuroscience of active sampling and curiosity","volume":"19","author":"J Gottlieb","year":"2018","journal-title":"Nature Reviews Neuroscience"},{"issue":"12","key":"pcbi.1011839.ref092","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1038\/nn.4385","article-title":"Experience-dependent spatial expectations in mouse visual cortex","volume":"19","author":"A Fiser","year":"2016","journal-title":"Nature Neuroscience"},{"issue":"6","key":"pcbi.1011839.ref093","doi-asserted-by":"crossref","first-page":"1016","DOI":"10.1016\/j.neuron.2020.12.005","article-title":"Defying expectations: How neurons compute prediction errors in visual cortex","volume":"108","author":"Z Padamsey","year":"2020","journal-title":"Neuron"},{"key":"pcbi.1011839.ref094","first-page":"158","volume-title":"Molecular Neuropharmacology: A Foundation for Clinical Neuroscience","author":"EJ Nestler","year":"2020","edition":"4"},{"key":"pcbi.1011839.ref095","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1038\/nrn.2016.165","article-title":"Ventral tegmental area: cellular heterogeneity, connectivity and behaviour","volume":"18","author":"M Morales","year":"2017","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011839.ref096","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1016\/j.tics.2020.07.006","article-title":"Cellular mechanisms of conscious processing","volume":"24","author":"J Aru","year":"2020","journal-title":"Trends Cogn Sci"},{"key":"pcbi.1011839.ref097","doi-asserted-by":"crossref","first-page":"595","DOI":"10.1016\/S0166-2236(00)01922-6","article-title":"The thalamic matrix and thalamocortical synchrony","volume":"24","author":"EG Jones","year":"2001","journal-title":"Trends Neurosci"},{"key":"pcbi.1011839.ref098","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1038\/nn.3917","article-title":"The neocortical circuit: themes and variations","volume":"18","author":"KD Harris","year":"2015","journal-title":"Nat Rev Neurosci"},{"issue":"48","key":"pcbi.1011839.ref099","doi-asserted-by":"crossref","first-page":"19401","DOI":"10.1073\/pnas.1112895108","article-title":"Statistical learning of visual transitions in monkey inferotemporal cortex","volume":"108","author":"T Meyer","year":"2011","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"pcbi.1011839.ref100","doi-asserted-by":"crossref","first-page":"548","DOI":"10.3389\/fpsyg.2012.00548","article-title":"How prediction errors shape perception, attention, and motivation","volume":"3","author":"HE Den Ouden","year":"2012","journal-title":"Frontiers in Psychology"},{"issue":"12","key":"pcbi.1011839.ref101","doi-asserted-by":"crossref","first-page":"2709","DOI":"10.1162\/089976601317098501","article-title":"Intrinsic stabilization of output rates by spike-based Hebbian learning","volume":"13","author":"R Kempter","year":"2001","journal-title":"Neural computation"},{"key":"pcbi.1011839.ref102","volume-title":"Introduction to the Theory of Neural Computation","author":"J Hertz","year":"1991"},{"key":"pcbi.1011839.ref103","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1038\/1131","article-title":"Inter-trial neuronal activity in inferior temporal cortex: a putative vehicle to generate long-term visual associations","volume":"1","author":"V Yakovlev","year":"1998","journal-title":"Nat Neurosci"},{"key":"pcbi.1011839.ref104","doi-asserted-by":"crossref","first-page":"15801","DOI":"10.1523\/JNEUROSCI.1569-10.2010","article-title":"Task-dependent changes in short-term memory in the prefrontal cortex","volume":"30","author":"MR Warden","year":"2010","journal-title":"J Neurosci"},{"issue":"6","key":"pcbi.1011839.ref105","doi-asserted-by":"crossref","first-page":"1194","DOI":"10.1016\/j.neuron.2020.09.024","article-title":"Opposing influence of top-down and bottom-up input on excitatory layer 2\/3 neurons in mouse primary visual cortex","volume":"108","author":"R Jordan","year":"2020","journal-title":"Neuron"},{"key":"pcbi.1011839.ref106","first-page":"172","article-title":"Boosting visual cortex function and plasticity with acetylcholine to enhance visual perception","volume":"8","author":"JI Kang","year":"2014","journal-title":"Front Syst Neurosci"},{"key":"pcbi.1011839.ref107","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1038\/nrn2573","article-title":"The locus coeruleus and noradrenergic modulation of cognition","volume":"10","author":"SJ Sara","year":"2009","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011839.ref108","doi-asserted-by":"crossref","first-page":"RP85111","DOI":"10.7554\/eLife.85111","article-title":"The locus coeruleus broadcasts prediction errors across the cortex to promote sensorimotor plasticity","volume":"12","author":"R Jordan","year":"2023","journal-title":"eLife"},{"key":"pcbi.1011839.ref109","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1152\/jn.1998.80.1.1","article-title":"Predictive reward signal of dopamine neurons","volume":"80","author":"W Schultz","year":"1998","journal-title":"J Neurophysiology"},{"key":"pcbi.1011839.ref110","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1038\/nrn2022","article-title":"The short-latency dopamine signal: a role in discovering novel actions?","volume":"7","author":"P Redgrave","year":"2006","journal-title":"Nat Rev Neurosci"},{"key":"pcbi.1011839.ref111","doi-asserted-by":"crossref","first-page":"115","DOI":"10.2174\/157015906776359559","article-title":"On the origin of cortical dopamine: Is it a co-transmitter in noradrenergic neurons?","volume":"4","author":"P Devoto","year":"2006","journal-title":"Current Neuropharm"},{"key":"pcbi.1011839.ref112","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/nature19325","article-title":"Locus coeruleus and dopaminergic consolidation of everyday memory","volume":"537","author":"T Takeuchi","year":"2016","journal-title":"Nature"},{"key":"pcbi.1011839.ref113","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1038\/nature12160","article-title":"The importance of mixed selectivity in complex cognitive tasks","volume":"497","author":"M Rigotti","year":"2013","journal-title":"Nature"},{"key":"pcbi.1011839.ref114","article-title":"Constrained predictive coding as a biologically plausible model of the cortical hierarchy","author":"S Golkar","year":"2022","journal-title":"Advances in Neural Information Processing Systems"},{"key":"pcbi.1011839.ref115","doi-asserted-by":"crossref","first-page":"809","DOI":"10.1016\/j.neuron.2012.03.040","article-title":"Sensorimotor mismatch signals in primary visual cortex of the behaving mouse","volume":"74","author":"GB Keller","year":"2012","journal-title":"Neuron"},{"key":"pcbi.1011839.ref116","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1038\/s41583-019-0180-y","article-title":"Adaptive learning under expected and unexpected uncertainty","volume":"20","author":"A Soltani","year":"2019","journal-title":"Nature Reviews Neuroscience"},{"key":"pcbi.1011839.ref117","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1126\/science.1094987","article-title":"Cognitive inflexibility after prefrontal serotonin depletion","volume":"304","author":"HF Clarke","year":"2004","journal-title":"Science"},{"issue":"2","key":"pcbi.1011839.ref118","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1016\/j.ijpsycho.2011.10.003","article-title":"Mismatch negativity (MMN) reduction in schizophrenia\u2014Impaired prediction-error generation, estimation or salience?","volume":"83","author":"J Todd","year":"2012","journal-title":"International Journal of Psychophysiology"},{"issue":"5","key":"pcbi.1011839.ref119","doi-asserted-by":"crossref","first-page":"e37843","DOI":"10.1371\/journal.pone.0037843","article-title":"Spontaneous prediction error generation in schizophrenia","volume":"7","author":"Y Yamashita","year":"2012","journal-title":"PLoS One"},{"issue":"2","key":"pcbi.1011839.ref120","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/j.ijpsycho.2011.09.004","article-title":"Anticipating the future: automatic prediction failures in schizophrenia","volume":"83","author":"JM Ford","year":"2012","journal-title":"International Journal of Psychophysiology"},{"issue":"6","key":"pcbi.1011839.ref121","doi-asserted-by":"crossref","first-page":"1751","DOI":"10.1093\/brain\/awr059","article-title":"Expected value and prediction error abnormalities in depression and schizophrenia","volume":"134","author":"VB Gradin","year":"2011","journal-title":"Brain"},{"issue":"6","key":"pcbi.1011839.ref122","doi-asserted-by":"crossref","first-page":"5184","DOI":"10.1007\/s12035-017-0751-8","article-title":"Changes in tryptophan catabolite (TRYCAT) pathway patterning are associated with mild impairments in declarative memory in schizophrenia and deficits in semantic and episodic memory coupled with increased false-memory creation in deficit schizophrenia","volume":"55","author":"B Kanchanatawan","year":"2018","journal-title":"Molecular Neurobiology"},{"issue":"3","key":"pcbi.1011839.ref123","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1007\/s12640-018-9868-4","article-title":"In schizophrenia, depression, anxiety, and physiosomatic symptoms are strongly related to psychotic symptoms and excitation, impairments in episodic memory, and increased production of neurotoxic tryptophan catabolites: a multivariate and machine learning study","volume":"33","author":"B Kanchanatawan","year":"2018","journal-title":"Neurotoxicity Research"},{"issue":"8","key":"pcbi.1011839.ref124","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1017\/S0033291706007902","article-title":"Dissociable mechanisms for memory impairment in bipolar disorder and schizophrenia","volume":"36","author":"DC Glahn","year":"2006","journal-title":"Psychological Medicine"},{"issue":"6","key":"pcbi.1011839.ref125","doi-asserted-by":"crossref","first-page":"889","DOI":"10.1017\/S0033291708004558","article-title":"Working memory in schizophrenia: a meta-analysis","volume":"39","author":"N Forbes","year":"2009","journal-title":"Psychological Medicine"},{"key":"pcbi.1011839.ref126","doi-asserted-by":"crossref","unstructured":"Diehl PU, Neil D, Binas J, Cook M, Liu SC, Pfeiffer M. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In: 2015 International Joint Conference on Neural Networks (IJCNN). IEEE; 2015. p. 1\u20138.","DOI":"10.1109\/IJCNN.2015.7280696"},{"key":"pcbi.1011839.ref127","unstructured":"Kozdon K, Bentley P. Normalisation of weights and firing rates in spiking neural networks with Spike-Timing-Dependent Plasticity. arXiv preprint arXiv:191000122. 2019;."},{"issue":"7","key":"pcbi.1011839.ref128","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","author":"T Van Erven","year":"2014","journal-title":"IEEE Transactions on Information Theory"},{"key":"pcbi.1011839.ref129","doi-asserted-by":"crossref","first-page":"e18073","DOI":"10.7554\/eLife.18073","article-title":"Adaptive learning and decision-making under uncertainty by metaplastic synapses guided by a surprise detection system","volume":"5","author":"K Iigaya","year":"2016","journal-title":"Elife"},{"issue":"1","key":"pcbi.1011839.ref130","doi-asserted-by":"crossref","first-page":"8722","DOI":"10.1038\/s41598-017-06519-y","article-title":"Nonlinear Bayesian filtering and learning: a neuronal dynamics for perception","volume":"7","author":"A Kutschireiter","year":"2017","journal-title":"Scientific Reports"},{"issue":"1","key":"pcbi.1011839.ref131","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1103\/PhysRevE.51.738","article-title":"Time structure of the activity in neural network models","volume":"51","author":"W Gerstner","year":"1995","journal-title":"Phys Rev E"},{"key":"pcbi.1011839.ref132","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781107447615","volume-title":"Neuronal dynamics: From single neurons to networks and models of cognition","author":"W Gerstner","year":"2014"},{"issue":"4","key":"pcbi.1011839.ref133","doi-asserted-by":"crossref","first-page":"e1007640","DOI":"10.1371\/journal.pcbi.1007640","article-title":"On the choice of metric in gradient-based theories of brain function","volume":"16","author":"SC Surace","year":"2020","journal-title":"PLoS Computational Biology"},{"issue":"518","key":"pcbi.1011839.ref134","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"DM Blei","year":"2017","journal-title":"Journal of the American Statistical Association"},{"key":"pcbi.1011839.ref135","unstructured":"Head T, MechCoder, Louppe G, Shcherbatyi I, fcharras, Vin\u00edcius Z, et al. scikit-optimize\/scikit-optimize: v0.5.2; 2018. Available from: https:\/\/doi.org\/10.5281\/zenodo.1207017."},{"key":"pcbi.1011839.ref136","doi-asserted-by":"crossref","first-page":"e10773","DOI":"10.1371\/journal.pone.0010773","article-title":"The bank of standardized stimuli (boss), a new set of 480 normative photos of objects to be used as visual stimuli in cognitive research","volume":"5","author":"MB Brodeur","year":"2010","journal-title":"PloS one"},{"key":"pcbi.1011839.ref137","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511815706","volume-title":"Spiking Neuron Models: single neurons, populations, plasticity","author":"W Gerstner","year":"2002"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1011839","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2024,3,1]],"date-time":"2024-03-01T00:00:00Z","timestamp":1709251200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011839","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,11]],"date-time":"2024-11-11T19:25:34Z","timestamp":1731353134000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1011839"}},"subtitle":[],"editor":[{"given":"Abigail","family":"Morrison","sequence":"first","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2024,2,20]]},"references-count":137,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2,20]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1011839","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.09.13.507727","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,20]]}}}