{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,28]],"date-time":"2026-05-28T01:47:27Z","timestamp":1779932847448,"version":"3.53.1"},"reference-count":51,"publisher":"MIT Press","issue":"12","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,11,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Associative learning is a behavioral phenomenon in which individuals develop connections between stimuli or events based on their co-occurrence. Initially studied by Pavlov in his conditioning experiments, the fundamental principles of learning have been expanded on through the discovery of a wide range of learning phenomena. Computational models have been developed based on the concept of minimizing reward prediction errors. The Rescorla-Wagner model, in particular, is a well-known model that has greatly influenced the field of reinforcement learning. However, the simplicity of these models restricts their ability to fully explain the diverse range of behavioral phenomena associated with learning. In this study, we adopt the free energy principle, which suggests that living systems strive to minimize surprise or uncertainty under their internal models of the world. We consider the learning process as the minimization of free energy and investigate its relationship with the Rescorla-Wagner model, focusing on the informational aspects of learning, different types of surprise, and prediction errors based on beliefs and values. Furthermore, we explore how well-known behavioral phenomena such as blocking, overshadowing, and latent inhibition can be modeled within the active inference framework. We accomplish this by using the informational and novelty aspects of attention, which share similar ideas proposed by seemingly contradictory models such as Mackintosh and Pearce-Hall models. Thus, we demonstrate that the free energy principle, as a theoretical framework derived from first principles, can integrate the ideas and models of associative learning proposed based on empirical experiments and serve as a framework for a better understanding of the computational processes behind associative learning in the brain.<\/jats:p>","DOI":"10.1162\/neco_a_01711","type":"journal-article","created":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T17:07:52Z","timestamp":1727111272000},"page":"2602-2635","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":3,"title":["Associative Learning and Active Inference"],"prefix":"10.1162","volume":"36","author":[{"given":"Petr","family":"Anokhin","sequence":"first","affiliation":[{"name":"AIRI, Moscow, Russia 121170 anokhin@airi.net"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Artyom","family":"Sorokin","sequence":"additional","affiliation":[{"name":"AIRI, Moscow, Russia 121170 asorokin@airi.net"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mikhail","family":"Burtsev","sequence":"additional","affiliation":[{"name":"London Institute for Mathematical Sciences, Royal Institution, London W1S 4BS, U.K. mb@lims.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karl","family":"Friston","sequence":"additional","affiliation":[{"name":"Queen Square Institute of Neurology, University College London, U.K."},{"name":"VERSES AI Research Lab, Los Angeles, CA 90016, U.S.A. k.friston@ucl.ac.uk"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"2024111519200661800_bib1","volume-title":"Biology and neurophysiology of the conditioned reflex and its role in adaptive behavior.","author":"Anokhin","year":"1974"},{"key":"2024111519200661800_bib2","first-page":"9","article-title":"Planning by probabilistic inference","volume-title":"Proceedings of the 9th International Workshop on Artificial Intelligence and Statistics","author":"Attias","year":"2003"},{"issue":"5","key":"2024111519200661800_bib3","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1016\/j.neunet.2009.12.007","article-title":"Of bits and wows: A Bayesian theory of surprise with applications to attention","volume":"23","author":"Baldi","year":"2010","journal-title":"Neural Networks"},{"key":"2024111519200661800_bib4","doi-asserted-by":"publisher","first-page":"907","DOI":"10.3389\/fpsyg.2013.00907","article-title":"Novelty or surprise?","volume":"4","author":"Barto","year":"2013","journal-title":"Frontiers in Psychology"},{"key":"2024111519200661800_bib5","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1016\/j.tics.2012.08.006","article-title":"Planning as inference","volume":"16","author":"Botvinick","year":"2012","journal-title":"Trends in Cognitive Sciences"},{"key":"2024111519200661800_bib6","volume-title":"Learning and behavior: A contemporary synthesis","author":"Bouton","year":"2016"},{"key":"2024111519200661800_bib7","doi-asserted-by":"publisher","first-page":"102447","DOI":"10.1016\/j.jmp.2020.102447","article-title":"Active inference on discrete state-spaces: A synthesis","volume":"99","author":"Da Costa","year":"2020","journal-title":"Journal of Mathematical Psychology"},{"key":"2024111519200661800_bib8","doi-asserted-by":"publisher","first-page":"285","DOI":"10.3758\/BF03192884","article-title":"Attention and salience in associative blocking","volume":"34","author":"Denton","year":"2006","journal-title":"Learning and Behavior"},{"key":"2024111519200661800_bib9","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1038\/nrn2787","article-title":"The free-energy principle: A unified brain theory?","volume":"11","author":"Friston","year":"2010","journal-title":"Nature Reviews Neuroscience"},{"key":"2024111519200661800_bib10","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active inference and learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neuroscience and Biobehavioral Reviews"},{"key":"2024111519200661800_bib11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/neco_a_00999","article-title":"Active inference, curiosity and insight","volume":"29","author":"Friston","year":"2017","journal-title":"Neural Computation"},{"key":"2024111519200661800_bib12","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1007\/s00422-011-0424-z","article-title":"Action understanding and active inference","volume":"104","author":"Friston","year":"2011","journal-title":"Biological Cybernetics"},{"issue":"4","key":"2024111519200661800_bib13","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1080\/17588928.2015.1020053","article-title":"Active inference and epistemic value","volume":"6","author":"Friston","year":"2015","journal-title":"Cognitive Neuroscience"},{"key":"2024111519200661800_bib14","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1016\/j.nlm.2013.11.019","article-title":"Time to rethink the neural mechanisms of learning and memory","volume":"108","author":"Gallistel","year":"2014","journal-title":"Neurobiology of Learning and Memory"},{"key":"2024111519200661800_bib15","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.beproc.2013.08.012","article-title":"Temporal contingency","volume":"101","author":"Gallistel","year":"2014","journal-title":"Behavioural Processes"},{"key":"2024111519200661800_bib16","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1098\/rstb.1980.0090","article-title":"Perceptions as hypotheses","volume":"290","author":"Gregory","year":"1980","journal-title":"Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences"},{"issue":"73","key":"2024111519200661800_bib17","doi-asserted-by":"publisher","first-page":"4098","DOI":"10.21105\/joss.04098","article-title":"pymdp: A Python library for active inference in discrete state spaces","volume":"7","author":"Heins","year":"2022","journal-title":"Journal of Open Source Software"},{"issue":"2","key":"2024111519200661800_bib18","doi-asserted-by":"publisher","first-page":"398","DOI":"10.1162\/neco_a_01341","article-title":"Deeply felt affect: The emergence of valence in deep active inference","volume":"33","author":"Hesp","year":"2021","journal-title":"Neural Computation"},{"key":"2024111519200661800_bib19","doi-asserted-by":"publisher","first-page":"1295","DOI":"10.1016\/j.visres.2008.09.007","article-title":"Bayesian surprise attracts human attention","volume":"49","author":"Itti","year":"2008","journal-title":"Vision Research"},{"key":"2024111519200661800_bib20","doi-asserted-by":"publisher","first-page":"e1003094","DOI":"10.1371\/journal.pcbi.1003094","article-title":"Emotional valence and the free-energy principle","volume":"9","author":"Joffily","year":"2013","journal-title":"PLOS Computational Biology"},{"key":"2024111519200661800_bib21","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1016\/j.tins.2004.10.007","article-title":"The Bayesian brain: The role of uncertainty in neural coding and computation","volume":"27","author":"Knill","year":"2005","journal-title":"Trends in Neurosciences"},{"key":"2024111519200661800_bib22","doi-asserted-by":"publisher","first-page":"104401","DOI":"10.1016\/j.neubiorev.2021.10.023","article-title":"Precision control for a flexible body representation","volume":"134","author":"Limanowski","year":"2022","journal-title":"Neuroscience and Biobehavioral Reviews"},{"key":"2024111519200661800_bib23","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1016\/S0896-6273(01)00186-6","article-title":"Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep","volume":"29","author":"Louie","year":"2001","journal-title":"Neuron"},{"key":"2024111519200661800_bib24","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1037\/0097-7403.2.2.163","article-title":"Latent inhibition and conditioned attention theory","volume":"2","author":"Lubow","year":"1976","journal-title":"Journal of Experimental Psychology: Animal Behavior Processes"},{"key":"2024111519200661800_bib25","doi-asserted-by":"publisher","first-page":"276","DOI":"10.1037\/h0076778","article-title":"A theory of attention: Variations in the associability of stimuli with reinforcement","volume":"82","author":"Mackintosh","year":"1975","journal-title":"Psychological Review"},{"key":"2024111519200661800_bib26","volume-title":"Brain and intelligence in vertebrates","author":"Macphail","year":"1982"},{"key":"2024111519200661800_bib27","doi-asserted-by":"publisher","DOI":"10.1037\/xge0000200","article-title":"The elusive nature of the blocking effect: 15 failures to replicate","volume":"145","author":"Maes","year":"2016","journal-title":"Journal of Experimental Psychology General"},{"issue":"3","key":"2024111519200661800_bib28","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1037\/0033-2909.117.3.363","article-title":"Assessment of the Rescorla-Wagner model","volume":"117","author":"Miller","year":"1995","journal-title":"Psychological Bulletin"},{"issue":"2","key":"2024111519200661800_bib29","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1162\/neco_a_01354","article-title":"Whence the expected free energy?","volume":"33","author":"Millidge","year":"2021","journal-title":"Neural Computation"},{"issue":"7540","key":"2024111519200661800_bib30","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"2024111519200661800_bib31","doi-asserted-by":"publisher","first-page":"6","DOI":"10.3389\/neuro.12.006.2007","article-title":"What is intrinsic motivation? A typology of computational approaches","volume":"1","author":"Oudeyer","year":"2007","journal-title":"Frontiers in Neurorobotics"},{"issue":"1","key":"2024111519200661800_bib32","doi-asserted-by":"publisher","first-page":"14678","DOI":"10.1038\/s41598-017-15249-0","article-title":"Working memory, attention, and salience in active inference","volume":"7","author":"Parr","year":"2017","journal-title":"Scientific Reports"},{"key":"2024111519200661800_bib33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.copsyc.2018.10.006","article-title":"Attention or salience?","volume":"29","author":"Parr","year":"2018","journal-title":"Current Opinion in Psychology"},{"key":"2024111519200661800_bib34","doi-asserted-by":"publisher","first-page":"772641","DOI":"10.3389\/fnsys.2021.772641","article-title":"Understanding, explanation, and active inference","volume":"15","author":"Parr","year":"2021","journal-title":"Frontiers in Systems Neuroscience"},{"key":"2024111519200661800_bib36","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/12441.001.0001","volume-title":"Active inference: The free energy principle in mind, brain, and behavior","author":"Parr","year":"2022"},{"key":"2024111519200661800_bib37","volume-title":"Conditioned reflexes: An investigation of the physiological activity of the cerebral cortex","author":"Pavlov","year":"1927"},{"issue":"6","key":"2024111519200661800_bib38","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1037\/0033-295X.87.6.532","article-title":"A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli","volume":"87","author":"Pearce","year":"1980","journal-title":"Psychological Review"},{"key":"2024111519200661800_bib39","article-title":"Two theories of attention: A review and a possible integration","volume-title":"Attention and associative learning","author":"Pearce","year":"2010"},{"key":"2024111519200661800_bib40","doi-asserted-by":"publisher","first-page":"e1003383","DOI":"10.1371\/journal.pcbi.1003383","article-title":"Forward and backward inference in spatial cognition","volume":"9","author":"Penny","year":"2013","journal-title":"PLOS Computational Biology"},{"key":"2024111519200661800_bib41","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2014.06.011","article-title":"Internally generated sequences in learning and executing goal-directed behavior","volume":"18","author":"Pezzulo","year":"2014","journal-title":"Trends in Cognitive Sciences"},{"issue":"3","key":"2024111519200661800_bib42","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2205.11543","article-title":"On Bayesian mechanics: A physics of and by beliefs","volume":"13","author":"Ramstead","year":"2023","journal-title":"Interface Focus"},{"key":"2024111519200661800_bib43","first-page":"64","article-title":"A theory of Pavlovian conditioning: Variations on the effectiveness of reinforcement and non-reinforcement","volume-title":"Classical conditioning II: Current research and theory","author":"Rescorla","year":"1972"},{"key":"2024111519200661800_bib44","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1016\/0028-3932(87)90041-8","article-title":"Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention","volume":"25","author":"Rizzolatti","year":"1987","journal-title":"Neuropsychologia"},{"issue":"3","key":"2024111519200661800_bib45","doi-asserted-by":"publisher","first-page":"674","DOI":"10.1162\/neco_a_01357","article-title":"Active inference: Demystified and compared","volume":"33","author":"Sajid","year":"2021","journal-title":"Neural Computation"},{"key":"2024111519200661800_bib46","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1017\/9781009026949.007","article-title":"Active inference, Bayesian optimal design, and expected utility","volume-title":"The drive for knowledge: The science of human information seeking","author":"Sajid","year":"2022"},{"key":"2024111519200661800_bib47","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1109\/TAMD.2010.2056368","article-title":"Formal theory of creativity, fun, and intrinsic motivation (1990\u20132010)","volume":"2","author":"Schmidhuber","year":"2010","journal-title":"IEEE Transactions on Autonomous Mental Development"},{"key":"2024111519200661800_bib48","doi-asserted-by":"publisher","first-page":"23","DOI":"10.31887\/DCNS.2016.18.1\/wschultz","article-title":"Dopamine reward prediction error coding","volume":"18","author":"Schultz","year":"2016","journal-title":"Dialogues in Clinical Neuroscience"},{"key":"2024111519200661800_bib49","doi-asserted-by":"publisher","first-page":"e41703","DOI":"10.7554\/eLife.41703","article-title":"Computational mechanisms of curiosity and goal-directed exploration","volume":"8","author":"Schwartenbeck","year":"2019","journal-title":"eLife"},{"key":"2024111519200661800_bib50","doi-asserted-by":"publisher","first-page":"102632","DOI":"10.1016\/j.jmp.2021.102632","article-title":"A step-by-step tutorial on active inference and its application to empirical data","volume":"107","author":"Smith","year":"2022","journal-title":"Journal of Mathematical Psychology"},{"key":"2024111519200661800_bib51","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the method of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Machine Learning"},{"key":"2024111519200661800_bib52","first-page":"497","article-title":"Time-derivative models of Pavlovian reinforcement","volume-title":"Learning and computational neuroscience: Foundations of adaptive networks","author":"Sutton","year":"1990"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/12\/2602\/2479627\/neco_a_01711.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/36\/12\/2602\/2479627\/neco_a_01711.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T19:20:18Z","timestamp":1731698418000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/36\/12\/2602\/124536\/Associative-Learning-and-Active-Inference"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":51,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,11,19]]},"published-print":{"date-parts":[[2024,11,19]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01711","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,12]]},"published":{"date-parts":[[2024,11,19]]}}}