{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T00:45:46Z","timestamp":1775263546484,"version":"3.50.1"},"reference-count":48,"publisher":"MIT Press - Journals","issue":"3","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,2,17]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Animals make efficient probabilistic inferences based on uncertain and noisy information from the outside environment. It is known that probabilistic population codes, which have been proposed as a neural basis for encoding probability distributions, allow general neural networks (NNs) to perform near-optimal point estimation. However, the mechanism of sampling-based probabilistic inference has not been clarified. In this study, we trained two types of artificial NNs, feedforward NN (FFNN) and recurrent NN (RNN), to perform sampling-based probabilistic inference. Then we analyzed and compared their mechanisms of sampling. We found that sampling in RNN was performed by a mechanism that efficiently uses the properties of dynamical systems, unlike FFNN. In addition, we found that sampling in RNNs acted as an inductive bias, enabling a more accurate estimation than in maximum a posteriori estimation. These results provide important arguments for discussing the relationship between dynamical systems and information processing in NNs.<\/jats:p>","DOI":"10.1162\/neco_a_01477","type":"journal-article","created":{"date-parts":[[2022,1,13]],"date-time":"2022-01-13T20:23:10Z","timestamp":1642105390000},"page":"804-827","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":2,"title":["Dynamical Mechanism of Sampling-Based Probabilistic Inference Under Probabilistic Population Codes"],"prefix":"10.1162","volume":"34","author":[{"given":"Kohei","family":"Ichikawa","sequence":"first","affiliation":[{"name":"Graduate School of Arts and Sciences, University of Tokyo, Tokyo 153-0041, Japan"},{"name":"ACES, Bunkyo-ku, Tokyo-to 223-0034, Japan kohei-ichikawa991@g.ecc.u-tokyo.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Asaki","family":"Kataoka","sequence":"additional","affiliation":[{"name":"Graduate School of Arts and Sciences, University of Tokyo, Tokyo 153-0041, Japan"},{"name":"ACES, Bunkyo-ku, Tokyo-to 223-0034, Japan asaki-kataoka@g.ecc.u-tokyo.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2022,2,17]]},"reference":[{"issue":"12","key":"2022022323294643100_B1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pcbi.1005186","article-title":"The Hamiltonian brain: Efficient probabilistic inference with excitatory-inhibitory neural circuit dynamics","volume":"12","author":"Aitchison","year":"2016","journal-title":"PLOS Computational Biology"},{"issue":"4","key":"2022022323294643100_B2","doi-asserted-by":"publisher","first-page":"452","DOI":"10.1016\/j.conb.2009.06.008","article-title":"Multisensory integration: Psychophysics, neurophysiology, and computation","volume":"19","author":"Angelaki","year":"2009","journal-title":"Current Opinion in Neurobiology"},{"issue":"5","key":"2022022323294643100_B3","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1038\/nrn1888","article-title":"Neural correlations, population coding and computation","volume":"7","author":"Averbeck","year":"2006","journal-title":"Nature Reviews Neuroscience"},{"issue":"7","key":"2022022323294643100_B4","doi-asserted-by":"crossref","first-page":"2723","DOI":"10.1073\/pnas.1816766116","article-title":"Stimulus complexity shapes response correlations in primary visual cortex","volume":"116","author":"B\u00e1nyai","year":"2019","journal-title":"PNAS"},{"key":"2022022323294643100_B5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.conb.2017.06.003","article-title":"Recurrent neural networks as versatile tools of neuroscience research","volume":"46","author":"Barak","year":"2017","journal-title":"Current Opinion in Neurobiology"},{"issue":"1","key":"2022022323294643100_B6","first-page":"149","article-title":"A model of inductive bias learning","volume":"12","author":"Baxter","year":"2000","journal-title":"J. Artif. Int. Res."},{"issue":"43","key":"2022022323294643100_B7","doi-asserted-by":"publisher","first-page":"15310","DOI":"10.1523\/JNEUROSCI.1706-11.2011","article-title":"Marginalization in neural circuits with divisive normalization","volume":"31","author":"Beck","year":"2011","journal-title":"Journal of Neuroscience"},{"issue":"6","key":"2022022323294643100_B8","doi-asserted-by":"publisher","first-page":"1142","DOI":"10.1016\/j.neuron.2008.09.021","article-title":"Probabilistic population codes for Bayesian decision making","volume":"60","author":"Beck","year":"2008","journal-title":"Neuron"},{"key":"2022022323294643100_B9","author":"Bengio","year":"2015","journal-title":"Towards biologically plausible deep learning"},{"key":"2022022323294643100_B10","article-title":"Classification and geometry of general perceptual manifolds","volume":"8","author":"Chung","year":"2018","journal-title":"Phys. Rev. X"},{"issue":"1","key":"2022022323294643100_B11","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-020-14578-5","article-title":"Separability and geometry of object manifolds in deep neural networks","volume":"11","author":"Cohen","year":"2020","journal-title":"Nature Communications"},{"key":"2022022323294643100_B12","author":"Doya","year":"2007","journal-title":"Bayesian brain: Probabilistic approaches to neural coding"},{"issue":"9","key":"2022022323294643100_B13","doi-asserted-by":"publisher","first-page":"1138","DOI":"10.1038\/s41593-020-0671-1","article-title":"Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference","volume":"23","author":"Echeveste","year":"2020","journal-title":"Nature Neuroscience"},{"issue":"6870","key":"2022022323294643100_B14","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/415429a","article-title":"Humans integrate visual and haptic information in a statistically optimal fashion","volume":"415","author":"Ernst","year":"2002","journal-title":"Nature"},{"issue":"3","key":"2022022323294643100_B15","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1016\/j.neuron.2016.03.020","article-title":"Perceptual decision-making as probabilistic inference by neural sampling","volume":"90","author":"Haefner","year":"2016","journal-title":"Neuron"},{"issue":"2","key":"2022022323294643100_B16","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1016\/0004-3702(88)90002-1","article-title":"Quantifying inductive bias: AI learning algorithms and Valiant's learning framework","volume":"36","author":"Haussler","year":"1988","journal-title":"Artificial Intelligence"},{"issue":"105","key":"2022022323294643100_B17","first-page":"3403","article-title":"On the inductive bias of dropout","volume":"16","author":"Helmbold","year":"2015","journal-title":"Journal of Machine Learning Research"},{"issue":"2065","key":"2022022323294643100_B18","doi-asserted-by":"publisher","DOI":"10.1098\/rsta.2015.0202","article-title":"Principal component analysis: A review and recent developments","volume":"374","author":"Jolliffe","year":"2016","journal-title":"Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences"},{"key":"2022022323294643100_B19","author":"Kingma","year":"2014","journal-title":"Adam: A method for stochastic optimization"},{"issue":"12","key":"2022022323294643100_B20","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1016\/j.tins.2004.10.007","article-title":"The Bayesian brain: The role of uncertainty in neural coding and computation","volume":"27","author":"Knill","year":"2004","journal-title":"Trends in Neurosciences"},{"key":"2022022323294643100_B21","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"issue":"1","key":"2022022323294643100_B22","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms13276","article-title":"Random synaptic feedback weights support error backpropagation for deep learning","volume":"7","author":"Lillicrap","year":"2016","journal-title":"Nature Communications"},{"issue":"6","key":"2022022323294643100_B23","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1038\/s41583-020-0277-3","article-title":"Backpropagation and the brain","volume":"21","author":"Lillicrap","year":"2020","journal-title":"Nature Reviews Neuroscience"},{"issue":"11","key":"2022022323294643100_B24","doi-asserted-by":"publisher","first-page":"1432","DOI":"10.1038\/nn1790","article-title":"Bayesian inference with probabilistic population codes","volume":"9","author":"Ma","year":"2006","journal-title":"Nature Neuroscience"},{"issue":"2","key":"2022022323294643100_B25","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/j.conb.2008.07.004","article-title":"Spiking networks for Bayesian inference and choice","volume":"18","author":"Ma","year":"2008","journal-title":"Current Opinion in Neurobiology"},{"key":"2022022323294643100_B26","volume-title":"Advances in neural information processing systems","author":"Maheswaranathan","year":"2019"},{"issue":"7474","key":"2022022323294643100_B27","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1038\/nature12742","article-title":"Context-dependent computation by recurrent dynamics in prefrontal cortex","volume":"503","author":"Mante","year":"2013","journal-title":"Nature"},{"issue":"6728","key":"2022022323294643100_B28","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1038\/19303","article-title":"Humans use internal models to estimate gravity and linear acceleration","volume":"398","author":"Merfeld","year":"1999","journal-title":"Nature"},{"issue":"10","key":"2022022323294643100_B29","doi-asserted-by":"publisher","first-page":"1410","DOI":"10.1038\/nn.3807","article-title":"Information-limiting correlations","volume":"17","author":"Moreno-Bote","year":"2014","journal-title":"Nature Neuroscience"},{"issue":"30","key":"2022022323294643100_B30","doi-asserted-by":"publisher","first-page":"12491","DOI":"10.1073\/pnas.1101430108","article-title":"Bayesian sampling in visual perception","volume":"108","author":"Moreno-Bote","year":"2011","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2022022323294643100_B31","first-page":"807","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Nair","year":"2010"},{"issue":"2","key":"2022022323294643100_B32","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1016\/j.neuron.2016.09.038","article-title":"Neural variability and sampling-based probabilistic representations in the visual cortex.","volume":"92","author":"Orb\u00e1n","year":"2016","journal-title":"Neuron"},{"issue":"1","key":"2022022323294643100_B33","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-017-00181-8","article-title":"Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback","volume":"8","author":"Orhan","year":"2017","journal-title":"Nature Communications"},{"issue":"2","key":"2022022323294643100_B34","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1038\/s41593-018-0314-y","article-title":"A diverse range of factors affect the nature of neural representations underlying short-term memory.","volume":"22","author":"Orhan","year":"2019","journal-title":"Nature Neuroscience"},{"issue":"50","key":"2022022323294643100_B35","doi-asserted-by":"publisher","first-page":"20332","DOI":"10.1073\/pnas.1219756110","article-title":"Trial-to-trial, uncertainty-based adjustment of decision boundaries in visual categorization","volume":"110","author":"Qamar","year":"2013","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"11","key":"2022022323294643100_B36","doi-asserted-by":"publisher","first-page":"1761","DOI":"10.1038\/s41593-019-0520-2","article-title":"A deep learning framework for neuroscience","volume":"22","author":"Richards","year":"2019","journal-title":"Nature Neuroscience"},{"key":"2022022323294643100_B37","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.3.013176","article-title":"Quality of internal representation shapes learning performance in feedback neural networks","volume":"3","author":"Susman","year":"2021","journal-title":"Phys. Rev. Research"},{"issue":"3","key":"2022022323294643100_B38","doi-asserted-by":"publisher","first-page":"626","DOI":"10.1162\/NECO_a_00409","article-title":"Opening the black box: Low-dimensional dynamics in high-dimensional recurrent neural networks","volume":"25","author":"Sussillo","year":"2013","journal-title":"Neural Computation"},{"issue":"1","key":"2022022323294643100_B39","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/S0169-7439(97)00061-0","article-title":"Introduction to multi-layer feedforward neural networks","volume":"39","author":"Svozil","year":"1997","journal-title":"Chemometrics and Intelligent Laboratory Systems"},{"issue":"3","key":"2022022323294643100_B40","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.neures.2013.03.010","article-title":"Population codes in the visual cortex","volume":"76","author":"Tanabe","year":"2013","journal-title":"Neuroscience Research"},{"issue":"8","key":"2022022323294643100_B41","doi-asserted-by":"publisher","first-page":"775","DOI":"10.1016\/0042-6989(83)90200-6","article-title":"The statistical reliability of signals in single neurons in cat and monkey visual cortex","volume":"23","author":"Tolhurst","year":"1983","journal-title":"Vision Research"},{"issue":"3","key":"2022022323294643100_B42","doi-asserted-by":"publisher","first-page":"1355","DOI":"10.1152\/jn.1999.81.3.1355","article-title":"Integration of proprioceptive and visual position-information: An experimentally supported model.","volume":"81","author":"van Beers","year":"1999","journal-title":"Journal of Neurophysiology"},{"key":"2022022323294643100_B43","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1146\/annurev-neuro-092619-094115","article-title":"Computation through neural population dynamics","volume":"43","author":"Vyas","year":"2020","journal-title":"Annu. Rev. Neurosci."},{"issue":"1","key":"2022022323294643100_B44","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1038\/s41593-019-0554-5","article-title":"A neural basis of probabilistic computation in visual cortex","volume":"23","author":"Walker","year":"2020","journal-title":"Nature Neuroscience"},{"issue":"10","key":"2022022323294643100_B45","doi-asserted-by":"publisher","first-page":"1550","DOI":"10.1109\/5.58337","article-title":"Backpropagation through time: What it does and how to do it","volume":"78","author":"Werbos","year":"1990","journal-title":"Proceedings of the IEEE"},{"key":"2022022323294643100_B46","first-page":"461","article-title":"Inductive bias and genetic programming.","author":"Whigham","year":"1995","journal-title":"IET Conference Proceedings"},{"issue":"3","key":"2022022323294643100_B47","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1016\/j.tics.2018.12.005","article-title":"Theories of error back-propagation in the brain","volume":"23","author":"Whittington","year":"2019","journal-title":"Trends in Cognitive Sciences"},{"issue":"6","key":"2022022323294643100_B48","doi-asserted-by":"publisher","first-page":"1048","DOI":"10.1016\/j.neuron.2020.09.005","article-title":"Artificial neural networks for neuroscientists: A primer","volume":"107","author":"Yang","year":"2020","journal-title":"Neuron"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/34\/3\/804\/1989432\/neco_a_01477.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/34\/3\/804\/1989432\/neco_a_01477.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T23:30:07Z","timestamp":1645659007000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/34\/3\/804\/109083\/Dynamical-Mechanism-of-Sampling-Based"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,17]]},"references-count":48,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2,17]]},"published-print":{"date-parts":[[2022,2,17]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01477","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,3]]},"published":{"date-parts":[[2022,2,17]]}}}