{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T01:06:26Z","timestamp":1768439186046,"version":"3.49.0"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,5,18]],"date-time":"2020-05-18T00:00:00Z","timestamp":1589760000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>It is crucial to ask how agents can achieve goals by generating action plans using only partial models of the world acquired through habituated sensory-motor experiences. Although many existing robotics studies use a forward model framework, there are generalization issues with high degrees of freedom. The current study shows that the predictive coding (PC) and active inference (AIF) frameworks, which employ a generative model, can develop better generalization by learning a prior distribution in a low dimensional latent state space representing probabilistic structures extracted from well habituated sensory-motor trajectories. In our proposed model, learning is carried out by inferring optimal latent variables as well as synaptic weights for maximizing the evidence lower bound, while goal-directed planning is accomplished by inferring latent variables for maximizing the estimated lower bound. Our proposed model was evaluated with both simple and complex robotic tasks in simulation, which demonstrated sufficient generalization in learning with limited training data by setting an intermediate value for a regularization coefficient. Furthermore, comparative simulation results show that the proposed model outperforms a conventional forward model in goal-directed planning, due to the learned prior confining the search of motor plans within the range of habituated trajectories.<\/jats:p>","DOI":"10.3390\/e22050564","type":"journal-article","created":{"date-parts":[[2020,5,18]],"date-time":"2020-05-18T11:34:14Z","timestamp":1589801654000},"page":"564","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1388-2906","authenticated-orcid":false,"given":"Takazumi","family":"Matsumoto","sequence":"first","affiliation":[{"name":"Okinawa Institute of Science and Technology, Okinawa 904-0495, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9131-9206","authenticated-orcid":false,"given":"Jun","family":"Tani","sequence":"additional","affiliation":[{"name":"Okinawa Institute of Science and Technology, Okinawa 904-0495, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1661","DOI":"10.1093\/qje\/qju024","article-title":"A Sparsity-based Model of Bounded Rationality","volume":"129","author":"Gabaix","year":"2014","journal-title":"Q. J. Econ."},{"key":"ref_2","first-page":"649","article-title":"Bounded Rationality","volume":"146","author":"Selten","year":"1990","journal-title":"J. Inst. Theor. Econ."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1038\/4580","article-title":"Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects","volume":"2","author":"Rao","year":"1999","journal-title":"Nat. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1016\/S0893-6080(99)00060-X","article-title":"Learning to perceive the world as articulated: An approach for hierarchical learning in sensory-motor systems","volume":"12","author":"Tani","year":"1999","journal-title":"Neural Netw."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1434","DOI":"10.1364\/JOSAA.20.001434","article-title":"Hierarchical Bayesian inference in the visual cortex","volume":"20","author":"Lee","year":"2003","journal-title":"J. Opt. Soc. Am. A"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1098\/rstb.2005.1622","article-title":"A theory of cortical responses","volume":"360","author":"Friston","year":"2005","journal-title":"Philos. Trans. R. Soc. B Biol. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hohwy, J. (2013). The Predictive Mind, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199682737.001.0001"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Clark, A. (2015). Surfing Uncertainty: Prediction, Action, and the Embodied Mind, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780190217013.001.0001"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1038\/s41593-018-0200-7","article-title":"Does predictive coding have a future?","volume":"21","author":"Friston","year":"2018","journal-title":"Nat. Neurosci."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Friston, K., Daunizeau, J., and Kiebel, S. (2009). Reinforcement Learning or Active Inference?. PLoS ONE, 4.","DOI":"10.1371\/journal.pone.0006421"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s00422-010-0364-z","article-title":"Action and behavior: A free-energy formulation","volume":"102","author":"Friston","year":"2010","journal-title":"Biol. Cybern."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s00422-011-0424-z","article-title":"Action understanding and active inference","volume":"104","author":"Friston","year":"2011","journal-title":"Biol. Cybern."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/j.jmp.2017.09.004","article-title":"The free energy principle for action and perception: A mathematical review","volume":"81","author":"Buckley","year":"2017","journal-title":"J. Math. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.tics.2018.01.009","article-title":"Hierarchical active inference: A theory of motivated control","volume":"22","author":"Pezzulo","year":"2018","journal-title":"Trends Cogn. Sci."},{"key":"ref_15","unstructured":"Oliver, G., Lanillos, P., and Cheng, G. (2019). Active inference body perception and action for humanoid robots. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1265","DOI":"10.1016\/S0893-6080(96)00035-4","article-title":"Forward Models for Physiological Motor Control","volume":"9","author":"Miall","year":"1996","journal-title":"Neural Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1007\/BF00201442","article-title":"Trajectory formation of arm movement by cascade neural network model based on minimum torque-change criterion","volume":"62","author":"Kawato","year":"1990","journal-title":"Biol. Cybern."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"718","DOI":"10.1016\/S0959-4388(99)00028-8","article-title":"Internal models for motor control and trajectory planning","volume":"9","author":"Kawato","year":"1999","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1109\/3477.499793","article-title":"Model-based learning for mobile robot navigation from the dynamical systems perspective","volume":"26","author":"Tani","year":"1996","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_20","unstructured":"Jordan, M.I. (1986, January 15\u201317). Attractor dynamics and parallelism in a connectionist sequential machine. Proceedings of the 8th Annual Conference of Cognitive Science Society, Amherst, MA, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1142\/S1793005709001283","article-title":"Creating novel goal-directed actions at criticality: A neuro-robotic experiment","volume":"5","author":"Arie","year":"2009","journal-title":"New Math. Nat. Comput."},{"key":"ref_22","unstructured":"Choi, M., Matsumoto, T., Jung, M., and Tani, J. (2018). Generating Goal-Directed Visuomotor Plans Based on Learning Using a Predictive Coding-type Deep Visuomotor Recurrent Neural Network Model. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Jung, M., Matsumoto, T., and Tani, J. (2019, January 3\u20138). Goal-Directed Behavior under Variational Predictive Coding: Dynamic Organization of Visual Attention and Working Memorys. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Macau, China.","DOI":"10.1109\/IROS40897.2019.8968597"},{"key":"ref_24","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). A Recurrent Latent Variable Model for Sequential Data. Advances in Neural Information Processing Systems 28, Curran Associates, Inc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2025","DOI":"10.1162\/neco_a_01228","article-title":"A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition","volume":"31","author":"Ahmadi","year":"2019","journal-title":"Neural Comput."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yamashita, Y., and Tani, J. (2008). Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment. PLoS Comput. Biol., 4.","DOI":"10.1371\/journal.pcbi.1000220"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1177\/105971239500300405","article-title":"On the Dynamics of Small Continuous-Time Recurrent Neural Networks","volume":"3","author":"Beer","year":"1995","journal-title":"Adapt. Behav."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1177\/1059712308089185","article-title":"Learning Multiple Goal-Directed Actions through Self-Organization of a Dynamic Neural Network Model: A Humanoid Robot Experiment","volume":"16","author":"Nishimoto","year":"2008","journal-title":"Adapt. Behav."},{"key":"ref_29","unstructured":"Kingma, D.P., and Welling, M. (2014, January 14\u201316). Auto-Encoding Variational Bayes. Proceedings of the 2nd International Conference on Learning Representations, Banff, AB, Canada."},{"key":"ref_30","first-page":"70","article-title":"A free energy principle for the brain","volume":"100","author":"Friston","year":"2006","journal-title":"J. Physiol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1109\/TSMCA.2003.809171","article-title":"Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment","volume":"33","author":"Tani","year":"2003","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1016\/j.neunet.2019.05.001","article-title":"Learning, planning, and control in a monolithic neural event inference architecture","volume":"117","author":"Butz","year":"2019","journal-title":"Neural Netw."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kirchhoff, M., Parr, T., Palacios, E., Friston, K., and Kiverstein, J. (2018). The Markov blankets of life: Autonomy, active inference and the free energy principle. J. R. Soc. Interface, 15.","DOI":"10.1098\/rsif.2017.0792"},{"key":"ref_34","unstructured":"Ha, D., and Schmidhuber, J. (2018). World Models. arXiv."},{"key":"ref_35","unstructured":"Hafner, D., Lillicrap, T., Fischer, I., Villegas, R., Ha, D., Lee, H., and Davidson, J. (2019, January 9\u201315). Learning Latent Dynamics for Planning from Pixels. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Tenenbaum, G., and Eklund, R.C. (2007). Why do athletes choke under pressure?. Handbook of Sport Psychology, John Wiley & Sons Inc.","DOI":"10.1002\/9781118270011"},{"key":"ref_37","first-page":"152","article-title":"Unfulfilled Prophecies in Sport Performance: Active Inference and the Choking Effect","volume":"27","author":"Cappuccio","year":"2019","journal-title":"J. Conscious. Stud."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/TEVC.2006.890271","article-title":"Intrinsic Motivation Systems for Autonomous Mental Development","volume":"11","author":"Oudeyer","year":"2007","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_39","unstructured":"Forestier, S., Mollard, Y., and Oudeyer, P.Y. (2017). Intrinsically Motivated Goal Exploration Processes with Automatic Curriculum Learning. arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/5\/564\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:30:01Z","timestamp":1760175001000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/5\/564"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,18]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["e22050564"],"URL":"https:\/\/doi.org\/10.3390\/e22050564","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,18]]}}}