{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T10:55:23Z","timestamp":1775732123433,"version":"3.50.1"},"reference-count":66,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T00:00:00Z","timestamp":1705536000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Flemish Government (AI Research Program)"},{"name":"Interuniversity microelectronics centre (IMEC)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Robust evidence suggests that humans explore their environment using a combination of topological landmarks and coarse-grained path integration. This approach relies on identifiable environmental features (topological landmarks) in tandem with estimations of distance and direction (coarse-grained path integration) to construct cognitive maps of the surroundings. This cognitive map is believed to exhibit a hierarchical structure, allowing efficient planning when solving complex navigation tasks. Inspired by human behaviour, this paper presents a scalable hierarchical active inference model for autonomous navigation, exploration, and goal-oriented behaviour. The model uses visual observation and motion perception to combine curiosity-driven exploration with goal-oriented behaviour. Motion is planned using different levels of reasoning, i.e., from context to place to motion. This allows for efficient navigation in new spaces and rapid progress toward a target. By incorporating these human navigational strategies and their hierarchical representation of the environment, this model proposes a new solution for autonomous navigation and exploration. The approach is validated through simulations in a mini-grid environment.<\/jats:p>","DOI":"10.3390\/e26010083","type":"journal-article","created":{"date-parts":[[2024,1,18]],"date-time":"2024-01-18T06:41:22Z","timestamp":1705560082000},"page":"83","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Spatial and Temporal Hierarchy for Autonomous Navigation Using Active Inference in Minigrid Environment"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1112-049X","authenticated-orcid":false,"given":"Daria","family":"de Tinguy","sequence":"first","affiliation":[{"name":"IMEC, Ghent University, 9000 Gent, Belgium"}]},{"given":"Toon","family":"Van de Maele","sequence":"additional","affiliation":[{"name":"VERSES AI Research Lab, Los Angeles, CA 90016, USA"}]},{"given":"Tim","family":"Verbelen","sequence":"additional","affiliation":[{"name":"VERSES AI Research Lab, Los Angeles, CA 90016, USA"}]},{"given":"Bart","family":"Dhoedt","sequence":"additional","affiliation":[{"name":"IMEC, Ghent University, 9000 Gent, Belgium"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e41703","DOI":"10.7554\/eLife.41703","article-title":"Computational mechanisms of curiosity and goal-directed exploration","volume":"8","author":"Schwartenbeck","year":"2019","journal-title":"eLife"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Levine, S., and Shah, D. (2022). Learning robotic navigation from experience: Principles, methods and recent results. Philos. Trans. R. Soc. B Biol. Sci., 378.","DOI":"10.1098\/rstb.2021.0447"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.1109\/TRO.2023.3248510","article-title":"A Survey on Active Simultaneous Localization and Mapping: State of the Art and New Frontiers","volume":"39","author":"Placed","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2392","DOI":"10.1038\/s41467-021-22559-5","article-title":"Clone-structured graph representations enable flexible learning and vicarious evaluation of cognitive maps","volume":"12","author":"George","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_5","unstructured":"Hafner, D., Pasukonis, J., Ba, J., and Lillicrap, T. (2023). Mastering Diverse Domains through World Models. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1038\/nn.4656","article-title":"The cognitive map in humans: Spatial navigation and beyond","volume":"20","author":"Epstein","year":"2017","journal-title":"Nat. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1037\/0278-7393.31.2.195","article-title":"Do Humans Integrate Routes Into a Cognitive Map? Map- Versus Landmark-Based Navigation of Novel Shortcuts","volume":"31","author":"Foo","year":"2005","journal-title":"J. Exp. Psychology. Learn. Mem. Cogn."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.tics.2020.10.004","article-title":"Structuring Knowledge with Cognitive Maps and Cognitive Graphs","volume":"25","author":"Peer","year":"2021","journal-title":"Trends Cogn. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1016\/j.neuron.2016.03.037","article-title":"Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network","volume":"90","author":"Balaguer","year":"2016","journal-title":"Neuron"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Tomov, M.S., Yagati, S., Kumar, A., Yang, W., and Gershman, S.J. (2018). Discovery of Hierarchical Representations for Efficient Planning. bioRxiv.","DOI":"10.1101\/499418"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/978-3-540-31965-8_2","article-title":"Geometric Robot Mapping","volume":"Volume 3429","author":"Andres","year":"2005","journal-title":"Discrete Geometry for Computer Imagery"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"23365","DOI":"10.3390\/s141223365","article-title":"Performance Analysis of the Microsoft Kinect Sensor for 2D Simultaneous Localization and Mapping (SLAM) Techniques","volume":"14","author":"Kamarudin","year":"2014","journal-title":"Sensors"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/j.robot.2010.12.003","article-title":"Simultaneous Path Planning and Topological Mapping (SP2ATM) for environment exploration and goal oriented navigation","volume":"59","author":"Ge","year":"2011","journal-title":"Robot. Auton. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kim, S.H., Kim, J.G., and Yang, T.K. (2008, January 9\u201311). Autonomous SLAM technique by integrating Grid and Topology map. Proceedings of the 2008 International Conference on Smart Manufacturing Application, Goyangi, Republic of Korea.","DOI":"10.1109\/ICSMA.2008.4505564"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Li, Z., Chen, G., Peng, B., and Zhu, X. (2018, January 14\u201316). Robot Navigation Method based on Intelligent Evolution. Proceedings of the 2018 IEEE 4th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China.","DOI":"10.1109\/ITOEC.2018.8740378"},{"key":"ref_16","unstructured":"Parisi, S., Dean, V., Pathak, D., and Gupta, A. (2021). Interesting Object, Curious Agent: Learning Task-Agnostic Exploration. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1016\/j.neunet.2022.03.037","article-title":"Deep learning, reinforcement learning, and world models","volume":"152","author":"Matsuo","year":"2022","journal-title":"Neural Netw."},{"key":"ref_18","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019, January 8\u201314). Shaping Belief States with Generative Environment Models for RL. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"20130475","DOI":"10.1098\/rsif.2013.0475","article-title":"Life as we know it","volume":"10","author":"Friston","year":"2013","journal-title":"J. R. Soc. Interface R. Soc."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1016\/j.neubiorev.2016.06.022","article-title":"Active inference and learning","volume":"68","author":"Friston","year":"2016","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_21","unstructured":"Ha, D., and Schmidhuber, J. (2018). World Models. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1016\/j.neunet.2021.09.011","article-title":"World model learning and inference","volume":"144","author":"Friston","year":"2021","journal-title":"Neural Netw."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Stoianov, I., Maisto, D., and Pezzulo, G. (2022). The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Prog. Neurobiol., 217.","DOI":"10.1016\/j.pneurobio.2022.102329"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.neunet.2021.05.010","article-title":"Robot navigation as hierarchical active inference","volume":"142","author":"Verbelen","year":"2021","journal-title":"Neural Netw."},{"key":"ref_25","unstructured":"Chevalier-Boisvert, M., Willems, L., and Pal, S. (2020, October 01). Minimalistic Gridworld Environment for OpenAI Gym. Available online: https:\/\/github.com\/maximecb\/gym-minigrid."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sadeghi, F., and Levine, S. (2016). CAD2RL: Real Single-Image Flight without a Single Real Image. arXiv.","DOI":"10.15607\/RSS.2017.XIII.034"},{"key":"ref_27","unstructured":"M\u00fcller, M., Dosovitskiy, A., Ghanem, B., and Koltun, V. (2018). Driving Policy Transfer via Modularity and Abstraction. arXiv."},{"key":"ref_28","unstructured":"Janai, J., G\u00fcney, F., Behl, A., and Geiger, A. (2017). Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art. arXiv."},{"key":"ref_29","unstructured":"Feng, D., Haase-Sch\u00fctz, C., Rosenbaum, L., Hertlein, H., Duffhauss, F., Gl\u00e4ser, C., Wiesbeck, W., and Dietmayer, K. (2019). Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1177\/0278364910369715","article-title":"Learning from Demonstration for Autonomous Navigation in Complex Unstructured Terrain","volume":"29","author":"Silver","year":"2010","journal-title":"Int. J. Robot. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Gupta, S., Davidson, J., Levine, S., Sukthankar, R., and Malik, J. (2017). Cognitive Mapping and Planning for Visual Navigation. arXiv.","DOI":"10.1109\/CVPR.2017.769"},{"key":"ref_32","unstructured":"Chaplot, D.S., Gandhi, D., Gupta, S., Gupta, A., and Salakhutdinov, R. (2020). Learning to Explore using Active Neural SLAM. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"554","author":"Tolman","year":"1948","journal-title":"Psychol. Rev."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Madl, T., Franklin, S., Chen, K., Trappl, R., and Montaldi, D. (2016). Exploring the Structure of Spatial Representations. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0157343"},{"key":"ref_35","unstructured":"Zakharov, A., Crosby, M., and Fountas, Z. (2020). Episodic Memory for Learning Subjective-Timescale Models. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Shah, D., Eysenbach, B., Rhinehart, N., and Levine, S. (2021). RECON: Rapid Exploration for Open-World Navigation with Latent Goal Models. arXiv.","DOI":"10.1109\/ICRA48506.2021.9561936"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"574372","DOI":"10.3389\/fncom.2020.574372","article-title":"Learning Generative State Space Models for Active Inference","volume":"14","author":"Wauthier","year":"2020","journal-title":"Front. Comput. Neurosci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Inaba, M., and Corke, P. (2016). Robotics Research, Springer.","DOI":"10.1007\/978-3-319-28872-7"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Neacsu, V., Mirza, M.B., Adams, R.A., and Friston, K.J. (2022). Structure learning enhances concept formation in synthetic Active Inference agents. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0277199"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"\u00c7atal, O., Jansen, W., Verbelen, T., Dhoedt, B., and Steckel, J. (2021). LatentSLAM: Unsupervised multi-sensor representation learning for localization and mapping. arXiv.","DOI":"10.1109\/ICRA48506.2021.9560768"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"787659","DOI":"10.3389\/fnsys.2022.787659","article-title":"Generalized Simultaneous Localization and Mapping (G-SLAM) as unification framework for natural and artificial intelligences: Towards reverse engineering the hippocampal\/entorhinal system and principles of high-level cognition","volume":"16","author":"Safron","year":"2022","journal-title":"Front. Syst. Neurosci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"49738","DOI":"10.1109\/ACCESS.2022.3172712","article-title":"Active Inference Integrated With Imitation Learning for Autonomous Driving","volume":"10","author":"Nozari","year":"2022","journal-title":"IEEE Access"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"20170792","DOI":"10.1098\/rsif.2017.0792","article-title":"The Markov blankets of life: Autonomy, active inference and the free energy principle","volume":"15","author":"Kirchhoff","year":"2018","journal-title":"J. R. Soc. Interface"},{"key":"ref_44","unstructured":"Beal, M. (2003). Variational Algorithms for a Bayesian Inference. [Ph.D. Thesis, University College London]."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s00422-018-0753-2","article-title":"Planning and navigation as active inference","volume":"112","author":"Kaplan","year":"2018","journal-title":"Biol. Cybern."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1162\/neco_a_01351","article-title":"Sophisticated Inference","volume":"33","author":"Friston","year":"2021","journal-title":"Neural Comput."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"825","DOI":"10.3389\/fnhum.2014.00825","article-title":"Uncertainty in perception and the Hierarchical Gaussian Filter","volume":"8","author":"Mathys","year":"2014","journal-title":"Front. Hum. Neurosci."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2365","DOI":"10.1001\/jama.2018.17977","article-title":"Bayesian Hierarchical Models","volume":"320","author":"McGlothlin","year":"2018","journal-title":"JAMA"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Milford, M., Wyeth, G., and Prasser, D. (May, January 26). RatSLAM: A hippocampal model for simultaneous localization and mapping. Proceedings of the IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA \u201904. 2004, New Orleans, LA, USA.","DOI":"10.1109\/ROBOT.2004.1307183"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1204","DOI":"10.1126\/science.aar6170","article-title":"Neural scene representation and rendering","volume":"360","author":"Eslami","year":"2018","journal-title":"Science"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"642780","DOI":"10.3389\/fnbot.2021.642780","article-title":"Active Vision for Robot Manipulators Using the Free Energy Principle","volume":"15","author":"Verbelen","year":"2021","journal-title":"Front. Neurorobotics"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.tics.2007.11.004","article-title":"Segmentation in the Perception and Memory of Events","volume":"12","author":"Kurby","year":"2008","journal-title":"Trends Cogn. Sci."},{"key":"ref_53","unstructured":"Verbelen, T., de Tinguy, D., Mazzaglia, P., Catal, O., and Safron, A. (December, January 28). Chunking space and time with information geometry. Proceedings of the NeurIPS 2022, Thirty-Sixth Conference on Neural Information Processing Systems, Information-Theoretic Principles in Cognitive Systems Workshop, New Orleans, LA, USA."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1038\/nature03721","article-title":"Microstructure of a spatial map in the entorhinal cortex","volume":"436","author":"Hafting","year":"2005","journal-title":"Nature"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Milford, M., Wyeth, G., and Prasser, D. (2006, January 9\u201315). RatSLAM on the Edge: Revealing a Coherent Representation from an Overloaded Rat Brain. Proceedings of the 2006 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Beijing, China.","DOI":"10.1109\/IROS.2006.281869"},{"key":"ref_56","unstructured":"Wang, H., Yu, Y., and Yuan, Q. (2011, January 15\u201317). Application of Dijkstra algorithm in robot path-planning. Proceedings of the 2011 Second International Conference on Mechanic Automation and Control Engineering, Inner Mongolia, China."},{"key":"ref_57","unstructured":"Kingma, D.P., and Ba, J. (2017). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_58","unstructured":"Burda, Y., Edwards, H., Storkey, A.J., and Klimov, O. (2018). Exploration by Random Network Distillation. arXiv."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Pathak, D., Agrawal, P., Efros, A.A., and Darrell, T. (2017). Curiosity-driven Exploration by Self-supervised Prediction. arXiv.","DOI":"10.1109\/CVPRW.2017.70"},{"key":"ref_60","unstructured":"Lee, D., Sugiyama, M., Luxburg, U., Guyon, I., and Garnett, R. (2016). Proceedings of the Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Candra, A., Budiman, M.A., and Hartanto, K. (2020, January 16\u201317). Dijkstra\u2019s and A-Star in Finding the Shortest Path: A Tutorial. Proceedings of the 2020 International Conference on Data Science, Artificial Intelligence, and Business Analytics (DATABIA), Medan, Indonesia.","DOI":"10.1109\/DATABIA50434.2020.9190342"},{"key":"ref_62","unstructured":"Asano, Y., Rupprecht, C., and Vedaldi, A. (2020, January 30). Self-labelling via simultaneous clustering and representation learning. Proceedings of the International Conference on Learning Representations, Addis Ababa, Ethiopia."},{"key":"ref_63","unstructured":"Tinguy, D.D., Mazzaglia, P., Verbelen, T., and Dhoedt, B. (2023). International Workshop on Active Inference, Springer."},{"key":"ref_64","unstructured":"Pasukonis, J., Lillicrap, T., and Hafner, D. (2022). Evaluating Long-Term Memory in 3D Mazes. arXiv."},{"key":"ref_65","unstructured":"Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., and Malik, J. (November, January 27). Habitat: A Platform for Embodied AI Research. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Perez, E., Strub, F., de Vries, H., Dumoulin, V., and Courville, A. (2017). FiLM: Visual Reasoning with a General Conditioning Layer. arXiv.","DOI":"10.1609\/aaai.v32i1.11671"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/1\/83\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:49:40Z","timestamp":1760104180000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/1\/83"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,18]]},"references-count":66,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["e26010083"],"URL":"https:\/\/doi.org\/10.3390\/e26010083","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,18]]}}}