{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T14:10:50Z","timestamp":1777126250388,"version":"3.51.4"},"reference-count":74,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,4,1]],"date-time":"2019-04-01T00:00:00Z","timestamp":1554076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Extensive studies have shown that many animals\u2019 capability of forming spatial representations for self-localization, path planning, and navigation relies on the functionalities of place and head-direction (HD) cells in the hippocampus. Although there are numerous hippocampal modeling approaches, only a few span the wide functionalities ranging from processing raw sensory signals to planning and action generation. This paper presents a vision-based navigation system that involves generating place and HD cells through learning from visual images, building topological maps based on learned cell representations and performing navigation using hierarchical reinforcement learning. First, place and HD cells are trained from sequences of visual stimuli in an unsupervised learning fashion. A modified Slow Feature Analysis (SFA) algorithm is proposed to learn different cell types in an intentional way by restricting their learning to separate phases of the spatial exploration. Then, to extract the encoded metric information from these unsupervised learning representations, a self-organized learning algorithm is adopted to learn over the emerged cell activities and to generate topological maps that reveal the topology of the environment and information about a robot\u2019s head direction, respectively. This enables the robot to perform self-localization and orientation detection based on the generated maps. Finally, goal-directed navigation is performed using reinforcement learning in continuous state spaces which are represented by the population activities of place cells. In particular, considering that the topological map provides a natural hierarchical representation of the environment, hierarchical reinforcement learning (HRL) is used to exploit this hierarchy to accelerate learning. The HRL works on different spatial scales, where a high-level policy learns to select subgoals and a low-level policy learns over primitive actions to specialize on the selected subgoals. Experimental results demonstrate that our system is able to navigate a robot to the desired position effectively, and the HRL shows a much better learning performance than the standard RL in solving our navigation tasks.<\/jats:p>","DOI":"10.3390\/s19071576","type":"journal-article","created":{"date-parts":[[2019,4,2]],"date-time":"2019-04-02T03:21:26Z","timestamp":1554175286000},"page":"1576","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Vision-Based Robot Navigation through Combining Unsupervised Learning and Hierarchical Reinforcement Learning"],"prefix":"10.3390","volume":"19","author":[{"given":"Xiaomao","family":"Zhou","sequence":"first","affiliation":[{"name":"College of Automation, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Bai","sequence":"additional","affiliation":[{"name":"College of Automation, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanbin","family":"Gao","sequence":"additional","affiliation":[{"name":"College of Automation, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuntao","family":"Han","sequence":"additional","affiliation":[{"name":"College of Automation, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychol. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1146\/annurev.ps.40.020189.001103","article-title":"Animal cognition: The representation of space, time and number","volume":"40","author":"Gallistel","year":"1989","journal-title":"Annu. Rev. Psychol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"263","DOI":"10.5840\/philstudies19802725","article-title":"The hippocampus as a cognitive map","volume":"27","author":"Breathnach","year":"1980","journal-title":"Philos. Stud."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1038\/nrn1932","article-title":"Path integration and the neural basis of the \u2018cognitive map\u2019","volume":"7","author":"McNaughton","year":"2006","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/0006-8993(71)90358-1","article-title":"The hippocampus as a spatial map: Preliminary evidence from unit activity in the freely-moving rat","volume":"34","author":"Dostrovsky","year":"1971","journal-title":"Brain Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1523\/JNEUROSCI.10-02-00420.1990","article-title":"Head-direction cells recorded from the postsubiculum in freely moving rats. I. Description and quantitative analysis","volume":"10","author":"Taube","year":"1990","journal-title":"J. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7079","DOI":"10.1523\/JNEUROSCI.15-11-07079.1995","article-title":"Interactions between location and task affect the spatial and directional firing of hippocampal neurons","volume":"15","author":"Markus","year":"1995","journal-title":"J. Neurosci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1007\/BF00243212","article-title":"Head-direction cells in the rat posterior cortex","volume":"101","author":"Chen","year":"1994","journal-title":"Exp. Brain Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"9020","DOI":"10.1523\/JNEUROSCI.18-21-09020.1998","article-title":"Firing properties of rat lateral mammillary single units: Head direction, head pitch, and angular head velocity","volume":"18","author":"Stackman","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/S0166-2236(00)01797-5","article-title":"The anatomical and computational basis of the rat head-direction cell signal","volume":"24","author":"Sharp","year":"2001","journal-title":"Trends Neurosci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1146\/annurev.neuro.31.061307.090723","article-title":"Place cells, grid cells, and the brain\u2019s spatial representation system","volume":"31","author":"Moser","year":"2008","journal-title":"Annu. Rev. Neurosci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1865","DOI":"10.1126\/science.1166466","article-title":"Representation of geometric borders in the entorhinal cortex","volume":"322","author":"Solstad","year":"2008","journal-title":"Science"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1007\/s004220000171","article-title":"Spatial cognition and neuro-mimetic navigation: A model of hippocampal place cell activity","volume":"83","author":"Arleo","year":"2000","journal-title":"Biol. Cybern."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sheynikhovich, D., Chavarriaga, R., Str\u00f6sslin, T., and Gerstner, W. (2005). Spatial representation and navigation in a bio-inspired robot. Biomimetic Neural Learning for Intelligent Robots, Springer.","DOI":"10.1007\/11521082_15"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chokshi, K., Wermter, S., and Weber, C. (2003). Learning localisation based on landmarks using self-organisation. Artificial Neural Networks and Neural Information Processing\u2014ICANN\/ICONIP 2003, Springer.","DOI":"10.1007\/3-540-44989-2_60"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1002\/1098-1063(2000)10:4<369::AID-HIPO3>3.0.CO;2-0","article-title":"Modeling place fields in terms of the cortical inputs to the hippocampus","volume":"10","author":"Hartley","year":"2000","journal-title":"Hippocampus"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3","DOI":"10.3389\/neuro.12.003.2007","article-title":"Neurobiologically inspired mobile robot navigation and planning","volume":"1","author":"Cuperlier","year":"2007","journal-title":"Front. Neurorobot."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1162\/089976602317318938","article-title":"Slow feature analysis: Unsupervised learning of invariances","volume":"14","author":"Wiskott","year":"2002","journal-title":"Neural Comput."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Franzius, M., Sprekeler, H., and Wiskott, L. (2007). Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Comput. Biol., 3.","DOI":"10.1371\/journal.pcbi.0030166"},{"key":"ref_20","first-page":"51","article-title":"Modeling place field activity with hierarchical slow feature analysis","volume":"9","author":"Wiskott","year":"2015","journal-title":"Front. Comput. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7411","DOI":"10.1523\/JNEUROSCI.18-18-07411.1998","article-title":"A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells","volume":"18","author":"Brown","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2112","DOI":"10.1523\/JNEUROSCI.16-06-02112.1996","article-title":"Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: A theory","volume":"16","author":"Zhang","year":"1996","journal-title":"J. Neurosci."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.bbr.2012.12.034","article-title":"Place cell activation predicts subsequent memory","volume":"254","author":"Robitsek","year":"2013","journal-title":"Behav. Brain Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1038\/nature12112","article-title":"Hippocampal place-cell sequences depict future paths to remembered goals","volume":"497","author":"Pfeiffer","year":"2013","journal-title":"Nature"},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Introduction to Reinforcement Learning, MIT Press Cambridge."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1238","DOI":"10.1177\/0278364913495721","article-title":"Reinforcement learning in robotics: A survey","volume":"32","author":"Kober","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_27","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (arXiv, 2015). Continuous control with deep reinforcement learning, arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., and Jurafsky, D. (arXiv, 2016). Deep reinforcement learning for dialogue generation, arXiv.","DOI":"10.18653\/v1\/D16-1127"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1023\/A:1022140919877","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto","year":"2003","journal-title":"Discret. Event Dyn. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1613\/jair.639","article-title":"Hierarchical reinforcement learning with the MAXQ value function decomposition","volume":"13","author":"Dietterich","year":"2000","journal-title":"J. Artif. Intell. Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, X., Weber, C., and Wermter, S. (2017). Robot localization and orientation detection based on place cells. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-319-68600-4_17"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zhou, X., Weber, C., and Wermter, S. (2018, January 8\u201313). A Self-organizing Method for Robot Navigation based on Learned Place and Head-direction cells. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489348"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"74","DOI":"10.3389\/fnsys.2013.00074","article-title":"The mechanisms for pattern completion and pattern separation in the hippocampus","volume":"7","author":"Rolls","year":"2013","journal-title":"Front. Syst. Neurosci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1080\/09548980601064846","article-title":"Entorhinal cortex grid cells can map to hippocampal place cells by competitive learning","volume":"17","author":"Rolls","year":"2006","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1026","DOI":"10.1002\/hipo.20244","article-title":"From grid cells to place cells: A mathematical model","volume":"16","author":"Solstad","year":"2006","journal-title":"Hippocampus"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1177\/0278364909340592","article-title":"Persistent navigation and mapping using a biologically inspired SLAM system","volume":"29","author":"Milford","year":"2010","journal-title":"Int. J. Robot. Res."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tejera, G., Barrera, A., Llofriu, M., and Weitzenfeld, A. (2013, January 25\u201329). Solving uncertainty during robot navigation by integrating grid cell and place cell firing based on rat spatial cognition studies. Proceedings of the 2013 16th International Conference on Advanced Robotics (ICAR), Montevideo, Uruguay.","DOI":"10.1109\/ICAR.2013.6766544"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Giovannangeli, C., and Gaussier, P. (2008, January 22\u201326). Autonomous vision-based navigation: Goal-oriented action planning by transient states prediction, cognitive map building, and sensory-motor learning. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, IROS 2008, Nice, France.","DOI":"10.1109\/IROS.2008.4650872"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1016\/j.neunet.2005.08.012","article-title":"Robust self-localisation and navigation based on hippocampal place cells","volume":"18","author":"Sheynikhovich","year":"2005","journal-title":"Neural Netw."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1111\/j.1460-9568.2012.08015.x","article-title":"A goal-directed spatial navigation model using forward trajectory planning based on grid cells","volume":"35","author":"Erdem","year":"2012","journal-title":"Eur. J. Neurosci."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhou, X., Weber, C., Bothe, C., and Wermter, S. (2018). A Hybrid Planning Strategy Through Learning from Vision for Target-Directed Navigation. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-030-01421-6_30"},{"key":"ref_42","unstructured":"Kulkarni, T.D., Narasimhan, K., Saeedi, A., and Tenenbaum, J. (2016, January 5\u201310). Hierarchical deep reinforcement learning: Integrating temporal abstraction and intrinsic motivation. Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Tang, D., Li, X., Gao, J., Wang, C., Li, L., and Jebara, T. (arXiv, 2018). Subgoal Discovery for Hierarchical Dialogue Policy Learning, arXiv.","DOI":"10.18653\/v1\/D18-1253"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Peng, B., Li, X., Li, L., Gao, J., Celikyilmaz, A., Lee, S., and Wong, K.F. (arXiv, 2017). Composite task-completion dialogue policy learning via hierarchical deep reinforcement learning, arXiv.","DOI":"10.18653\/v1\/D17-1237"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/S0004-3702(99)00052-1","article-title":"Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning","volume":"112","author":"Sutton","year":"1999","journal-title":"Artif. Intell."},{"key":"ref_46","unstructured":"Sorg, J., and Singh, S. (2010, January 10\u201314). Linear options. Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, Toronto, ON, Canada."},{"key":"ref_47","unstructured":"Szepesvari, C., Sutton, R.S., Modayil, J., and Bhatnagar, S. (2014, January 8\u201313). Universal option models. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_48","unstructured":"Goel, S., and Huber, M. (2003, January 12\u201314). Subgoal discovery for hierarchical reinforcement learning using learned policies. Proceedings of the FLAIRS Conference, St. Augustine, FL, USA."},{"key":"ref_49","unstructured":"\u015eim\u015fek, \u00d6., and Barto, A.G. (2009, January 6\u20138). Skill characterization based on betweenness. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Menache, I., Mannor, S., and Shimkin, N. (2002). Q-cut\u2014Dynamic discovery of sub-goals in reinforcement learning. European Conference on Machine Learning, Springer.","DOI":"10.1007\/3-540-36755-1_25"},{"key":"ref_51","unstructured":"Lakshminarayanan, A.S., Krishnamurthy, R., Kumar, P., and Ravindran, B. (arXiv, 2016). Option discovery in hierarchical reinforcement learning using spatio-temporal clustering, arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1016\/S0893-6080(02)00078-3","article-title":"A self-organising network that grows when required","volume":"15","author":"Marsland","year":"2002","journal-title":"Neural Netw."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1464","DOI":"10.1109\/5.58325","article-title":"The self-organizing map","volume":"78","author":"Kohonen","year":"1990","journal-title":"Proc. IEEE"},{"key":"ref_54","unstructured":"Fritzke, B. (December, January 27). A growing neural gas network learns topologies. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_56","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (arXiv, 2013). Playing atari with deep reinforcement learning, arXiv."},{"key":"ref_57","unstructured":"Berkes, P., and Zito, T. (2018, April 30). Modular Toolkit for Data Processing (MDP Version 2.1). Available online: http:\/\/mdp-toolkit.sourceforge.net."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1002\/hipo.20666","article-title":"The velocity-related firing property of hippocampal place cells is dependent on self-movement","volume":"20","author":"Lu","year":"2010","journal-title":"Hippocampus"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"1502","DOI":"10.1109\/TNNLS.2015.2441735","article-title":"Compound rank-k projections for bilinear analysis","volume":"27","author":"Chang","year":"2016","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"1617","DOI":"10.1109\/TPAMI.2016.2608901","article-title":"Semantic pooling for complex event analysis in untrimmed videos","volume":"39","author":"Chang","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"2100","DOI":"10.1109\/TKDE.2017.2728531","article-title":"Beyond trace ratio: Weighted harmonic mean of trace ratios for multiclass discriminant analysis","volume":"29","author":"Li","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_63","first-page":"104","article-title":"RatLab: An easy to use tool for place code simulations","volume":"7","author":"Wiskott","year":"2013","journal-title":"Front. Comput. Neurosci."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1016\/0042-6989(79)90143-3","article-title":"A schematic eye for the rat","volume":"19","author":"Hughes","year":"1979","journal-title":"Vis. Res."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1093\/bjps\/40.3.323","article-title":"Note on entropy, disorder and disorganization","volume":"40","author":"Denbigh","year":"1989","journal-title":"Br. J. Philos. Sci."},{"key":"ref_66","unstructured":"Thrun, S., M\u00f6ller, K., and Linden, A. (1991, January 2\u20135). Planning with an adaptive world model. Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_67","unstructured":"Davison, M.L. (1983). Multidimensional Scaling, Wiley."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1002\/hipo.1052","article-title":"Evidence for a relationship between place-cell spatial firing and spatial memory performance","volume":"11","author":"Save","year":"2001","journal-title":"Hippocampus"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1002\/hipo.20200","article-title":"Hippocampal and cortical place cell plasticity: Implications for episodic memory","volume":"16","author":"Frank","year":"2006","journal-title":"Hippocampus"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1126\/science.1071795","article-title":"Requirement for hippocampal CA3 NMDA receptors in associative memory recall","volume":"297","author":"Nakazawa","year":"2002","journal-title":"Science"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1016\/S0959-4388(97)80011-6","article-title":"Hippocampal lesions and path integration","volume":"7","author":"Whishaw","year":"1997","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_72","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning (ICML), New York, NY, USA."},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Fei-Fei, L., and Farhadi, A. (June, January 29). Target-driven visual navigation in indoor scenes using deep reinforcement learning. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989381"},{"key":"ref_74","unstructured":"Brodeur, S., Perez, E., Anand, A., Golemo, F., Celotti, L., Strub, F., Rouat, J., Larochelle, H., and Courville, A. (arXiv, 2017). HoME: A household multimodal environment, arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1576\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:42:06Z","timestamp":1760186526000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1576"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,1]]},"references-count":74,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["s19071576"],"URL":"https:\/\/doi.org\/10.3390\/s19071576","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,4,1]]}}}