{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T07:04:22Z","timestamp":1784790262824,"version":"3.55.0"},"reference-count":44,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T00:00:00Z","timestamp":1784764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science Committee of the Ministry of Education and Science of the Republic of Kazakhstan","award":["AP22686767"],"award-info":[{"award-number":["AP22686767"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous semantic indoor navigation requires robust low-level control and high-level understanding of objects and spatial context in cluttered and partially occluded environments. While deep reinforcement learning (DRL) methods such as twin delayed deep deterministic policy gradient (TD3) enable reactive obstacle avoidance, they typically struggle with long-horizon semantic navigation, where object-location memory and language-level reasoning are required. We present a lightweight hierarchical two-stage framework that, to the best of our knowledge, is introduced for the first time to integrate a locally deployed vision\u2013language model (VLM), semantic object coordinate memory, and a TD3-based DRL controller for language-conditioned indoor navigation. In Stage 1, the robot performs semantic exploration using odometry, 2D LiDAR, and VLM-based object recognition to build a geometric map and store detected object categories with their estimated world coordinates in a structured javaScript object notation (JSON) semantic memory. In Stage 2, a natural language query is used to retrieve the target object coordinates from memory and pass them to a TD3 target point navigation policy, which performs mapless navigation using odometry and RealSense RGB-D perception. The proposed framework combines open-vocabulary VLM-based object coordinate estimation, LiDAR mapping, RGB-D perception, and language grounding within a unified semantic memory representation. Experiments in a ROS-integrated realistic simulation demonstrate consistent goal-reaching performance and improved navigation efficiency compared with an Artificial Potential Field baseline using the same VLM and a DRL + GPT-4o mini configuration. We also compare the proposed VLM-based recognition module with YOLO-World v2.6 and Grounding DINO, showing that the VLM-based approach provides more reliable semantic grounding and target-coordinate estimation in the tested indoor navigation scenarios, particularly for flexible natural language object queries.<\/jats:p>","DOI":"10.3390\/s26154661","type":"journal-article","created":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T06:11:42Z","timestamp":1784787102000},"page":"4661","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["A Hierarchical VLM-to-TD3 Framework with Novel Object Coordinate Estimation and Persistent Spatial Memory for Semantically Guided Indoor Navigation"],"prefix":"10.3390","volume":"26","author":[{"given":"Yernar","family":"Akhmetbek","sequence":"first","affiliation":[{"name":"Computer Science Department, SDU University, Almatinskaya St. 1\/1, Almaty Region, Kaskelen 040900, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1249-3643","authenticated-orcid":false,"given":"Ayaulym","family":"Parmash","sequence":"additional","affiliation":[{"name":"Computer Science Department, SDU University, Almatinskaya St. 1\/1, Almaty Region, Kaskelen 040900, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0042-0777","authenticated-orcid":false,"given":"Temirlan","family":"Meiramkhanov","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Data Science, Astana IT University, Mangilik Yel Ave., Astana 020000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Azamat","family":"Yesmukhametov","sequence":"additional","affiliation":[{"name":"Institute of Smart Systems and Artificial Intelligence (ISSAI), Nazarbayev University, Qabanbay Batyr Ave., Block 1, Astana 010000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aigul","family":"Meirmanova","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Data Science, Astana IT University, Mangilik Yel Ave., Astana 020000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Darkhan","family":"Zholtayev","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Data Science, Astana IT University, Mangilik Yel Ave., Astana 020000, Kazakhstan"},{"name":"Institute of Smart Systems and Artificial Intelligence (ISSAI), Nazarbayev University, Qabanbay Batyr Ave., Block 1, Astana 010000, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1426","DOI":"10.1002\/rob.22332","article-title":"Implementation of PID controller and enhanced red deer algorithm in optimal path planning of substation inspection robots","volume":"41","author":"Tang","year":"2024","journal-title":"J. Field Robot."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Yasuda, Y.D.V., Martins, L.E.G., and Cappabianco, F.A.M. (2020). Autonomous Visual Navigation for Mobile Robots. ACM Comput. Surv., 53.","DOI":"10.1145\/3368961"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Nahavandi, S., Alizadehsani, R., Nahavandi, D., Mohamed, S., Mohajer, N., Rokonuzzaman, M., and Hossain, I. (2025). A Comprehensive Review on Autonomous Navigation. ACM Comput. Surv., 57.","DOI":"10.1145\/3727642"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Samarakoon, S.M.B.P., Muthugala, M.A.V.J., and Jayasekara, A.G.B.P. (2022). A Review on Human\u2013Robot Proxemics. Electronics, 11.","DOI":"10.3390\/electronics11162490"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ruiz, S.G., Rubio, A.J., Hidalgo-Paniagua, A., and Bandera, A. (2023). Evolution of Socially-Aware Robot Navigation. Electronics, 12.","DOI":"10.3390\/electronics12071570"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Lee, M.R., and Yusuf, S.H. (2022). Mobile Robot Navigation Using Deep Reinforcement Learning. Processes, 10.","DOI":"10.3390\/pr10122748"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2064","DOI":"10.1109\/TNNLS.2019.2927869","article-title":"Deep Reinforcement Learning-Based Automatic Exploration for Navigation in Unknown Environment","volume":"31","author":"Li","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_8","first-page":"349","article-title":"Deep Deterministic Policy Gradient for Navigation of Mobile Robots","volume":"40","author":"Jesus","year":"2021","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"15592","DOI":"10.1109\/ACCESS.2021.3053396","article-title":"A Hybrid Tracking Control Strategy for Nonholonomic Wheeled Mobile Robot Incorporating Deep Reinforcement Learning Approach","volume":"9","author":"Gao","year":"2021","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yan, K., Gao, J., and Li, Y. (2023). Deep reinforcement learning based mobile robot navigation using sensor fusion. Proceedings of the 2023 42nd Chinese Control Conference (CCC), IEEE.","DOI":"10.23919\/CCC58697.2023.10240555"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Fei-Fei, L., and Farhadi, A. (2017). Target-driven visual navigation in indoor scenes using deep reinforcement learning. Proceedings of the 2017 IEEE international conference on robotics and automation (ICRA), IEEE.","DOI":"10.1109\/ICRA.2017.7989381"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1049\/cit2.12043","article-title":"Target-driven visual navigation in indoor scenes using reinforcement learning and imitation learning","volume":"7","author":"Fang","year":"2022","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chen, W., Li, K., and Wang, Y. (2025). Visual target-driven robot crowd navigation with limited FOV using self-attention enhanced deep reinforcement learning. Sensors, 25.","DOI":"10.3390\/s25030639"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kober, J., and Peters, J. (2014). Reinforcement Learning in Robotics: A Survey. Learning Motor Skills; Springer Tracts in Advanced Robotics, Springer.","DOI":"10.1007\/978-3-319-03194-1"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wang, X., Sun, Y., Xie, Y., Jiang, B., and Xiao, J. (2023). Deep reinforcement learning-aided autonomous navigation with landmark generators. Front. Neurorobot., 17.","DOI":"10.3389\/fnbot.2023.1200214"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chen, G., Yao, S., Ma, J., Pan, L., Chen, Y., Xu, P., Ji, J., and Chen, X. (2020). Distributed Non-Communicating Multi-Robot Collision Avoidance via Map-Based Deep Reinforcement Learning. Sensors, 20.","DOI":"10.3390\/s20174836"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, Y.F., Everett, M., Liu, M., and How, J.P. (2017). Socially aware motion planning with deep reinforcement learning. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.","DOI":"10.1109\/IROS.2017.8202312"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chen, J., Li, G., Kumar, S., Ghanem, B., and Yu, F. (2023). How to not train your dragon: Training-free embodied object goal navigation with semantic frontiers. arXiv.","DOI":"10.15607\/RSS.2023.XIX.075"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Yu, B., Kasaei, H., and Cao, M. (2023). L3mvn: Leveraging large language models for visual target navigation. Proceedings of the 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.","DOI":"10.1109\/IROS55552.2023.10342512"},{"key":"ref_20","unstructured":"Wu, P., Mu, Y., Wu, B., Hou, Y., Ma, J., Zhang, S., and Liu, C. (2024). Voronav: Voronoi-based zero-shot object navigation with large language model. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5285","DOI":"10.52202\/079017-0171","article-title":"Sg-nav: Online 3d scene graph prompting for llm-based zero-shot object navigation","volume":"37","author":"Yin","year":"2024","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","unstructured":"Zhong, L., Gao, C., Ding, Z., Liao, Y., Ma, H., Zhang, S., Zhou, X., and Liu, S. (2024). Topv-nav: Unlocking the top-view spatial reasoning potential of mllm for zero-shot object navigation. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kuang, Y., Lin, H., and Jiang, M. (2024). Openfmnav: Towards open-set zero-shot object navigation via vision-language foundation models. arXiv.","DOI":"10.18653\/v1\/2024.findings-naacl.24"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gong, Z., Li, R., Hu, T., Qiu, R., Kong, L., Zhang, L., Ding, Y., Zhang, L., and Liang, J. (2025). Stairway to Success: Zero-Shot Floor-Aware Object-Goal Navigation via LLM-Driven Coarse-to-Fine Exploration. arXiv.","DOI":"10.1109\/LRA.2026.3655265"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Shah, H., Xing, J., Messikommer, N., Sun, B., Pollefeys, M., and Scaramuzza, D. (2025, January 11\u201315). Foresightnav: Learning scene imagination for efficient exploration. Proceedings of the Computer Vision and Pattern Recognition Conference, Nashville, TN, USA.","DOI":"10.1109\/CVPRW67362.2025.00519"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Nie, D., Guo, X., Duan, Y., Zhang, R., and Chen, L. (2025). Wmnav: Integrating vision-language models into world models for object goal navigation. arXiv.","DOI":"10.1109\/IROS60139.2025.11246684"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhang, L., Wang, H., Xiao, E., Zhang, X., Zhang, Q., Jiang, Z., and Xu, R. (2025). Multi-floor zero-shot object navigation policy. Proceedings of the 2025 IEEE International Conference on Robotics and Automation (ICRA), IEEE.","DOI":"10.1109\/ICRA55743.2025.11128607"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1080\/01691864.2024.2393409","article-title":"Reflex-based open-vocabulary navigation without prior knowledge using omnidirectional camera and multiple vision-language models","volume":"38","author":"Kawaharazuka","year":"2024","journal-title":"Adv. Robot."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhang, L., Zhang, Q., Wang, H., Xiao, E., Jiang, Z., Chen, H., and Xu, R. (2024). Trihelper: Zero-shot object navigation with dynamic assistance. Proceedings of the 2024 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE.","DOI":"10.1109\/IROS58592.2024.10802670"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yokoyama, N., Ha, S., and Batra, D. (2024). VLFM: Vision-language frontier maps for zero-shot semantic navigation. Proceedings of the Proceedings of the IEEE International Conference on Robotics and Automation, IEEE.","DOI":"10.1109\/ICRA57147.2024.10610712"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1142\/S1793351X25410077","article-title":"Language-Based Augmentation to Mitigate Shortcut Learning in Object-Goal Navigation","volume":"19","author":"Hoftijzer","year":"2025","journal-title":"Int. J. Semant. Comput."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ramakrishnan, S.K., Chaplot, D.S., Al-Halah, Z., Malik, J., and Grauman, K. (2022). Poni: Potential functions for objectgoal navigation with interaction-free learning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01832"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sun, X., Liu, L., Zhi, H., Qiu, R., and Liang, J. (2024). Prioritized semantic learning for zero-shot instance navigation. Proceedings of the European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-031-73254-6_10"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Khandelwal, A., Weihs, L., Mottaghi, R., and Kembhavi, A. (2022). Simple but effective: Clip embeddings for embodied ai. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, IEEE.","DOI":"10.1109\/CVPR52688.2022.01441"},{"key":"ref_35","unstructured":"Long, Y., Cai, W., Wang, H., Zhan, G., and Dong, H. (2024). Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"9737","DOI":"10.1109\/TNNLS.2024.3497992","article-title":"Survey on large language model-enhanced reinforcement learning: Concept, taxonomy, and methods","volume":"36","author":"Cao","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1109\/LRA.2021.3133591","article-title":"Goal-driven autonomous exploration through deep reinforcement learning","volume":"7","author":"Cimurs","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/TRO.2006.889486","article-title":"Improved techniques for grid mapping with rao-blackwellized particle filters","volume":"23","author":"Grisetti","year":"2007","journal-title":"IEEE Trans. Robot."},{"key":"ref_39","unstructured":"Hampel, F.R. (2001). Robust statistics: A brief introduction and overview. Proceedings of the Research Report\/Seminar f\u00fcr Statistik, Eidgen\u00f6ssische Technische Hochschule (ETH), ETH Z\u00fcrich, Seminar f\u00fcr Statistik."},{"key":"ref_40","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"7641","DOI":"10.1609\/aaai.v38i7.28597","article-title":"NavGPT: Explicit reasoning in vision-and-language navigation with large language models","volume":"Volume 38","author":"Zhou","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Huang, C., Mees, O., Zeng, A., and Burgard, W. (2022). Visual language maps for robot navigation. arXiv.","DOI":"10.1109\/ICRA48891.2023.10160969"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Cimurs, R., Lee, J.H., and Suh, I.H. (2020). Goal-oriented obstacle avoidance with deep reinforcement learning in continuous action space. Electronics, 9.","DOI":"10.3390\/electronics9030411"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Savva, M., Kadian, A., Maksymets, O., Zhao, Y., Wijmans, E., Jain, B., Straub, J., Liu, J., Koltun, V., and Malik, J. (2019). Habitat: A platform for embodied ai research. Proceedings of the IEEE\/CVF International Conference on Computer Vision, IEEE.","DOI":"10.1109\/ICCV.2019.00943"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/15\/4661\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,23]],"date-time":"2026-07-23T06:14:25Z","timestamp":1784787265000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/15\/4661"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,23]]},"references-count":44,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["s26154661"],"URL":"https:\/\/doi.org\/10.3390\/s26154661","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,23]]}}}