{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:31:50Z","timestamp":1759969910044,"version":"build-2065373602"},"reference-count":49,"publisher":"Cambridge University Press (CUP)","issue":"9","license":[{"start":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T00:00:00Z","timestamp":1756944000000},"content-version":"unspecified","delay-in-days":3,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotica"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Visual exploration is a task in which a camera-equipped robot seeks to efficiently visit all navigable areas of an environment within the shortest possible time. Most existing visual exploration methods rely on a static camera fixed to the robot\u2019s body to control its own movements. However, coupling the orientation of camera with robot\u2019s body limits the extra degrees of freedom to obtain more visual information. In this work, we adjust the camera orientation during robot motion by using a novel camera view planning (CVP) policy to improve the exploration efficiency. Specifically, we reformulate the CVP problem as a reinforcement learning problem. However, two new challenges need to be addressed: 1) determining how to learn an effective CVP policy in complex indoor environments and 2) figuring out how to synchronize it with the robot motion. To solve the above issues, we create a reward function considering factors such as exploration area, observed semantic objects, and the motion conflicts between the camera and the robot\u2019s body. Moreover, to better coordinate the policies of the camera and the robot\u2019s body, the CVP policy takes the body actions and the egocentric 2D spatial maps with exploration, occupancy, and trajectory information into account to make motion decisions. Experimental results show that after using the proposed CVP policy, the exploration area is expanded by 21.72% and 25.6% on average in the small-scale indoor scene with few structured obstacles and large-scale indoor scene with cluttered obstacles, respectively.<\/jats:p>","DOI":"10.1017\/s026357472510221x","type":"journal-article","created":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T07:31:13Z","timestamp":1756971073000},"page":"3257-3276","source":"Crossref","is-referenced-by-count":0,"title":["Learning \u201cwhere to look\u201d for visual exploration of autonomous mobile robot in indoor unknown environments"],"prefix":"10.1017","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9532-4272","authenticated-orcid":false,"given":"Sheng","family":"Jin","sequence":"first","affiliation":[{"name":"Tianjin University"},{"name":"Suzhou City University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuai-Dong","family":"Yang","sequence":"additional","affiliation":[{"name":"Suzhou City University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao-Yu","family":"Wang","sequence":"additional","affiliation":[{"name":"Tianjin University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chen-Yang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Changzhou Institute of Technology"},{"name":"Nanjing University of Science and Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qing-Hao","family":"Meng","sequence":"additional","affiliation":[{"name":"Tianjin University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2025,9,4]]},"reference":[{"key":"S026357472510221X_ref18","unstructured":"[18] Burda, Y. , Edwards, H. , Pathak, D. , Storkey, A. , Darrell, T. and Efros, A. A. , \u201cLarge-scale study of curiosity-driven learning,\u201d (2018). arXiv preprint arXiv: 1808.04355."},{"key":"S026357472510221X_ref39","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.patrec.2020.02.024","article-title":"Supervised learning of the next-best-view for 3D object reconstruction","volume":"133","author":"Mendoza","year":"2020","journal-title":"Pattern Recogn. Lett."},{"key":"S026357472510221X_ref7","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574722001205"},{"key":"S026357472510221X_ref14","doi-asserted-by":"crossref","unstructured":"[14] Dai, A. , Papatheodorou, S. , Funk, N. , Tzoumanikas, D. and Leutenegger, S. . \u201cFast frontier-based information-driven autonomous exploration with an MAV.\u201d In: 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2020) pp. 9570\u20139576.","DOI":"10.1109\/ICRA40945.2020.9196707"},{"key":"S026357472510221X_ref15","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2023.3290395"},{"key":"S026357472510221X_ref23","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aaw6326"},{"key":"S026357472510221X_ref26","doi-asserted-by":"crossref","unstructured":"[26] Savva, M. , Kadian, A. , Maksymets, O. , Zhao, Y. , Wijmans, E. , Jain, B. , Straub, J. , Liu, J. , Koltun, V. and Malik, J. . \u201cHabitat: A platform for embodied ai research.\u201d In: 2019 IEEE\/CVF International Conference on Computer Vision (CVPR), IEEE (2019) pp. 9339\u20139347.","DOI":"10.1109\/ICCV.2019.00943"},{"key":"S026357472510221X_ref22","doi-asserted-by":"crossref","unstructured":"[22] Jayaraman, D. and Grauman, K. . \u201cLearning to look around: Intelligently exploring unseen environments for unknown tasks.\u201d In: 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2018) pp. 1238\u20131247.","DOI":"10.1109\/CVPR.2018.00135"},{"key":"S026357472510221X_ref40","doi-asserted-by":"crossref","unstructured":"[40] Kaba, M. D. , Uzunbas, M. G. and Lim, S. N. . \u201cA reinforcement learning approach to the view planning problem.\u201d In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2017) pp. 6933\u20136941.","DOI":"10.1109\/CVPR.2017.541"},{"key":"S026357472510221X_ref45","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2705103"},{"key":"S026357472510221X_ref37","doi-asserted-by":"crossref","unstructured":"[37] Zwecher, E. , Iceland, E. , Levy, S. R. , Hayoun, S. Y. , Gal, O. and Barel, A. . \u201cIntegrating deep reinforcement and supervised learning to expedite indoor mapping.\u201d In: 2022 International Conference on Robotics and Automation (ICRA), IEEE (2022) pp. 10542\u201310548.","DOI":"10.1109\/ICRA46639.2022.9811861"},{"key":"S026357472510221X_ref31","unstructured":"[31] Chaplot, D. S. , Gandhi, D. , Gupta, S. , Gupta, A. and Salakhutdinov, R. . \u201cLearning to explore using active neural slam.\u201d In: 2020 International Conference on Learning Representations (ICLR) (2020)."},{"key":"S026357472510221X_ref30","first-page":"14961","volume-title":"2016 Neural Information Processing Systems (NIPS)","author":"Dean","year":"2016"},{"key":"S026357472510221X_ref36","doi-asserted-by":"crossref","unstructured":"[36] Shrestha, R. , Tian, F.-P. , Feng, W. , Tan, P. and Vaughan, R. . \u201cLearned map prediction for enhanced mobile robot exploration.\u201d In: 2019 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2019) pp. 1197\u20131204.","DOI":"10.1109\/ICRA.2019.8793769"},{"key":"S026357472510221X_ref13","unstructured":"[13] Yamauchi, B. . \u201cA frontier-based approach for autonomous exploration.\u201d In: 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation, IEEE (1997) pp. 146\u2013151."},{"key":"S026357472510221X_ref38","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1145\/2661229.2661242","article-title":"Quality-driven poisson-guided autoscanning","volume":"33","author":"Wu","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"S026357472510221X_ref46","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-012-9321-0"},{"key":"S026357472510221X_ref43","doi-asserted-by":"crossref","unstructured":"[43] Johns, E. , Leutenegger, S. and Davison, A. J. . \u201cPairwise decomposition of image sequences for active multi-view recognition.\u201d In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2016) pp. 3813\u20133822.","DOI":"10.1109\/CVPR.2016.414"},{"key":"S026357472510221X_ref48","doi-asserted-by":"crossref","unstructured":"[48] Chang, A. , Dai, A. , Funkhouser, T. , Halber, M. , Niessner, M. , Savva, M. , Song, S. , Zeng, A. and Zhang, Y. , \u201cMatterport3D: Learning from RGB-D data in indoor environments.\u201d In: 2017 International Conference on 3D Vision (3DV), IEEE (2017) pp. 667\u2013676.","DOI":"10.1109\/3DV.2017.00081"},{"key":"S026357472510221X_ref9","doi-asserted-by":"crossref","unstructured":"[9] Wu, Y. , Wu, Y. , Tamar, A. , Russell, S. , Gkioxari, G. and Tian, Y. . \u201cBayesian relational memory for semantic visual navigation.\u201d In: 2019 IEEE International Conference on Computer Vision (ICCV), IEEE (2019) pp. 2769\u20132779.","DOI":"10.1109\/ICCV.2019.00286"},{"key":"S026357472510221X_ref11","doi-asserted-by":"crossref","unstructured":"[11] Li, Y. , Debnath, A. , Stein, G. J. and Ko\u0161eck\u00e1, J. . \u201cLearning-augmented model-based planning for visual exploration.\u201d In: 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE (2023) pp. 5165\u20135171.","DOI":"10.1109\/IROS55552.2023.10341773"},{"key":"S026357472510221X_ref6","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-024-06115-4"},{"key":"S026357472510221X_ref32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58558-7_24"},{"key":"S026357472510221X_ref41","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2939431"},{"key":"S026357472510221X_ref21","unstructured":"[21] Chen, T. , Gupta, S. and Gupta, A. . \u201cLearning exploration policies for navigation.\u201d In: 2019 International Conference on Learning Representations (ICLR) (2019)."},{"key":"S026357472510221X_ref3","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574721001983"},{"key":"S026357472510221X_ref29","doi-asserted-by":"publisher","DOI":"10.1177\/0278364902021010834"},{"key":"S026357472510221X_ref1","unstructured":"[1] Ye, J. , Batra, D. , Wijmans, E. and Das, A. . \u201cAuxiliary Tasks Speed Up Learning Point Goal Navigation.\u201d In: Conference on Robot Learning (CoRL) (2020) pp. 498\u2013516."},{"key":"S026357472510221X_ref24","unstructured":"[24] Seifi, S. and Tuytelaars, T. , \u201cWhere to look next: Unsupervised active visual exploration on 360\u00b0 input (2019). arXiv preprint: arXiv: 1909.10304."},{"key":"S026357472510221X_ref33","unstructured":"[33] Chen, C. , Majumder, S. , Al-Halah, Z. , Gao, R. , Ramakrishnan, S. K. and Grauman, K. . \u201cLearning to set waypoints for audio-visual navigation.\u201d In: 2020 International Conference on Learning Representations (ICLR) (2021)."},{"key":"S026357472510221X_ref35","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2891991"},{"key":"S026357472510221X_ref2","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574722001539"},{"key":"S026357472510221X_ref20","unstructured":"[20] Savinov, N. , Raichuk, A. , Marinier, R. , Vincent, D. , Pollefeys, M. , Lillicrap, T. and Gelly, S. , \u201cEpisodic curiosity through reachability,\u201d arXiv preprint arXiv: 1810.02274."},{"key":"S026357472510221X_ref16","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574721001946"},{"key":"S026357472510221X_ref42","unstructured":"[42] Wu, Z. , Song, S. , Khosla, A. , Yu, F. , Zhang, L. , Tang, X. and Xiao, J. . \u201c3D shapenets: A deep representation for volumetric shapes.\u201d In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2015) pp. 1912\u20131920."},{"key":"S026357472510221X_ref5","doi-asserted-by":"crossref","unstructured":"[5] Gupta, S. , Davidson, J. , Levine, S. , Sukthankar, R. and Malik, J. . \u201cCognitive mapping and planning for visual navigation.\u201d In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2017) pp. 2616\u20132625.","DOI":"10.1109\/CVPR.2017.769"},{"key":"S026357472510221X_ref44","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2022.109621"},{"key":"S026357472510221X_ref47","unstructured":"[47] Schulman, J. , Wolski, F. , Dhariwal, P. , Radford, A. and Klimov, O. , \u201cProximal policy optimization algorithms (2017). arXiv preprint arXiv: 1707.06347."},{"key":"S026357472510221X_ref49","unstructured":"[49] Zeng, S. , Chang, X. , Xie, M. , Liu, X. , Bai, Y. , Pan, Z. , Xu, M. and Wei, X. , \u201cFutureSightDrive: Thinking visually with spatio-temporal CoT for autonomous driving (2025). arXiv preprint arXiv: 2505.17685."},{"key":"S026357472510221X_ref17","doi-asserted-by":"crossref","unstructured":"[17] Pathak, D. , Agrawal, P. , Efros, A. A. and Darrell, T. , \u201cCuriosity-driven exploration by self-supervised prediction.\u201d In: 2017 International Conference on Machine Learning (2017) pp. 2778\u20132787.","DOI":"10.1109\/CVPRW.2017.70"},{"key":"S026357472510221X_ref12","doi-asserted-by":"crossref","unstructured":"[12] Bigazzi, R. , Cornia, M. , Cascianelli, S. , Baraldi, L. and Cucchiara, R. . \u201cEmbodied agents for efficient exploration and smart scene description.\u201d In: 2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE (2023) pp. 6057\u20136064.","DOI":"10.1109\/ICRA48891.2023.10160668"},{"key":"S026357472510221X_ref8","unstructured":"[8] Chaplot, D. S. , Salakhutdinov, R. , Gupta, A. and Gupta, S. . \u201cNeural topological slam for visual navigation.\u201d In: 2020 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE (2020) pp. 12875\u201312884."},{"key":"S026357472510221X_ref27","unstructured":"[27] Holz, D. , Basilico, N. , Amigoni, F. and Behnke, S. . \u201cEvaluating the efficiency of frontier-based exploration strategies.\u201d In: 2010 International Symposium on Robotics (ISR) and 2010 German Conference on Robotics (ROBOTIK), IEEE (2010) pp. 36\u201343."},{"key":"S026357472510221X_ref4","first-page":"4247","volume-title":"2020 Neural Information Processing Systems (NIPS)","author":"Chaplot","year":"2020"},{"key":"S026357472510221X_ref25","doi-asserted-by":"publisher","DOI":"10.1017\/S0263574723000607"},{"key":"S026357472510221X_ref34","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-023-01909-4"},{"key":"S026357472510221X_ref10","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01437-z"},{"key":"S026357472510221X_ref19","first-page":"1479","volume-title":"2016 Neural Information Processing Systems (NIPS)","author":"Bellemare","year":"2016"},{"key":"S026357472510221X_ref28","doi-asserted-by":"crossref","unstructured":"[28] Dornhege, C. and Kleiner, A. . \u201cA frontier-void-based approach for autonomous exploration in 3D.\u201d In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics, IEEE (2011) pp. 351\u2013356.","DOI":"10.1109\/SSRR.2011.6106778"}],"container-title":["Robotica"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S026357472510221X","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T09:03:23Z","timestamp":1759914203000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S026357472510221X\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9]]},"references-count":49,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["S026357472510221X"],"URL":"https:\/\/doi.org\/10.1017\/s026357472510221x","relation":{},"ISSN":["0263-5747","1469-8668"],"issn-type":[{"type":"print","value":"0263-5747"},{"type":"electronic","value":"1469-8668"}],"subject":[],"published":{"date-parts":[[2025,9]]}}}