{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T15:51:46Z","timestamp":1780674706820,"version":"3.54.1"},"reference-count":39,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,15]],"date-time":"2021-02-15T00:00:00Z","timestamp":1613347200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Scientific Research Foundation for the Talents of USTC and the National Natural Science Foundation of China","award":["61971393, 61871361"],"award-info":[{"award-number":["61971393, 61871361"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>It is an essential capability of indoor mobile robots to avoid various kinds of obstacles. Recently, multimodal deep reinforcement learning (DRL) methods have demonstrated great capability for learning control policies in robotics by using different sensors. However, due to the complexity of indoor environment and the heterogeneity of different sensor modalities, it remains an open challenge to obtain reliable and robust multimodal information for obstacle avoidance. In this work, we propose a novel multimodal DRL method with auxiliary task (MDRLAT) for obstacle avoidance of indoor mobile robot. In MDRLAT, a powerful bilinear fusion module is proposed to fully capture the complementary information from two-dimensional (2D) laser range findings and depth images, and the generated multimodal representation is subsequently fed into dueling double deep Q-network to output control commands for mobile robot. In addition, an auxiliary task of velocity estimation is introduced to further improve representation learning in DRL. Experimental results show that MDRLAT achieves remarkable performance in terms of average accumulated reward, convergence speed, and success rate. Moreover, experiments in both virtual and real-world testing environments further demonstrate the outstanding generalization capability of our method.<\/jats:p>","DOI":"10.3390\/s21041363","type":"journal-article","created":{"date-parts":[[2021,2,15]],"date-time":"2021-02-15T03:22:26Z","timestamp":1613359346000},"page":"1363","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Multimodal Deep Reinforcement Learning with Auxiliary Task for Obstacle Avoidance of Indoor Mobile Robot"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3051-1197","authenticated-orcid":false,"given":"Hailuo","family":"Song","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ao","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1448-2157","authenticated-orcid":false,"given":"Tong","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minghui","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230027, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.artint.2014.11.003","article-title":"Deliberation for autonomous robots: A survey","volume":"247","author":"Ingrand","year":"2017","journal-title":"Artif. Intell."},{"key":"ref_2","first-page":"96","article-title":"Mobile robot navigation and obstacle avoidance techniques: A review","volume":"2","author":"Pandey","year":"2017","journal-title":"Int. Rob. Auto. J."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Jin, Z., Shao, Y., So, M., Sable, C., Shlayan, N., and Luchtenburg, D.M. (2019, January 27\u201330). A Multisensor Data Fusion Approach for Simultaneous Localization and Mapping. Proceedings of the IEEE Intelligent Transportation Systems Conference (ITSC), Auckland, New Zealand.","DOI":"10.1109\/ITSC.2019.8916930"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Long, P., Fanl, T., Liao, X., Liu, W., Zhang, H., and Pan, J. (2018, January 21\u201325). Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8461113"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Peng, Y., Qu, D., Zhong, Y., Xie, S., Luo, J., and Gu, J. (2015, January 8\u201310). The obstacle detection and obstacle avoidance algorithm based on 2-d lidar. Proceedings of the IEEE International Conference on Information and Automation, Lijiang, China.","DOI":"10.1109\/ICInfA.2015.7279550"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1007\/s00138-003-0133-2","article-title":"Fusion of laser and visual data for robot motion planning and collision avoidance","volume":"15","author":"Baltzakis","year":"2003","journal-title":"Mach. Vis. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1023\/A:1008966218715","article-title":"A stochastic map building method for mobile robot using 2-d laser range finder","volume":"7","author":"Kwon","year":"1999","journal-title":"Auton. Robot."},{"key":"ref_8","unstructured":"Misono, Y., Goto, Y., Tarutoko, Y., Kobayashi, K., and Watanabe, K. (2007, January 17\u201320). Development of laser rangefinder-based SLAM algorithm for mobile robot navigation. Proceedings of the SICE Annual Conference, Takamatsu, Japan."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Tai, L., Paolo, G., and Liu, M. (2017, January 24\u201328). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202134"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Xie, L., Wang, S., Rosa, S., Markham, A., and Trigoni, N. (2018, January 21\u201325). Learning with training wheels: Speeding up training with a simple controller for deep reinforcement learning. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8461203"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wu, K., Abolfazli Esfahani, M., Yuan, S., and Wang, H. (2018). Learn to steer through deep reinforcement learning. Sensors, 18.","DOI":"10.3390\/s18113650"},{"key":"ref_12","first-page":"22","article-title":"Sensor fusion of laser and stereo vision camera for depth estimation and obstacle avoidance","volume":"1","author":"Kumar","year":"2010","journal-title":"Int. J. Comput. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wu, K., Wang, H., Esfahani, M.A., and Yuan, S. (2019). BND*-DDQN: Learn to Steer Autonomously through Deep Reinforcement Learning. IEEE Trans. Cogn. Dev. Syst.","DOI":"10.3390\/s18113650"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tai, L., and Liu, M. (2016). Towards cognitive exploration through deep reinforcement learning for mobile robots. arXiv.","DOI":"10.1186\/s40638-016-0055-x"},{"key":"ref_15","unstructured":"Xie, L., Wang, S., Markham, A., and Trigoni, N. (2017). Towards monocular vision based obstacle avoidance through deep reinforcement learning. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Choi, J., Park, K., Kim, M., and Seok, S. (2019, January 20\u201324). Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793979"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, X., Li, S., Jia, S., and Xu, C. (2016, January 1\u20133). Mobile robot map building based on laser ranging and kinect. Proceedings of the IEEE International Conference on Information and Automation (ICIA), Ningbo, China.","DOI":"10.1109\/ICInfA.2016.7831932"},{"key":"ref_18","unstructured":"Lara-Guzm\u00e1n, O., Serrano, S.A., Carrillo-L\u00f3pez, D., and Sucar, L.E. (November, January 27). Rgb-d camera and 2d laser integration for robot navigation in dynamic environments. Proceedings of the Mexican International Conference on Artificial Intelligence, Xalapa, Mexico."},{"key":"ref_19","first-page":"126","article-title":"Map building based on a xtion pro live rgbd and a laser sensors","volume":"4","author":"Chavez","year":"2014","journal-title":"J. Inf. Technol. Softw. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Qureshi, A.H., Nakamura, Y., Yoshikawa, Y., and Ishiguro, H. (2016, January 5\u201317). Robot gains social intelligence through multimodal deep reinforcement learning. Proceedings of the IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), Cancun, Mexico.","DOI":"10.1109\/HUMANOIDS.2016.7803357"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lee, M.A., Zhu, Y., Srinivasan, K., Shah, P., Savarese, S., Fei-Fei, L., Garg, A., and Bohg, J. (2019, January 20\u201324). Making sense of vision and touch: Self-supervised learning of multimodal representations for contact-rich tasks. Proceedings of the International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793485"},{"key":"ref_22","unstructured":"Liu, G.-H., Siravuru, A., Prabhakar, S., Veloso, M., and Kantor, G. (2017). Learning end-to-end multimodal sensor policies for autonomous navigation. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"39830","DOI":"10.1109\/ACCESS.2020.2975643","article-title":"A Review on Challenges of Autonomous Mobile Robot and Sensor Fusion Methods","volume":"8","author":"Alatise","year":"2020","journal-title":"IEEE Access"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1613\/jair.1666","article-title":"Risk-sensitive reinforcement learning applied to control under constraints","volume":"24","author":"Geibel","year":"2005","journal-title":"J. Artif. Intell. Res."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2015). Deep reinforcement learning with double q-learning. arXiv.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_26","unstructured":"Nair, V., and Hinton, G.E. (2010, January 21\u201324). Rectified linear units improve restricted boltzmann machines. Proceedings of the International Conference on Machine Learning (ICML), Haifa, Israsel."},{"key":"ref_27","unstructured":"Kim, J.-H., On, K.-W., Lim, W., Kim, J., Ha, J.-W., and Zhang, B.-T. (2016). Hadamard product for low-rank bilinear pooling. arXiv."},{"key":"ref_28","unstructured":"Wang, Z., Schaul, T., Hessel, M., Hasselt, H., Lanctot, M., and Freitas, N. (2016, January 19\u201324). Dueling network architectures for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_29","unstructured":"Lin, X., Baweja, H., Kantor, G., and Held, D. (2019, January 8\u201314). Adaptive Auxiliary Task Weighting for Reinforcement Learning. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kartal, B., Hernandez-Leal, P., and Taylor, M.E. (2019, January 8\u201312). Terminal prediction as an auxiliary task for deep reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, Atlanta, GA, USA.","DOI":"10.1609\/aiide.v15i1.5222"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_32","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_33","unstructured":"Koenig, N., and Howard, A. (October, January 28). Design and use paradigms for gazebo, an open-source multi-robot simulator. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Sendai, Japan."},{"key":"ref_34","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). Tensorflow: A system for large-scale machine learning. Proceedings of the 12th Symposium on Operating Systems Design and Implementation (OSDI), Savannah, GA, USA."},{"key":"ref_35","unstructured":"Rasouli, A., and Tsotsos, J.K. (2017). The Effect of Color Space Selection on Detectability and Discriminability of Colored Objects. arXiv."},{"key":"ref_36","unstructured":"(2021, February 06). Move_Base. Available online: http:\/\/wiki.ros.org\/move_base\/."},{"key":"ref_37","unstructured":"(2021, February 06). Turtlebot2. Available online: https:\/\/www.turtlebot.com\/turtlebot2\/."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"13","DOI":"10.5772\/5715","article-title":"Optical flow based robot obstacle avoidance","volume":"4","author":"Souhila","year":"2007","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_39","unstructured":"OroojlooyJadid, A., and Hajinezhad, D. (2019). A review of cooperative multi-agent deep reinforcement learning. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1363\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:24:20Z","timestamp":1760160260000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1363"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,15]]},"references-count":39,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041363"],"URL":"https:\/\/doi.org\/10.3390\/s21041363","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,15]]}}}