{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,17]],"date-time":"2026-05-17T06:17:37Z","timestamp":1778998657678,"version":"3.51.4"},"reference-count":36,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,7,15]],"date-time":"2022-07-15T00:00:00Z","timestamp":1657843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Belt and Road Special Foundation of the State Key Laboratory of Hydrology-Water Resources and Hydraulic Engineering","award":["2021490811"],"award-info":[{"award-number":["2021490811"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Guidance systems for visually impaired persons have become a popular topic in recent years. Existing guidance systems on the market typically utilize auxiliary tools and methods such as GPS, UWB, or a simple white cane that exploits the user\u2019s single tactile or auditory sense. These guidance methodologies can be inadequate in a complex indoor environment. This paper proposes a multi-sensory guidance system for the visually impaired that can provide tactile and auditory advice using ORB-SLAM and YOLO techniques. Based on an RGB-D camera, the local obstacle avoidance system is realized at the tactile level through point cloud filtering that can inform the user via a vibrating motor. Our proposed method can generate a dense navigation map to implement global obstacle avoidance and path planning for the user through the coordinate transformation. Real-time target detection and a voice-prompt system based on YOLO are also incorporated at the auditory level. We implemented the proposed system as a smart cane. Experiments are performed using four different test scenarios. Experimental results demonstrate that the impediments in the walking path can be reliably located and classified in real-time. Our proposed system can function as a capable auxiliary to help visually impaired people navigate securely by integrating YOLO with ORB-SLAM.<\/jats:p>","DOI":"10.3390\/info13070343","type":"journal-article","created":{"date-parts":[[2022,7,17]],"date-time":"2022-07-17T22:57:06Z","timestamp":1658098626000},"page":"343","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["A Multi-Sensory Guidance System for the Visually Impaired Using YOLO and ORB-SLAM"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1637-1511","authenticated-orcid":false,"given":"Zaipeng","family":"Xie","sequence":"first","affiliation":[{"name":"Key Laboratory of Water Big Data Technology of Ministry of Water Resources, Nanjing 211100, China"},{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhaobin","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yida","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianan","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fangming","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Hohai University, Nanjing 211100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rui, C., Liu, Y., Shen, J., Li, Z., and Xie, Z. (2021, January 17\u201319). A Multi-Sensory Blind Guidance System Based on YOLO and ORB-SLAM. Proceedings of the 2021 IEEE International Conference on Progress in Informatics and Computing (PIC), Shanghai, China.","DOI":"10.1109\/PIC53636.2021.9687018"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"26712","DOI":"10.1109\/ACCESS.2021.3052415","article-title":"Analysis of navigation assistants for blind and visually impaired people: A systematic review","volume":"9","author":"Khan","year":"2021","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Plikynas, D., \u017dvironas, A., Budrionis, A., and Gudauskis, M. (2020). Indoor navigation systems for visually impaired persons: Mapping the features of existing technologies to user needs. Sensors, 20.","DOI":"10.3390\/s20030636"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/j.patrec.2018.10.031","article-title":"Wearable assistive devices for visually impaired: A state of the art survey","volume":"137","author":"Tapu","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2814","DOI":"10.1109\/JSEN.2018.2890423","article-title":"Developing walking assistants for visually impaired people: A review","volume":"19","author":"Islam","year":"2019","journal-title":"IEEE Sensors J."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1046\/j.1475-1313.2002.00016.x","article-title":"Relationship between vision impairment and ability to perform activities of daily living","volume":"22","author":"Haymes","year":"2002","journal-title":"Ophthalmic Physiol. Opt."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1007\/s10071-021-01472-w","article-title":"Human\u2013dog relationships as a working framework for exploring human\u2013robot attachment:  A multidisciplinary review","volume":"24","author":"Krueger","year":"2021","journal-title":"Anim. Cogn."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/j.ajo.2021.04.032","article-title":"Importance and severity dependence of physical activity by GPS-tracked location in glaucoma patients","volume":"230","author":"Mihailovic","year":"2021","journal-title":"Am. J. Ophthalmol."},{"key":"ref_9","unstructured":"Saud, S.N., Raya, L., Abdullah, M.I., and Isa, M.Z.A. Smart navigation aids for blind and vision impairment people. Proceedings of the International Conference on Computational Intelligence in Information System."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sen, A., Sen, K., and Das, J. (2018, January 28\u201331). Ultrasonic blind stick for completely blind people to avoid any kind of obstacles. Proceedings of the 2018 IEEE Sensors, New Delhi, India.","DOI":"10.1109\/ICSENS.2018.8589680"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Gigl, T., Janssen, G.J., Dizdarevic, V., Witrisal, K., and Irahhauten, Z. (2007, January 22\u201322). Analysis of a UWB indoor positioning system based on received signal strength. Proceedings of the 4th IEEE Workshop on Positioning, Navigation and Communication, Hannover, Germany.","DOI":"10.1109\/WPNC.2007.353618"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/s10846-021-01327-z","article-title":"A survey of machine learning techniques for indoor localization and navigation systems","volume":"101","author":"Roy","year":"2021","journal-title":"J. Intell. Robot. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"104032","DOI":"10.1016\/j.engappai.2020.104032","article-title":"SLAM; definition and evolution","volume":"97","author":"Taheri","year":"2021","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"479857","DOI":"10.1155\/2015\/479857","article-title":"A navigation system for the visually impaired: A fusion of vision and depth sensor","volume":"2015","author":"Kanwal","year":"2015","journal-title":"Appl. Bionics Biomech."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1109\/TCE.2018.2812498","article-title":"Virtual-blind-road following-based wearable navigation device for blind people","volume":"64","author":"Bai","year":"2018","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chen, Z., Liu, X., Kojima, M., Huang, Q., and Arai, T. (2021). A wearable navigation device for visually impaired people based on the real-time semantic visual SLAM system. Sensors, 21.","DOI":"10.3390\/s21041536"},{"key":"ref_18","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An incremental improvement. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"27574","DOI":"10.1109\/ACCESS.2020.2966328","article-title":"Research on small target detection in driving scenarios based on improved YOLO network","volume":"8","author":"Xu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1016\/j.robot.2013.07.010","article-title":"Path planning with obstacle avoidance based on visibility binary tree algorithm","volume":"61","author":"Rashid","year":"2013","journal-title":"Robot. Auton. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Real, S., and Araujo, A. (2019). Navigation systems for the blind and visually impaired: Past work, challenges, and open problems. Sensors, 19.","DOI":"10.3390\/s19153404"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3935","DOI":"10.3390\/s20143935","article-title":"A review of technologies and techniques for indoor navigation systems for the visually impaired","volume":"20","author":"Simoes","year":"2020","journal-title":"Sensors"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1080\/10400435.2020.1743381","article-title":"Smartphone-based computer vision travelling aids for blind and visually impaired individuals: A systematic review","volume":"34","author":"Budrionis","year":"2020","journal-title":"Assist. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s12369-017-0403-1","article-title":"An indoor navigation service robot system based on vibration tactile feedback","volume":"9","author":"Peng","year":"2017","journal-title":"Int. J. Soc. Robot."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Kupferstein, E., Rojnirun, H., Findlater, L., and Azenkot, S. The effectiveness of visual and audio wayfinding guidance on smartglasses for people with low vision. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems.","DOI":"10.1145\/3313831.3376516"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Romeo, K., Pissaloux, E., Gay, S.L., Truong, N.T., and Djoussouf, L. (2022). The MAPS: Toward a novel mobility assistance system for visually impaired people. Sensors, 22.","DOI":"10.3390\/s22093316"},{"key":"ref_27","unstructured":"Metsiritrakul, K., Suchato, A., and Punyabukkana, P. (2017, January 25\u201328). Obstacle avoidance feedback system for the blind using stereo sound. Proceedings of the 11th International Convention on Rehabilitation Engineering and Assistive Technology, Kobe, Japan."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/MCE.2018.2797741","article-title":"Improving mobility for the visually impaired: A wearable indoor positioning system based on visual markers","volume":"7","author":"Lee","year":"2018","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Presti, G., Ahmetovic, D., Ducci, M., Bernareggi, C., Ludovico, L., Barate, A., Avanzini, F., and Mascetti, S. (2019, January 28\u201330). WatchOut: Obstacle sonification for people with visual impairment or blindness. Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS), Pittsburgh, PA, USA.","DOI":"10.1145\/3308561.3353779"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"10199","DOI":"10.1109\/JSEN.2020.2990609","article-title":"Design and implementation of an intelligent assistive system for visually impaired people for aerial obstacle avoidance and fall detection","volume":"20","author":"Chang","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"13428","DOI":"10.1109\/ACCESS.2022.3146320","article-title":"Smart assistive system for visually impaired people obstruction avoidance through object detection and classification","volume":"10","author":"Masud","year":"2022","journal-title":"IEEE Access"},{"key":"ref_32","unstructured":"Qiu, H., Wang, C., Wang, J., Wang, N., and Zeng, W. (November, January 27). Cross view fusion for 3D human pose estimation. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ashish, D., Munjal, S., Mani, M., and Srivastava, S. (2021). Path finding algorithms. Emerging Technologies in Data Mining and Information Security, Springer.","DOI":"10.1007\/978-981-15-9927-9_33"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ahn, M.S., Chae, H., Noh, D., Nam, H., and Hong, D. (2019, January 24\u201327). Analysis and noise modeling of the intel realsense D435 for mobile robots. Proceedings of the 16th IEEE International Conference on Ubiquitous Robots (UR), Jeju, Korea.","DOI":"10.1109\/URAI.2019.8768489"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"eabm6074","DOI":"10.1126\/scirobotics.abm6074","article-title":"Robot Operating System 2: Design, architecture, and uses in the wild","volume":"7","author":"Macenski","year":"2022","journal-title":"Sci. Robot."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016, January 27\u201330). You only look once: Unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.91"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/7\/343\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:51:10Z","timestamp":1760140270000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/7\/343"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,15]]},"references-count":36,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,7]]}},"alternative-id":["info13070343"],"URL":"https:\/\/doi.org\/10.3390\/info13070343","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,15]]}}}