{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T16:34:21Z","timestamp":1778344461193,"version":"3.51.4"},"reference-count":51,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,27]],"date-time":"2023-12-27T00:00:00Z","timestamp":1703635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Conseil R\u00e9gional de Bourgogne Franche-Comt\u00e9, France, and the Fond Europ\u00e9en de D\u00e9veloppement R\u00e9gional (FEDER)","award":["BG0027904"],"award-info":[{"award-number":["BG0027904"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Blindness affects millions of people worldwide, leading to difficulties in daily travel and a loss of independence due to a lack of spatial information. This article proposes a new navigation aid to help people with severe blindness reach their destination. Blind people are guided by a short 3D spatialised sound that indicates the target point to follow. This sound is combined with other sonified information on potential obstacles in the vicinity. The proposed system is based on inertial sensors, GPS data, and the cartographic knowledge of pedestrian paths to define the trajectory. In addition, visual clues are used to refine the trajectory with ground floor information and obstacle information using a camera to provide 3D spatial information. The proposed method is based on a deep learning approach. The different neural networks used in this approach are evaluated on datasets that regroup navigations from pedestrians\u2019 point-of-view. This method achieves low latency and real-time processing without relying on remote connections, instead using a low-power embedded GPU target and a multithreaded approach for video processing, sound generation, and acquisition. This system could significantly improve the quality of life and autonomy of blind people, allowing them to reliably and efficiently navigate in their environment.<\/jats:p>","DOI":"10.3390\/s24010166","type":"journal-article","created":{"date-parts":[[2023,12,28]],"date-time":"2023-12-28T03:30:34Z","timestamp":1703734234000},"page":"166","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Outdoor Navigation Assistive System Based on Robust and Real-Time Visual\u2013Auditory Substitution Approach"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-5257-1277","authenticated-orcid":false,"given":"Florian","family":"Scalvini","sequence":"first","affiliation":[{"name":"Laboratory ImViA EA 7535, Universit\u00e9 de Bourgogne, 21078 Dijon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3666-7846","authenticated-orcid":false,"given":"Camille","family":"Bordeau","sequence":"additional","affiliation":[{"name":"LEAD, CNRS UMR 5022, Universit\u00e9 de Bourgogne, 21078 Dijon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0653-3177","authenticated-orcid":false,"given":"Maxime","family":"Ambard","sequence":"additional","affiliation":[{"name":"LEAD, CNRS UMR 5022, Universit\u00e9 de Bourgogne, 21078 Dijon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1651-4837","authenticated-orcid":false,"given":"Cyrille","family":"Migniot","sequence":"additional","affiliation":[{"name":"Laboratory ImViA EA 7535, Universit\u00e9 de Bourgogne, 21078 Dijon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0907-4140","authenticated-orcid":false,"given":"Julien","family":"Dubois","sequence":"additional","affiliation":[{"name":"Laboratory ImViA EA 7535, Universit\u00e9 de Bourgogne, 21078 Dijon, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e130","DOI":"10.1016\/S2214-109X(20)30425-3","article-title":"Trends in prevalence of blindness and distance and near vision impairment over 30 years: An analysis for the Global Burden of Disease Study","volume":"9","author":"Bourne","year":"2021","journal-title":"Lancet Glob. Health"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1449","DOI":"10.1111\/cdev.12397","article-title":"Hearing Where the Eyes See: Children Use an Irrelevant Visual Cue When Localizing Sounds","volume":"86","author":"Petrini","year":"2015","journal-title":"Child Dev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1073\/pnas.1011843108","article-title":"Visual influence on path integration in darkness indicates a multimodal representation of large-scale space","volume":"108","author":"Tcheang","year":"2011","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"150","DOI":"10.1038\/356150a0","article-title":"Receptive field dynamics in adult primary visual cortex","volume":"356","author":"Gilbert","year":"1992","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/0006-8993(79)90349-4","article-title":"Synaptic density in human frontal cortex\u2014Developmental changes and effects of aging","volume":"163","author":"Peter","year":"1979","journal-title":"Brain Res."},{"key":"ref_6","first-page":"318","article-title":"Social Isolation of the Blind: An Underrated Aspect of Disability and Dependency","volume":"32","author":"Macfarland","year":"1966","journal-title":"J. Rehabil."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1080\/09286580801939353","article-title":"The Economic Costs and Benefits of Dog Guides for the Blind","volume":"15","author":"Wirth","year":"2008","journal-title":"Ophthalmic Epidemiol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Elmannai, W., and Elleithy, K. (2017). Sensor-Based Assistive Devices for Visually-Impaired People: Current Status, Challenges, and Future Directions. Sensors, 17.","DOI":"10.3390\/s17030565"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lowery, A.J., Rosenfeld, J.V., Lewis, P.M., Browne, D., Mohan, A., Brunton, E., Yan, E., Maller, J., Mann, C., and Rajan, R. (2015, January 25\u201329). Restoration of vision using wireless cortical implants: The Monash Vision Group project. Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society, Milan, Italy.","DOI":"10.1109\/EMBC.2015.7318543"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Szpiro, S., Zhao, Y., and Azenkot, S. (2016, January 12\u201316). Finding a store, searching for a product: A study of daily challenges of low vision people. Proceedings of the International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971723"},{"key":"ref_11","unstructured":"Durette, B., Louveton, N., Alleysson, D., and H\u00e9rault, J. (2008, January 17\u201318). Visuo-auditory sensory substitution for mobility assistance: Testing TheVIBE. Proceedings of the Workshop on Computer Vision Applications for the Visually Impaired, James Coughlan and Roberto Manduchi, Marseille, France."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Kammoun, S., Mac\u00e9, M.J.M., Oriola, B., and Jouffrais, C. (2011, January 5\u20139). Toward a Better Guidance in Wearable Electronic Orientation Aids. Proceedings of the Human\u2013Computer Interaction\u2013INTERACT 2011: 13th IFIP TC 13 International Conference, Lisbon, Portugal.","DOI":"10.1007\/978-3-642-23768-3_98"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1109\/TSMCC.2009.2021255","article-title":"Wearable Obstacle Avoidance Electronic Travel Aids for Blind: A Survey","volume":"40","author":"Dakopoulos","year":"2010","journal-title":"IEEE Trans. Syst. Man Cybern. Part (Appl. Rev.)"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ivanov, R. (2010, January 17\u201318). Indoor navigation system for visually impaired. Proceedings of the International Conference on Computer Systems and Technologies, Sofia, Bulgaria.","DOI":"10.1145\/1839379.1839405"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ahmetovic, D., Gleason, C., Ruan, C., Kitani, K., Takagi, H., and Asakawa, C. (2016, January 6\u20139). NavCog: A navigational cognitive assistant for the blind. Proceedings of the International Conference on Human\u2013Computer Interaction with Mobile Devices and Services, Florence, Italy.","DOI":"10.1145\/2935334.2935361"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cheraghi, S.A., Namboodiri, V., and Walker, L. (2017, January 13\u201317). GuideBeacon: Beacon-based indoor wayfinding for the blind, visually impaired, and disoriented. Proceedings of the International Conference on Pervasive Computing and Communications, Kona, HI, USA.","DOI":"10.1109\/PERCOM.2017.7917858"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Scalvini, F., Bordeau, C., Ambard, M., Migniot, C., Argon, S., and Dubois, J. (2022, January 19\u201321). Visual-auditory substitution device for indoor navigation based on fast visual marker detection. Proceedings of the International Conference on Signal-Image Technology & Internet-Based Systems, Dijon, France.","DOI":"10.1109\/SITIS57111.2022.00029"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Legge, G.E., Beckmann, P.J., Tjan, B.S., Havey, G., Kramer, K., Rolkosky, D., Gage, R., Chen, M., Puchakayala, S., and Rangarajan, A. (2013). Indoor Navigation by People with Visual Impairment Using a Digital Sign System. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0076783"},{"key":"ref_19","first-page":"27","article-title":"A context aware handheld wayfinding system for individuals with cognitive impairments","volume":"10","author":"Chang","year":"2008","journal-title":"Int. Conf. Comput. Access."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1142\/S0219467810003937","article-title":"Outdoor\/indoor vision-based localization for blind pedestrian navigation assistance","volume":"10","author":"Treuillet","year":"2010","journal-title":"Int. J. Image Graph."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1567","DOI":"10.1109\/TMM.2020.3001500","article-title":"V-Eye: A Vision-based Navigation System for the Visually Impaired","volume":"23","author":"Duh","year":"2020","journal-title":"Trans. Multimed."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Oliveira, J.F. (2013, January 1\u20134). The path force feedback belt. Proceedings of the International Conference on Information Technology in Asia, Kota Samarahan, Malaysia.","DOI":"10.1109\/CITA.2013.6637564"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sahoo, N., Lin, H.W., and Chang, Y.H. (2019). Design and Implementation of a Walking Stick Aid for Visually Challenged People. Sensors, 19.","DOI":"10.3390\/s19010130"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Brilhault, A., Kammoun, S., Gutierrez, O., Truillet, P., and Jouffrais, C. (2011, January 7\u201310). Fusion of Artificial Vision and GPS to Improve Blind Pedestrian Positioning. Proceedings of the International Conference on New Technologies, Mobility and Security, Paris, France.","DOI":"10.1109\/NTMS.2011.5721061"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Vel\u00e1zquez, R., Pissaloux, E., Rodrigo, P., Carrasco, M., Giannoccaro, N., and Lay-Ekuakille, A. (2018). An Outdoor Navigation System for Blind Pedestrians Using GPS and Tactile-Foot Feedback. Appl. Sci., 8.","DOI":"10.3390\/app8040578"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1511","DOI":"10.1177\/2399808320933907","article-title":"Route planning for blind pedestrians using OpenStreetMap","volume":"48","author":"Cohen","year":"2021","journal-title":"Environ. Plan. Urban Anal. City Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1007\/BF01386390","article-title":"A Note on Two Problems in Connexion with Graphs","volume":"1","author":"Dijkstra","year":"1959","journal-title":"Numer. Math."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chaurasia, S.R., and Kavitha, K.N. (2014, January 27\u201328). An electronic walking stick for blinds. Proceedings of the International Conference on Information Communication and Embedded Systems, Chennai, India.","DOI":"10.1109\/ICICES.2014.7033988"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Scalvini, F., Bordeau, C., Ambard, M., Migniot, C., and Dubois, J. (2022, January 23\u201327). Low-Latency Human\u2013Computer Auditory Interface Based on Real-Time Vision Analysis. Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Singapore.","DOI":"10.1109\/ICASSP43922.2022.9747094"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"See, A.R., Sasing, B.G., and Advincula, W.D. (2022). A Smartphone-Based Mobility Assistant Using Depth Imaging for Visually Impaired and Blind. Appl. Sci., 12.","DOI":"10.3390\/app12062802"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Younis, O., Al-Nuaimy, W., Rowe, F., and Alomari, M. (2019). A Smart Context-Aware Hazard Attention System to Help People with Peripheral Vision Loss. Sensors, 19.","DOI":"10.3390\/s19071630"},{"key":"ref_32","unstructured":"Shirbhate, S., Rathod, B.A., Nagarjuna, S., and Rao, D.N. (2021, January 15\u201316). Assistive Outdoor Navigation Application for Visually Impaired. Proceedings of the Advancements in Mechanical, Electronics and Electrical Engineering, Bhopal, India."},{"key":"ref_33","unstructured":"Shadi, S., Hadi, S., Nazari, M.A., and Hardt, W. (2019, January 12\u201314). Outdoor Navigation for Visually Impaired based on Deep Learning. Proceedings of the Actual Problems of System and Software Engineering-APSSE, Moscow, Russia."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"33029","DOI":"10.1109\/ACCESS.2018.2817164","article-title":"A Highly Accurate and Reliable Data Fusion Framework for Guiding the Visually Impaired","volume":"6","author":"Elmannai","year":"2018","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Lin, Y., Wang, K., Yi, W., and Lian, S. (2019, January 27\u201328). Deep Learning Based Wearable Assistive System for Visually Impaired People. Proceedings of the International Conference on Computer Vision Workshop, Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00312"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Poggi, M., and Mattoccia, S. (2016, January 27\u201330). A wearable mobility aid for the visually impaired based on embedded 3D vision and deep learning. Proceedings of the Symposium on Computers and Communication, Messina, Italy.","DOI":"10.1109\/ISCC.2016.7543741"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1279","DOI":"10.1109\/10.720206","article-title":"A real-time experimental prototype for enhancement of vision rehabilitation using auditory substitution","volume":"45","author":"Capelle","year":"1998","journal-title":"Trans. Biomed. Eng."},{"key":"ref_38","unstructured":"Nie, M., Ren, J., Li, Z., Niu, J., Qiu, Y., Zhu, Y., and Tong, S. (2009, January 3\u20136). SoundView: An auditory guidance system based on environment understanding for the visually impaired people. Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ferrand, S., Alouges, F., and Aussal, M. (2018, January 11\u201313). An Augmented Reality Audio Device Helping Blind People Navigation. Proceedings of the 13th International Conference on Computers Helping People with Special Needs, Linz, Austria.","DOI":"10.1007\/978-3-319-94274-2_5"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Planinec, V., Reijniers, J., Horvat, M., Peremans, H., and Jambro\u0161i\u0107, K. (2023). The Accuracy of Dynamic Sound Source Localization and Recognition Ability of Individual Head-Related Transfer Functions in Binaural Audio Systems with Head Tracking. Appl. Sci., 13.","DOI":"10.3390\/app13095254"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Mashiata, M., Ali, T., Das, P., Tasneem, Z., Badal, M.F.R., Sarker, S.K., Hasan, M.M., Abhi, S.H., Islam, M.R., and Ali, M.F. (2022). Towards assisting visually impaired individuals: A review on current status and future prospects. Biosens. Bioelectron. X, 12.","DOI":"10.1016\/j.biosx.2022.100265"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","article-title":"A Formal Basis for the Heuristic Determination of Minimum Cost Paths","volume":"4","author":"Hart","year":"1968","journal-title":"Trans. Syst. Sci. Cybern."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Bewley, A., Ge, Z., Ott, L., Ramos, F., and Upcroft, B. (2016, January 25\u201328). Simple online and realtime tracking. Proceedings of the International Conference on Image Processing, Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7533003"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1002\/nav.3800020109","article-title":"The Hungarian method for the assignment problem","volume":"2","author":"Kuhn","year":"1955","journal-title":"Nav. Res. Logist."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1115\/1.3662552","article-title":"A New Approach to Linear Filtering and Prediction Problems","volume":"82","author":"Kalman","year":"1960","journal-title":"J. Basic Eng."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1079998","DOI":"10.3389\/fpsyg.2023.1079998","article-title":"Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution","volume":"14","author":"Bordeau","year":"2023","journal-title":"Front. Psychol."},{"key":"ref_47","unstructured":"Algazi, V., Duda, R., Thompson, D., and Avendano, C. (2001, January 24). The CIPIC HRTF database. Proceedings of the Workshop on the Applications of Signal Processing to Audio and Acoustics, New Platz, NY, USA."},{"key":"ref_48","unstructured":"Park, K., Oh, Y., Ham, S., Joo, K., Kim, H., Kum, H., and Kweon, I.S. (January, January 24). SideGuide: A Large-scale Sidewalk Dataset for Guiding Impaired People. Proceedings of the International Conference on Intelligent Robots and Systems, Las Vegas, NV, USA."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Scalvini, F., Bordeau, C., Ambard, M., Migniot, C., Vergnaud, M., and Dubois, J. (2023). uB-VisioGeoloc: An Image Sequences Dataset of Pedestrian Navigation Including Geolocalised-Inertial Information and Spatial Sound Rendering of the Urban Environment\u2019s Obstacles, Imagerie et Vision Artificielle (UR 7535). Available online: https:\/\/search-data.ubfc.fr\/FR-13002091000019-2023-07-13_uB-VisioGeoloc.html.","DOI":"10.2139\/ssrn.4521793"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Neuhold, G., Ollmann, T., Bul\u00f2, S.R., and Kontschieder, P. (2017, January 22\u201329). The Mapillary Vistas Dataset for Semantic Understanding of Street Scenes. Proceedings of the International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.534"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1007\/s11554-014-0456-4","article-title":"Fast prototyping of a SoC-based smart-camera: A real-time fall detection case study","volume":"12","author":"Senouci","year":"2016","journal-title":"J. Real-Time Image Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/166\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:43:00Z","timestamp":1760132580000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/166"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,27]]},"references-count":51,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24010166"],"URL":"https:\/\/doi.org\/10.3390\/s24010166","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,27]]}}}