{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,16]],"date-time":"2026-01-16T01:12:12Z","timestamp":1768525932957,"version":"3.49.0"},"reference-count":25,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,1]],"date-time":"2021-01-01T00:00:00Z","timestamp":1609459200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004281","name":"Narodowe Centrum Nauki","doi-asserted-by":"publisher","award":["2016\/23\/B\/HS6\/00275"],"award-info":[{"award-number":["2016\/23\/B\/HS6\/00275"]}],"id":[{"id":"10.13039\/501100004281","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Detecting characteristics of 3D scenes is considered one of the biggest challenges for visually impaired people. This ability is nonetheless crucial for orientation and navigation in the natural environment. Although there are several Electronic Travel Aids aiming at enhancing orientation and mobility for the blind, only a few of them combine passing both 2D and 3D information, including colour. Moreover, existing devices either focus on a small part of an image or allow interpretation of a mere few points in the field of view. Here, we propose a concept of visual echolocation with integrated colour sonification as an extension of Colorophone\u2014an assistive device for visually impaired people. The concept aims at mimicking the process of echolocation and thus provides 2D, 3D and additionally colour information of the whole scene. Even though the final implementation will be realised by a 3D camera, it is first simulated, as a proof of concept, by using VIRCO\u2014a Virtual Reality training and evaluation system for Colorophone. The first experiments showed that it is possible to sonify colour and distance of the whole scene, which opens up a possibility to implement the developed algorithm on a hardware-based stereo camera platform. An introductory user evaluation of the system has been conducted in order to assess the effectiveness of the proposed solution for perceiving distance, position and colour of the objects placed in Virtual Reality.<\/jats:p>","DOI":"10.3390\/s21010237","type":"journal-article","created":{"date-parts":[[2021,1,1]],"date-time":"2021-01-01T22:35:48Z","timestamp":1609540548000},"page":"237","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Visual Echolocation Concept for the Colorophone Sensory Substitution Device Using Virtual Reality"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6421-978X","authenticated-orcid":false,"given":"Patrycja","family":"Bizo\u0144-Angov","sequence":"first","affiliation":[{"name":"Consciousness Lab, Institute of Psychology, Jagiellonian University, 30-060 Krak\u00f3w, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4761-9006","authenticated-orcid":false,"given":"Dominik","family":"Osi\u0144ski","sequence":"additional","affiliation":[{"name":"Department of Electronic Systems, Norwegian University of Science and Technology, NO-7491 Trondheim, Norway"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7347-2696","authenticated-orcid":false,"given":"Micha\u0142","family":"Wierzcho\u0144","sequence":"additional","affiliation":[{"name":"Consciousness Lab, Institute of Psychology, Jagiellonian University, 30-060 Krak\u00f3w, Poland"},{"name":"Jagiellonian Human-Centered Artificial Intelligence Laboratory, Jagiellonian University, 30-348 Krak\u00f3w, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3513-0601","authenticated-orcid":false,"given":"Jaros\u0142aw","family":"Konieczny","sequence":"additional","affiliation":[{"name":"Department of Process Control, AGH University of Science and Technology, 30-059 Krak\u00f3w, Poland"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1142\/S0219635205000914","article-title":"Brainport: An Alternative Input to the Brain","volume":"4","author":"Danilov","year":"2006","journal-title":"J. Integr. Neurosci."},{"key":"ref_2","unstructured":"(2020, October 26). The vOICe Home Page. Available online: https:\/\/www.seeingwithsound.com\/."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Osinski, D., and Hjelme, D.R. (2018, January 4\u20136). A Sensory Substitution Device Inspired by the Human Visual System. Proceedings of the 11th International Conference on Human System Interaction (HSI), Gda\u0144sk, Poland.","DOI":"10.1109\/HSI.2018.8431078"},{"key":"ref_4","first-page":"247","article-title":"EyeMusic: Introducing a \u201cVisual\u201d Colorful Experience for the Blind Using Auditory Sensory Substitution","volume":"32","author":"Abboud","year":"2014","journal-title":"Restor. Neurol. Neurosci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Caraiman, S., Zvoristeanu, O., Burlacu, A., and Herghelegiu, P. (2019). Stereo Vision Based Sensory Substitution for the Visually Impaired. Sensors, 19.","DOI":"10.3390\/s19122771"},{"key":"ref_6","first-page":"696","article-title":"Naviton\u2014A Prototype Mobility Aid for Auditory Presentation of Three-Dimensional Scenes to the Visually Impaired","volume":"60","author":"Bujacz","year":"2012","journal-title":"J. Audio Eng. Soc."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1108\/JAT-08-2013-0025","article-title":"See ColOr: An Extended Sensory Substitution Device for the Visually Impaired","volume":"8","author":"Gomez","year":"2014","journal-title":"J. Assist. Technol."},{"key":"ref_8","unstructured":"(2020, October 26). SoundSight Home Page. Available online: https:\/\/soundsight.co.uk\/."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mira, J., and S\u00e1nchez-Andr\u00e9s, J.V. (1999). Development of a New Space Perception System for Blind People, Based on the Creation of a Virtual Acoustic Space. Engineering Applications of Bio-Inspired Artificial Neural Networks, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/BFb0100465"},{"key":"ref_10","first-page":"147","article-title":"Design and Development of an Indoor Navigation and Object Identification System for the Blind","volume":"77\u201378","author":"Hub","year":"2003","journal-title":"Assoc. Comput. Mach."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.heares.2014.01.010","article-title":"A summary of research investigating echolocation abilities of blind and sighted humans","volume":"310","author":"Kolarik","year":"2014","journal-title":"Hear. Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1109\/10.81565","article-title":"A blind mobility aid modeled after echolocation of bats","volume":"38","author":"Ifukube","year":"1991","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_13","first-page":"1526","article-title":"A Device for Human Ultrasonic Echolocation","volume":"62","author":"Teng","year":"1991","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_14","unstructured":"(2020, October 26). Colorophone Home Page. Available online: https:\/\/www.colorophone.com\/."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Bizo\u0144, P., Gwia\u017adzi\u0144ski, P., Hat, K., Osi\u0144ski, D., Reuter, M., Fauchald, E., Verma, S., Skylstad, T.K., S\u00f8rli, S., and Thorsen, J.E. (2019, January 19\u201322). VIRCO: A Virtual Reality Tool for Long-Term Training and Evaluation of the Cognitive Skills Development in an Interactive Sensory Substitution Environment. Proceedings of the Joint IEEE 9th International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob), Oslo, Norway.","DOI":"10.1109\/DEVLRN.2019.8850696"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1068\/p6898","article-title":"Human Echolocation: Pitch versus Loudness Information","volume":"40","author":"Schenkman","year":"2011","journal-title":"Perception"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1126\/science.137.3528.399","article-title":"Sonar System of the Blind","volume":"137","author":"Kellogg","year":"1962","journal-title":"Science"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1126\/science.155.3763.656","article-title":"Human Echo Perception","volume":"155","author":"Rice","year":"1967","journal-title":"Science"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"483","DOI":"10.1007\/s00221-011-2951-1","article-title":"Ultrafine Spatial Acuity of Blind Expert Human Echolocators","volume":"216","author":"Teng","year":"2012","journal-title":"Exp. Brain Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"133","DOI":"10.2307\/1416946","article-title":"\u201cFacial Vision\u201d: The Perception of Obstacles by the Blind","volume":"57","author":"Supa","year":"1944","journal-title":"Am. J. Psychol."},{"key":"ref_21","unstructured":"(2020, October 26). The Colorophone VE Example Page. Available online: https:\/\/www.colorophone.com\/visual-echolocation-boxes."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hamilton-Fletcher, G., Alvarez, J., Obrist, M., and Ward, J. (2020, October 26). SoundSight: A Mobile Sensory Substitution Device That Sonifies Colour, Distance, and Temperature. Available online: https:\/\/psyarxiv.com\/xsjwr\/.","DOI":"10.31234\/osf.io\/xsjwr"},{"key":"ref_23","unstructured":"(2020, October 26). National Instruments Home Page. Available online: https:\/\/www.ni.com."},{"key":"ref_24","unstructured":"(2020, October 26). Unity Home Page. Available online: https:\/\/unity.com\/."},{"key":"ref_25","unstructured":"Fritsch, D. (2011). Semi-Global Matching-Motivation, Developments and Applications. Photogrammetric Week 11, Wichmann."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/1\/237\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:06:11Z","timestamp":1760159171000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/1\/237"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,1]]},"references-count":25,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21010237"],"URL":"https:\/\/doi.org\/10.3390\/s21010237","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,1]]}}}