{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T20:46:35Z","timestamp":1789505195444,"version":"build-2803163510"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T00:00:00Z","timestamp":1731542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Sign languages are one of the main rehabilitation methods for dealing with hearing loss. Like any other language, the geographical location will influence on how signs are made. Particularly in Colombia, the hard of hearing population is lacking from education in the Colombian Sign Language, mainly due of the reduce number of interpreters in the educational sector. To help mitigate this problem, Machine Learning binded to data gloves or Computer Vision technologies have emerged to be the accessory of sign translation systems and educational tools, however, in Colombia the presence of this solutions is scarce. On the other hand, humanoid robots such as the NAO have shown significant results when used to support a learning process. This paper proposes a performance evaluation for the design of an activity to support the learning process of all the 11 color-based signs from the Colombian Sign Language. Which consists of an evaluation method with two modes activated through user interaction, the first mode will allow to choose the color sign to be evaluated, and the second will decide randomly the color sign. To achieve this, MediaPipe tool was used to extract torso and hand coordinates, which were the input for a Neural Network. The performance of the Neural Network was evaluated running continuously in two scenarios, first, video capture from the webcam of the computer which showed an overall F1 score of 91.6% and a prediction time of 85.2 m, second, wireless video streaming with NAO H25 V6 camera which had an F1 score of 93.8% and a prediction time of 2.29 s. In addition, we took advantage of the joint redundancy that NAO H25 V6 has, since with its 25 degrees of freedom we were able to use gestures that created nonverbal human-robot interactions, which may be useful in future works where we want to implement this activity with a deaf community.<\/jats:p>","DOI":"10.3389\/frobt.2024.1475069","type":"journal-article","created":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T06:22:05Z","timestamp":1731565325000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Learning signs with NAO: humanoid robot as a tool for helping to learn Colombian Sign Language"],"prefix":"10.3389","volume":"11","author":[{"given":"Juan E.","family":"Mora-Zarate","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Claudia L.","family":"Garz\u00f3n-Castro","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jorge A.","family":"Castellanos Rivillas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,11,14]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1007\/s12008-022-01065-y","article-title":"NAO robots as context to teach numerical methods","volume":"16","author":"Cruz-Ram\u00edrez","year":"2022","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"B2","article-title":"Oferta educativa","year":"2024"},{"key":"B3","doi-asserted-by":"publisher","first-page":"89","DOI":"10.3991\/IJOE.V20I03.46765","article-title":"Design of a sign language-to-natural language translator using artificial intelligence","volume":"20","author":"Gonzalez","year":"2024","journal-title":"Int. J. Online Biomed. Eng."},{"key":"B4","doi-asserted-by":"publisher","first-page":"7","DOI":"10.3390\/TECHNOLOGIES12010007","article-title":"Towards a bidirectional mexican sign language\u2013spanish translation system: a deep learning approach","volume":"12","author":"Gonz\u00e1lez-Rodr\u00edguez","year":"2024","journal-title":"Technologies"},{"key":"B5","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1007\/s12369-019-00609-9","article-title":"Sign recognition system for an assistive robot sign tutor for children","volume":"12","author":"G\u00fcrp\u0131nar","year":"2020","journal-title":"Int. J. Soc. Robotics"},{"key":"B6","first-page":"897","article-title":"Development and validation of a motion dictionary to create emotional gestures for the NAO robot","author":"Hellou","year":"2023"},{"key":"B7","doi-asserted-by":"publisher","first-page":"645","DOI":"10.1007\/s12369-021-00818-1","article-title":"One-shot learning from demonstration approach toward a reciprocal sign language-based HRI","volume":"16","author":"Hosseini","year":"2024","journal-title":"Int. J. Soc. Robotics"},{"key":"B8","doi-asserted-by":"publisher","first-page":"265","DOI":"10.20965\/JACIII.2024.P0265","article-title":"Applying LSTM and GRU methods to recognize and interpret hand gestures, poses, and face-based sign language in real time","volume":"28","author":"Ilham","year":"2024","journal-title":"J. Adv. Comput. Intell. Intelligent Inf."},{"key":"B9","volume-title":"Caracterizaci\u00f3n de ciudadanos","year":"2020"},{"key":"B10","doi-asserted-by":"publisher","first-page":"6693","DOI":"10.3390\/S23156693","article-title":"Dataglove for sign language recognition of people with hearing and speech impairment via wearable inertial sensors","volume":"23","author":"Ji","year":"2023","journal-title":"Sensors"},{"key":"B11","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1007\/s44163-024-00113-8","article-title":"Using LSTM to translate thai sign language to text in real time","volume":"4","author":"Jintanachaiwat","year":"2024","journal-title":"Discov. Artif. Intell."},{"key":"B12","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1007\/s42979-024-02776-7","article-title":"GIDSL: Indian-Gujarati isolated dynamic sign language recognition using deep learning","volume":"5","author":"Joshi","year":"2024","journal-title":"SN Comput. Sci."},{"key":"B13","doi-asserted-by":"publisher","first-page":"3228","DOI":"10.3390\/APP14083228","article-title":"A performance comparison of japanese sign language recognition with ViT and CNN using angular features","volume":"14","author":"Kondo","year":"2024","journal-title":"Appl. Sci."},{"key":"B14","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1080\/10494820.2022.2086577","article-title":"Humanoid robot as a tutor in a team-based training activity","volume":"32","author":"Kurtz","year":"2024","journal-title":"Interact. Learn. Environ."},{"key":"B15","doi-asserted-by":"publisher","first-page":"1168888","DOI":"10.3389\/fnins.2023.1168888","article-title":"A multimodal human-robot sign language interaction framework applied in social robots","volume":"17","author":"Li","year":"2023","journal-title":"Front. Neurosci."},{"key":"B16","doi-asserted-by":"publisher","first-page":"72","DOI":"10.3390\/COMPUTATION12040072","article-title":"Progression learning convolution neural model-based sign language recognition using wearable glove devices","volume":"12","author":"Liang","year":"2024","journal-title":"Computation"},{"key":"B17","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s10846-018-0860-2","article-title":"Design and realization of a sign language educational humanoid robot","volume":"95","author":"Meghdari","year":"2019","journal-title":"J. Intelligent Robotic Syst. Theory Appl."},{"key":"B18","doi-asserted-by":"publisher","first-page":"833","DOI":"10.3390\/MACHINES11080833","article-title":"Augmenting mobile app with NAO robot for autism education","volume":"11","author":"Mutawa","year":"2023","journal-title":"Mach. 2023"},{"key":"B19","doi-asserted-by":"publisher","first-page":"2807","DOI":"10.1109\/JSEN.2023.3339963","article-title":"Development of an OTDR-based hand glove optical sensor for sign language prediction","volume":"24","author":"Pal","year":"2024","journal-title":"IEEE Sensors J."},{"key":"B20","doi-asserted-by":"publisher","first-page":"103201","DOI":"10.1016\/J.BSPC.2021.103201","article-title":"Automatic sign language recognition based on accelerometry and surface electromyography signals: a study for Colombian Sign Language","volume":"71","author":"Pereira-Montiel","year":"2022","journal-title":"Biomed. Signal Process. Control"},{"key":"B21","doi-asserted-by":"publisher","first-page":"22261","DOI":"10.1007\/s11042-023-15583-8","article-title":"Real-time computer vision-based gestures recognition system for bangla sign language using multiple linguistic features analysis","volume":"83","author":"Rahaman","year":"2024","journal-title":"Multimedia Tools Appl."},{"key":"B22","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/S10462-024-10895-Z","article-title":"From rule-based models to deep learning transformers architectures for natural language processing and sign language translation systems: survey, taxonomy and performance evaluation","volume":"57","author":"Shahin","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"B23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3432211","article-title":"Teaching american sign language in mixed reality","volume":"4","author":"Shao","year":"2020","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"B24","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/s11554-024-01435-7","article-title":"Real-time attention-based embedded LSTM for dynamic sign language recognition on edge devices","volume":"21","author":"Sharma","year":"2024","journal-title":"J. Real-Time Image Process."},{"key":"B25","doi-asserted-by":"publisher","first-page":"181340","DOI":"10.1109\/ACCESS.2020.3028072","article-title":"Autsl: a large scale multi-modal Turkish sign language dataset and baseline methods","volume":"8","author":"Sincan","year":"2020","journal-title":"IEEE Access"},{"key":"B26","doi-asserted-by":"publisher","first-page":"25","DOI":"10.3390\/BDCC8030025","article-title":"Sign-to-Text translation from panamanian sign language to Spanish in continuous capture mode with deep neural networks","volume":"8","author":"Teran-Quezada","year":"2024","journal-title":"Big Data Cognitive Comput. 2024"},{"key":"B27","doi-asserted-by":"publisher","first-page":"476","DOI":"10.1145\/3568294.3580130","article-title":"Let\u2019s roll together: children helping a robot play a dice game","volume":"20","author":"Timmerman","year":"2023","journal-title":"ACM\/IEEE Int. Conf. Human-Robot Interact."},{"key":"B28","article-title":"Curso b\u00e1sico lengua de se\u00f1as colombiana","year":"2024","journal-title":"Utadeo"},{"key":"B29","article-title":"Introducci\u00f3n a la lengua de se\u00f1as colombiana","year":"2024","journal-title":"Uniandes"},{"key":"B30","article-title":"Lengua de se\u00f1as colombiana \u2013 nivel a1","year":"2024","journal-title":"Ecci"},{"key":"B31","unstructured":"Deafness and hearing loss\n          \n          \n          2024"},{"key":"B32","unstructured":"Disability\n          \n          \n          2023"},{"key":"B33","doi-asserted-by":"publisher","first-page":"248","DOI":"10.1016\/J.VRIH.2021.05.004","article-title":"Teaching chinese sign language with a smartphone","volume":"3","author":"Zhang","year":"2021","journal-title":"Virtual Real. and Intelligent Hardw."}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1475069\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,14]],"date-time":"2024-11-14T06:22:10Z","timestamp":1731565330000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1475069\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,14]]},"references-count":33,"alternative-id":["10.3389\/frobt.2024.1475069"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1475069","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,14]]},"article-number":"1475069"}}