{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T22:38:09Z","timestamp":1777502289792,"version":"3.51.4"},"reference-count":43,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T00:00:00Z","timestamp":1740960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100009392","name":"Prince Sattam bin Abdulaziz University","doi-asserted-by":"publisher","award":["PSAU\/2024\/01\/29707"],"award-info":[{"award-number":["PSAU\/2024\/01\/29707"]}],"id":[{"id":"10.13039\/100009392","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Bioengineering"],"abstract":"<jats:p>Educating deafblind children is a highly specialized field that requires computer-assisted learning tools to address the challenges of auditory and visual impairments. The objective is to reduce their difficulties in communication with their peers and to empower them to learn independently in a classroom environment. Braille and assistive tools have become profoundly beneficial for deafblind children, serving as an essential means of communication and knowledge acquisition, enabling them to live independently. This study aims to develop an assistive tool that bridges the limitations of conventional tactile methodologies by incorporating the latest artificial intelligence techniques, enabling children to learn with greater ease. The research leverages Morse code technology to facilitate communication with deafblind children. The speaker\u2019s lip movements are converted into text using the deep learning techniques of a 3D convolutional neural network and a bidirectional long short-term memory neural network. Experimental evaluations of this text conversion model show a word error rate of 2% and an accuracy rate of 98%. The text is then converted into Morse code and communicated to the deafblind child through a wearable device. The significance of this assistive tool lies in its discreet design, resembling a smartwatch. Adolescents can wear the proposed wearable device confidently without feeling self-conscious or embarrassed.<\/jats:p>","DOI":"10.3390\/bioengineering12030253","type":"journal-article","created":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T05:52:16Z","timestamp":1740981136000},"page":"253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["A Novel Tactile Learning Assistive Tool for the Visually and Hearing Impaired with 3D-CNN and Bidirectional LSTM Leveraging Morse Code Technology"],"prefix":"10.3390","volume":"12","author":[{"given":"Shabana Ziyad","family":"Puthu Vedu","sequence":"first","affiliation":[{"name":"Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Al Kharj 11942, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"May","family":"Altulyan","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, College of Computer Engineering and Sciences, Prince Sattam Bin Abdulaziz University, Al Kharj 11942, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7676-9014","authenticated-orcid":false,"given":"Pradeep Kumar","family":"Singh","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Central University of Jammu, Jammu and Kashmir 181 143, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,3]]},"reference":[{"key":"ref_1","first-page":"44","article-title":"The Global Initiative Vison 2020: The Right to Sight Childhood Blindness","volume":"12","author":"Yorston","year":"1999","journal-title":"Community Eye Health"},{"key":"ref_2","unstructured":"(2024, October 25). National Deafblind Child Count Report. Available online: https:\/\/ncdb-childcount-2022.netlify.app\/01-intro#overview-of-deafblindness."},{"key":"ref_3","unstructured":"(2024, October 25). Deafblindness and Communication. Available online: https:\/\/www.communicationhub.com.au\/CommunicationHub\/Communication_Hub\/Resources\/Fact_Sheets\/Deafblindness_and_communication.aspx."},{"key":"ref_4","unstructured":"(2024, October 25). Deaf-Blindness\u2014Introduction to Language and Communication. Available online: https:\/\/my.vanderbilt.edu\/speds2430\/2020-topics\/deaf-blindness\/."},{"key":"ref_5","unstructured":"(2024, October 25). What Is Braille? The American Foundation for the Blind. Available online: https:\/\/www.afb.org\/blindness-and-low-vision\/braille\/what-braille."},{"key":"ref_6","first-page":"52","article-title":"The Use of Assistive Technology for Students with Disabilities in Technical Colleges in Ondo State","volume":"14","author":"Vincent","year":"2024","journal-title":"J. Educ. Res. Pract."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"128284","DOI":"10.1016\/j.neucom.2024.128284","article-title":"Assistive systems for visually impaired people: A survey on current requirements and advancements","volume":"606","author":"Preeti","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"46153","DOI":"10.1007\/s11042-023-17409-z","article-title":"Assistive technology-based solutions in learning mathematics for visually impaired people: Exploring issues, challenges and opportunities","volume":"82","author":"Shoaib","year":"2023","journal-title":"Multimed. Tools App."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Flores, R.A., and del-Rio-Guerra, M.S. (2023). Recent Developments in Haptic Devices Designed for Hearing-Impaired People: A Literature Review. Sensors, 23.","DOI":"10.3390\/s23062968"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3508364","article-title":"Tactile Materials in Practice: Understanding the Experiences of Teachers of the Visually Impaired","volume":"15","author":"Phutane","year":"2022","journal-title":"ACM Trans. Access. Comput."},{"key":"ref_11","unstructured":"Naja, B.A. (2024, March 19). Al-Noor Institute Brings Light to the Lives of Blind. Saudigazette. Available online: http:\/\/saudigazette.com.sa\/article\/525529."},{"key":"ref_12","first-page":"880","article-title":"Special Education in Saudi Arabia","volume":"5","author":"Zaed","year":"2016","journal-title":"Int. J. Technol. Incl. Educ."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zdravkova, K., Krasniqi, V., Dalipi, F., and Ferati, M. (2022). Cutting-edge communication and learning assistive technologies for disabled children: An artificial intelligence perspective. Front. Artif. Intell., 5.","DOI":"10.3389\/frai.2022.970430"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e188","DOI":"10.7861\/fhj.2021-0095","article-title":"Artificial intelligence in healthcare: Transforming the practice of medicine","volume":"8","author":"Bajwa","year":"2021","journal-title":"Future Healthc. J."},{"key":"ref_15","unstructured":"Assael, Y.M., Shillingford, B., Whiteson, S., and de Freitas, N. (2016). LipNet: End-to-End Sentence-level Lipreading. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lu, Y., and Li, H. (2019). Automatic Lip-Reading System Based on Deep Convolutional Neural Network and Attention-Based Long Short-Term Memory. Appl. Sci., 9.","DOI":"10.3390\/app9081599"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2054003","DOI":"10.1142\/S0218001420540038","article-title":"Automatic Lip Reading Using Convolution Neural Network and Bidirectional Long Short-term Memory","volume":"34","author":"Lu","year":"2020","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4533","DOI":"10.53730\/ijhs.v6nS2.6078","article-title":"Word Recognition Through Mapping of Lip Movements from Speech Utterance Using Audiovisual Fusion and MLP","volume":"6","author":"Srikanth","year":"2022","journal-title":"Int. J. Health Sci."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhang, G., and Lu, Y. (2023). Research on a Lip Reading Algorithm Based on Efficient-GhostNet. Electronics, 12.","DOI":"10.3390\/electronics12051151"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"89","DOI":"10.5194\/isprs-archives-XLVIII-2-W3-2023-89-2023","article-title":"Improved Automatic Lip-Reading Based on The Evaluation of Intensity Level Of Speaker\u2019s Emotion","volume":"XLVIII-2-W3-2023","author":"Ivanko","year":"2023","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1911","DOI":"10.1007\/s11423-022-10127-7","article-title":"Assistive technology for the inclusion of students with disabilities: A systematic review","volume":"70","year":"2022","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_22","unstructured":"(2024, November 08). TEDI: Research, Department of Health & Rehabilitation Studies, University of Cape Town. Available online: http:\/\/www.dhrs.uct.ac.za\/dhrs\/divisions\/disability\/tedi\/research."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1080\/17483107.2019.1604829","article-title":"Assistive technology for students with visual disability in schools for the blind in Delhi","volume":"15","author":"Senjam","year":"2019","journal-title":"Disabil. Rehabil. Assist. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"501","DOI":"10.3390\/digital4020025","article-title":"Assistive Technology for Higher Education Students with Disabilities: A Qualitative Research","volume":"4","author":"Papadopoulos","year":"2024","journal-title":"Digital"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"111906","DOI":"10.1016\/j.asoc.2024.111906","article-title":"AI-based visual speech recognition towards realistic avatars and lip-reading applications in the metaverse","volume":"164","author":"Li","year":"2024","journal-title":"Appl. Soft Comput."},{"key":"ref_26","unstructured":"(2024, October 17). The GRID Audiovisual Sentence Corpus. Available online: https:\/\/spandh.dcs.shef.ac.uk\/gridcorpus."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lei, Z., Pusateri, E., Han, S., Liu, L., Xu, M., Ng, T., Travadi, R., Zhang, Y., Hannemann, M., and Siu, M.H. (2023). Personalization of CTC-based End-to-End Speech Recognition Using Pronunciation-Driven Subword Tokenization. arXiv.","DOI":"10.1109\/ICASSP48485.2024.10447571"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Molchanov, P., Yang, X., Gupta, S., Kim, K., Tyree, S., and Kautz, J. (2016, January 27\u201330). Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.456"},{"key":"ref_29","unstructured":"(2024, October 27). Weights & Biases W&B. Available online: https:\/\/wandb.ai\/authors\/text-recognition-crnn-ctc\/reports\/Text-Recognition-With-CRNN-CTC-Network--VmlldzoxNTI5NDI."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"012076","DOI":"10.1088\/1742-6596\/1651\/1\/012076","article-title":"The Research of Lip Reading Based on STCNN and Conv LSTM","volume":"1651","author":"Zhu","year":"2020","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_31","unstructured":"(2024, November 25). AdaMER-CTC: Connectionist Temporal Classification with Adaptive Maximum Entropy Regularization for Automatic Speech Recognition. Available online: https:\/\/arxiv.org\/html\/2403.11578v1."},{"key":"ref_32","unstructured":"(2024, October 26). 59M.1677: International Morse Code. Available online: https:\/\/www.itu.int\/rec\/R-REC-M.1677-1-200910-I\/."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1007\/978-3-319-08599-9_59","article-title":"Teaching Morse Language to a Deafblind Person for Reading and Writing SMS on an Ordinary Vibrating Smartphone","volume":"8548","author":"Arato","year":"2014","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_34","unstructured":"(2024, October 26). Purpose: Use a Binary Tree Data Structure to Hold|Chegg.com. Available online: https:\/\/www.chegg.com\/homework-help\/questions-and-answers\/purpose-use-binary-tree-data-structure-hold-morse-code-values-letters-english-alphabet-lab-q38033610."},{"key":"ref_35","unstructured":"Shrestha, K., Voila, P., and Jones, M. (2024, November 08). Lip Reading Using Neural Network and Deep Learning. Available online: https:\/\/api.semanticscholar.org\/CorpusID:207788337."},{"key":"ref_36","unstructured":"Ahmed, A., and Steve, R. (2018, January 15\u201320). Word Error Rate Estimation for Speech Recognition: E-WER. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Australia."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Popovic, M., and Ney, H. (2007, January 23). Word error rates. Proceedings of the Second Workshop on Statistical Machine Translation, Prague, Czech Republic.","DOI":"10.3115\/1626355.1626362"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"4548","DOI":"10.1109\/TMC.2023.3294416","article-title":"Acoustic-Based Lip Reading for Mobile Devices: Dataset, Benchmark and a Self-Distillation-Based Approach","volume":"23","author":"Yin","year":"2024","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_39","first-page":"6479124","article-title":"Indonesian Lip-Reading Detection and Recognition Based on Lip Shape Using Face Mesh and Long-Term Recurrent Convolutional Network","volume":"2024","author":"Aripin","year":"2024","journal-title":"Appl. Comput. Intell. Soft Comput."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"4448861","DOI":"10.1155\/2023\/4448861","article-title":"Lip Print Recognition Algorithm Based on Convolutional Network","volume":"2023","author":"Zhou","year":"2023","journal-title":"J. Appl. Math."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kim, M., Kim, H.I., and Ro, Y.M. (2024). Prompt Tuning of Deep Neural Networks for Speaker-adaptive Visual Speech Recognition. arXiv.","DOI":"10.1109\/TPAMI.2024.3484658"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Yeo, J.H., Kim, C.W., Kim, H., Rha, H., Han, S., Cheng, W.H., and Ro, Y.M. (2024). Personalized Lip Reading: Adapting to Your Unique Lip Movements with Vision and Language. arXiv.","DOI":"10.1609\/aaai.v39i9.33026"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"410","DOI":"10.3390\/biomedinformatics4010023","article-title":"Lip-Reading Advancements: A 3D Convolutional Neural Network\/Long Short-Term Memory Fusion for Precise Word Recognition","volume":"4","author":"Exarchos","year":"2024","journal-title":"BioMedInformatics"}],"container-title":["Bioengineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5354\/12\/3\/253\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:46:02Z","timestamp":1760028362000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5354\/12\/3\/253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,3]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["bioengineering12030253"],"URL":"https:\/\/doi.org\/10.3390\/bioengineering12030253","relation":{},"ISSN":["2306-5354"],"issn-type":[{"value":"2306-5354","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,3]]}}}