{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:45:29Z","timestamp":1783010729466,"version":"3.54.6"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,5,27]],"date-time":"2022-05-27T00:00:00Z","timestamp":1653609600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:p>Natural Language Processing (NLP) is a group of theoretically inspired computer structures for analyzing and modeling clearly going on texts at one or extra degrees of linguistic evaluation to acquire human-like language processing for quite a few activities and applications. Hearing and visually impaired people are unable to see entirely or have very low vision, as well as being unable to hear completely or having a hard time hearing. It is difficult to get information since both hearing and vision, which are crucial organs for receiving information, are harmed. Hearing and visually impaired people are considered to have a substantial information deficit, as opposed to people who just have one handicap, such as blindness or deafness. Visually and hearing-impaired people who are unable to communicate with the outside world may experience emotional loneliness, which can lead to stress and, in extreme cases, serious mental illness. As a result, overcoming information handicap is a critical issue for visually and hearing-impaired people who want to live active, independent lives in society. The major objective of this study is to recognize Arabic speech in real time and convert it to Arabic text using Convolutional Neural Network-based algorithms before saving it to an SD card. The Arabic text is then translated into Arabic Braille characters, which are then used to control the Braille pattern via a Braille display with a solenoid drive. The Braille lettering triggered on the finger was deciphered by visually and hearing challenged participants who were proficient in Braille reading. The CNN, in combination with the ReLU model learning parameters, is fine-tuned for optimization, resulting in a model training accuracy of 90%. The tuned parameters model's testing results show that adding the ReLU activation function to the CNN model improves recognition accuracy by 84 % when speaking Arabic digits.<\/jats:p>","DOI":"10.3389\/fpubh.2022.898355","type":"journal-article","created":{"date-parts":[[2022,6,1]],"date-time":"2022-06-01T08:43:27Z","timestamp":1654073007000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Convolutional Neural Network Based Real Time Arabic Speech Recognition to Arabic Braille for Hearing and Visually Impaired"],"prefix":"10.3389","volume":"10","author":[{"given":"Surbhi","family":"Bhatia","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ajantha","family":"Devi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Razan Ibrahim","family":"Alsuwailem","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arwa","family":"Mashat","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,5,27]]},"reference":[{"key":"B1","volume-title":"Person With Deafblind and Normalization","author":"Fukushima","year":"1997"},{"key":"B2","first-page":"37","article-title":"Bidirectional telecommunication using wearable i\/o terminal for finger braille","volume-title":"Proceedings of the Human Interface Symposium","author":"Uehara","year":"2000"},{"key":"B3","first-page":"731","article-title":"A study on a finger-braille input system based on acceleration of finger movements","volume-title":"IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, Vol. 85","author":"Hoshino","year":"2002"},{"key":"B4","doi-asserted-by":"publisher","first-page":"25","DOI":"10.5772\/51472","article-title":"Bracelet type braille interface","volume":"51","author":"Ochi","year":"2003","journal-title":"Correspond Hum Interf"},{"key":"B5","doi-asserted-by":"crossref","first-page":"3202","DOI":"10.1109\/ICMA.2007.4304074","article-title":"Finger Braille teaching system for people who communicate with deafblind people","volume-title":"2007 International Conference on Mechatronics and Automation.","author":"Matsuda","year":"2007"},{"key":"B6","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1109\/ICMA.2007.4304074","article-title":"Development of finger Braille teaching system-teaching of dotting finger and position using speech recognition","volume":"19","author":"Matsuda","year":"2007","journal-title":"J Life Support Eng."},{"key":"B7","first-page":"57","article-title":"Computer based Tamil Braille system\u2013a review","volume-title":"11th International Tamil Internet Conference, Annamalai University, Chidambaram, India, Jointly Conducted by International Forum for Information Technology (INFITT) and Annamalai University","author":"Baboo","year":"2012"},{"key":"B8","unstructured":"A review on speech to text conversion methods306772\n            KhilariP\n            BhopeVP\n          Int J Adv Res Comput Eng Technol.42015"},{"key":"B9","unstructured":"An analysis on types of speech recognition and algorithms3505\n            DeviVA\n            SuganyaMV\n          Int J Comp Sci Trends Technol.42016"},{"key":"B10","doi-asserted-by":"publisher","first-page":"796","DOI":"10.3923\/itj.2009.796.800","article-title":"Speech recognition algorithm of parallel subband HMM based on wavelet analysis and neural network","volume":"8","author":"Ping","year":"2009","journal-title":"Inform Technol J."},{"key":"B11","doi-asserted-by":"publisher","first-page":"1272","DOI":"10.1109\/JPROC.2003.817117","article-title":"Interacting with computers by voice: automatic speech recognition and synthesis","volume":"91","author":"O'Shaughnessy","year":"2003","journal-title":"Proc IEEE."},{"key":"B12","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1007\/s10772-010-9088-7","article-title":"Front end analysis of speech recognition: a review","volume":"14","author":"Anusuya","year":"2011","journal-title":"Int J Speech Technol."},{"key":"B13","first-page":"1154","article-title":"Feature extraction based on wavelet domain hidden Markov tree model for robust speech recognition","volume-title":"Australasian Joint Conference on Artificial Intelligence.","author":"Jung","year":"2004"},{"key":"B14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/SPED.2011.5940729","article-title":"Comparison of feature extraction methods for speech recognition in noise-free and in traffic noise environment","volume-title":"2011 6th Conference on Speech Technology and Human-Computer Dialogue (SpeD).","author":"S\u00e1rosi","year":"2011"},{"key":"B15","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1109\/5.18626","article-title":"A tutorial on hidden Markov models and selected applications in speech recognition","volume":"77","author":"Rabiner","year":"1989","journal-title":"Proc IEEE."},{"key":"B16","article-title":"Sphinx-4: A flexible open source framework for speech recognition","author":"Walker","year":""},{"key":"B17","first-page":"1","article-title":"Multi dialect Arabic speech parallel corpora","volume-title":"2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA).","author":"Almeman","year":"2013"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1300","DOI":"10.1109\/21.44052","article-title":"Machine recognition and correction of printed Arabic text","volume":"19","author":"Amin","year":"1989","journal-title":"IEEE Transac Syst Man Cybernet."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1147","DOI":"10.1016\/j.csl.2012.10.002","article-title":"Acoustic model adaptation using in-domain background models for dysarthric speech recognition","volume":"27","author":"Sharma","year":"2013","journal-title":"Comp Speech Lang."},{"key":"B20","doi-asserted-by":"publisher","first-page":"1176","DOI":"10.1016\/j.sigpro.2008.12.022","article-title":"A multiple feature\/resolution scheme to Arabic (Indian) numerals recognition using hidden Markov models","volume":"89","author":"Awaidah","year":"2009","journal-title":"Signal Process."},{"key":"B21","unstructured":"MFCC and its applications in speaker recognition1922\n            TiwariV\n          Int J Emerg Technol.12010"},{"key":"B22","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1109\/97.928676","article-title":"Mel filter-like admissible wavelet packet structure for speech recognition","volume":"8","author":"Farooq","year":"2001","journal-title":"IEEE Signal Proc Lett."},{"key":"B23","first-page":"2150","article-title":"Tone classification in Mandarin Chinese using convolutional neural networks","volume-title":"Interspeech","author":"Chen","year":"2016"},{"key":"B24","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1109\/I2CACIS.2019.8825004","article-title":"Spoken arabic digits recognition using deep learning","volume-title":"2019 IEEE International Conference on Automatic Control and Intelligent Systems (I2CACIS).","author":"Wazir","year":"2019"},{"key":"B25","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1109\/SNPD.2017.8022720","article-title":"Recognizing Arabic letter utterance using convolutional neural network","volume-title":"2017 18th IEEE\/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel\/Distributed Computing (SNPD).","author":"Rajagede","year":"2017"},{"key":"B26","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MSP.2012.2205597","article-title":"Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups","volume":"29","author":"Hinton","year":"2012","journal-title":"IEEE Signal Proc Mag."},{"key":"B27","first-page":"417","article-title":"Neural-net classifiers useful for speech recognition","volume-title":"Proceedings International Conference on Neural Networks, Vol. 4","author":"Lippmann","year":"1987"},{"key":"B28","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1109\/29.21701","article-title":"Phoneme recognition using time-delay neural networks","volume":"37","author":"Waibel","year":"1989","journal-title":"IEEE Transac Acoustics Speech Signal Process."},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-1906-8","volume-title":"Novel Techniques for Dialectal Arabic Speech Recognition","author":"Elmahdy","year":"2012"},{"key":"B30","first-page":"1","article-title":"Conversion of speech to Braille: interaction device for visual and hearing impaired","volume-title":"2017 Fourth International Conference on Signal Processing, Communication and Networking (ICSCN).","author":"Devi","year":"2017"},{"key":"B31","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1007\/978-3-642-13681-8_15","article-title":"Speech recognition system of Arabic alphabet based on a telephony Arabic corpus","volume-title":"International Conference on Image and Signal Processing.","author":"Ajami Alotaibi","year":"2010"},{"key":"B32","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.ins.2004.07.008","article-title":"Investigating spoken Arabic digits in speech recognition setting","volume":"173","author":"Alotaibi","year":"2005","journal-title":"Inform Sci."},{"key":"B33","doi-asserted-by":"publisher","first-page":"1533","DOI":"10.1109\/TASLP.2014.2339736","article-title":"Convolutional neural networks for speech recognition","volume":"22","author":"Abdel-Hamid","year":"2014","journal-title":"IEEE ACM Trans Audio Speech Lang Process."},{"key":"B34","doi-asserted-by":"publisher","first-page":"125","DOI":"10.21608\/ijicis.2021.51555.1046","article-title":"Fast detection of distributed denial of service attacks in VoIP networks using convolutional neural networks","volume":"20","author":"Nazih","year":"2020","journal-title":"Int J Intellig Comput Inform Sci."},{"key":"B35","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1007\/s13244-018-0639-9","article-title":"Convolutional neural networks: an overview and application in radiology","volume":"9","author":"Yamashita","year":"2018","journal-title":"Insights Imaging."},{"key":"B36","doi-asserted-by":"publisher","first-page":"2020","DOI":"10.3390\/app10062020","article-title":"A review of deep learning based methods for acoustic scene classification","volume":"10","author":"Abe\u00dfer","year":"2020","journal-title":"Applied Sciences."},{"key":"B37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ICASSP.2013.6638952","article-title":"Unsupervised feature learning for audio classification using convolutional deep belief networks","volume":"22","author":"Lee","year":"2009","journal-title":"Adv Neural Inform Process Syst."},{"key":"B38","doi-asserted-by":"crossref","first-page":"4277","DOI":"10.1109\/ICASSP.2012.6288864","article-title":"Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition","volume-title":"2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).","author":"Abdel-Hamid","year":"2012"},{"key":"B39","doi-asserted-by":"crossref","first-page":"6669","DOI":"10.1109\/ICASSP.2013.6638952","article-title":"A deep convolutional neural network using heterogeneous pooling for trading acoustic invariance with phonetic confusion","volume-title":"2013 IEEE International Conference on Acoustics, Speech and Signal Processing.","author":"Deng","year":"2013"},{"key":"B40","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.21437\/Interspeech.2013-744","article-title":"Exploring convolutional neural network structures and optimization techniques for speech recognition","volume-title":"Interspeech, Vol. 2013","author":"Abdel-Hamid","year":"2013"},{"key":"B41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/PICC.2015.7455806","article-title":"Speaker identification and verification of noisy speech using multitaper MFCC and Gaussian Mixture models","volume-title":"2015 International Conference on Power, Instrumentation, Control and Computing (PICC).","author":"Veena","year":"2015"},{"key":"B42","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1109\/ICCSIT.2010.5563892","article-title":"Improved tree model for Arabic speech recognition","volume-title":"2010 3rd International Conference on Computer Science and Information Technology.","author":"Hammami","year":"2010"},{"key":"B43","doi-asserted-by":"publisher","first-page":"3799","DOI":"10.1007\/s13369-014-1048-0","article-title":"Automatic speaker recognition using multi-directional local features (MDLF)","volume":"39","author":"Mahmood","year":"2014","journal-title":"Arabian J Sci Eng."},{"key":"B44","article-title":"Arabic isolated word speaker dependent recognition system","volume-title":"Source Framework for Speech Recognition","author":"Elkourd","year":"2004"},{"key":"B45","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.specom.2016.11.004","article-title":"Speaker recognition based on Arabic phonemes","volume":"86","author":"Alsulaiman","year":"2017","journal-title":"Speech Commun."},{"key":"B46","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1007\/s10772-017-9456-7","article-title":"Constructing accurate and robust HMM\/GMM models for an Arabic speech recognition system","volume":"20","author":"Khelifa","year":"2017","journal-title":"Int J Speech Technol."},{"key":"B47","doi-asserted-by":"publisher","first-page":"5503","DOI":"10.32604\/cmc.2022.021125","article-title":"Fusion of infrared and visible images using fuzzy based siamese convolutional network","volume":"70","author":"Bhalla","year":"2022","journal-title":"Comput. Mater"},{"key":"B48","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1007\/978-1-4471-0833-7_26","article-title":"Robust speech recognition using fuzzy matrix\/vector quantisation, neural networks and hidden Markov models","volume-title":"International Conference on Advances in Pattern Recognition.","author":"Cong","year":"1999"},{"key":"B49","first-page":"557","article-title":"VQ Codebook design using genetic algorithms for speech line spectral frequencies","volume-title":"International Symposium on Intelligence Computation and Applications.","author":"Merazka","year":"2012"},{"key":"B50","first-page":"305","article-title":"On the use of MFCC feature vector clustering for efficient text dependent speaker recognition","volume-title":"Proceedings of the International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA).","author":"Samal","year":"2013"},{"key":"B51","first-page":"519","article-title":"SVM and HMM modeling techniques for speech recognition using LPCC and MFCC features","volume-title":"Proceedings of the 3rd International Conference on Frontiers of Intelligent Computing: Theory and Applications (FICTA).","author":"Ananthi","year":"2014"},{"key":"B52","article-title":"Tamil braille system: A conversion methodology of tamil into contracted braille script (grade2)","volume-title":"UACEE International Journal of Advances in Computer Science and its Applications - IJCSIA, Vol. 3","author":"Baboo","year":"2013"},{"key":"B53","first-page":"4809","volume-title":"Portable Device Converting Braille and Text Document to Speech Output Using Raspberry Pi","author":"Devi","year":"2016"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2022.898355\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,6,1]],"date-time":"2022-06-01T08:44:03Z","timestamp":1654073043000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2022.898355\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,27]]},"references-count":53,"alternative-id":["10.3389\/fpubh.2022.898355"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2022.898355","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,27]]},"article-number":"898355"}}