{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T14:50:43Z","timestamp":1781275843830,"version":"3.54.1"},"reference-count":43,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T00:00:00Z","timestamp":1717632000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Sign language is an essential means of communication for individuals with hearing disabilities. However, there is a significant shortage of sign language interpreters in some languages, especially in Saudi Arabia. This shortage results in a large proportion of the hearing-impaired population being deprived of services, especially in public places. This paper aims to address this gap in accessibility by leveraging technology to develop systems capable of recognizing Arabic Sign Language (ArSL) using deep learning techniques. In this paper, we propose a hybrid model to capture the spatio-temporal aspects of sign language (i.e., letters and words). The hybrid model consists of a Convolutional Neural Network (CNN) classifier to extract spatial features from sign language data and a Long Short-Term Memory (LSTM) classifier to extract spatial and temporal characteristics to handle sequential data (i.e., hand movements). To demonstrate the feasibility of our proposed hybrid model, we created a dataset of 20 different words, resulting in 4000 images for ArSL: 10 static gesture words and 500 videos for 10 dynamic gesture words. Our proposed hybrid model demonstrates promising performance, with the CNN and LSTM classifiers achieving accuracy rates of 94.40% and 82.70%, respectively. These results indicate that our approach can significantly enhance communication accessibility for the hearing-impaired community in Saudi Arabia. Thus, this paper represents a major step toward promoting inclusivity and improving the quality of life for the hearing impaired.<\/jats:p>","DOI":"10.3390\/s24113683","type":"journal-article","created":{"date-parts":[[2024,6,6]],"date-time":"2024-06-06T06:40:26Z","timestamp":1717656026000},"page":"3683","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Real-Time Arabic Sign Language Recognition Using a Hybrid Deep Learning Model"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5076-8163","authenticated-orcid":false,"given":"Talal H.","family":"Noor","sequence":"first","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3344-2847","authenticated-orcid":false,"given":"Ayman","family":"Noor","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed F.","family":"Alharbi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed","family":"Faisal","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rakan","family":"Alrashidi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmed S.","family":"Alsaedi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ghada","family":"Alharbi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2630-5972","authenticated-orcid":false,"given":"Tawfeeq","family":"Alsanoosy","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7974-7638","authenticated-orcid":false,"given":"Abdullah","family":"Alsaeedi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Engineering, Taibah University, Madinah 42353, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"113794","DOI":"10.1016\/j.eswa.2020.113794","article-title":"Sign language recognition: A deep survey","volume":"164","author":"Rastgoo","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_2","unstructured":"Costello, E. (2008). Random House Webster\u2019s Compact American Sign Language Dictionary, Penguin Random House."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.neucom.2016.08.132","article-title":"A multimodal framework for sensor based sign language recognition","volume":"259","author":"Kumar","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1007\/s11220-019-0225-3","article-title":"Multiple proposals for continuous arabic sign language recognition","volume":"20","author":"Hassan","year":"2019","journal-title":"Sens. Imaging"},{"key":"ref_5","unstructured":"Ministry of Health, S.A. (2024, February 15). The Deaf and Sign Language, Available online: https:\/\/www.moh.gov.sa\/en\/Ministry\/Information-and-services\/Pages\/Sign-language.aspx."},{"key":"ref_6","unstructured":"Ministry of Health, S.A. (2024, February 10). We Are with You, Available online: https:\/\/www.moh.gov.sa\/en\/Ministry\/Projects\/with-you\/Pages\/default.aspx."},{"key":"ref_7","unstructured":"Center for Strategic and International Studies (CSIS) (2024, January 05). Reading the Signs: Diverse Arabic Sign Languages. Available online: https:\/\/www.csis.org\/analysis\/reading-signs-diverse-arabic-sign-languages."},{"key":"ref_8","first-page":"185","article-title":"Image based Arabic sign language recognition system","volume":"9","author":"Alzohairi","year":"2018","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4567989","DOI":"10.1155\/2022\/4567989","article-title":"Sign language recognition for Arabic alphabets using transfer learning technique","volume":"2022","author":"Zakariah","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1007\/s11831-019-09384-2","article-title":"Sign language recognition systems: A decade systematic literature review","volume":"28","author":"Wadhawan","year":"2021","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Noor, T.H., Noor, A., and Elmezain, M. (2022). Poisonous Plants Species Prediction Using a Convolutional Neural Network and Support Vector Machine Hybrid Model. Electronics, 11.","DOI":"10.3390\/electronics11223690"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.isprsjprs.2020.12.010","article-title":"Review on Convolutional Neural Networks (CNN) in vegetation remote sensing","volume":"173","author":"Kattenborn","year":"2021","journal-title":"ISPRS J. Photogramm. Remote. Sens."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, B., Chen, Y., Yan, Z., and Liu, W. (2024). Integrating Remote Sensing Data and CNN-LSTM-Attention Techniques for Improved Forest Stock Volume Estimation: A Comprehensive Analysis of Baishanzu Forest Park, China. Remote Sens., 16.","DOI":"10.3390\/rs16020324"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"108147","DOI":"10.1016\/j.engappai.2024.108147","article-title":"Traffic management approaches using machine learning and deep learning techniques: A survey","volume":"133","author":"Almukhalfi","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_15","first-page":"161","article-title":"Deep learning model for predicting consumers\u2019 interests of IoT recommendation system","volume":"13","author":"Noor","year":"2022","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Abib, G., Castel, F., Satouri, N., Afifi, H., and Said, A.M. (June, January 28). Survey and Enhancements on Deploying LSTM Recurrent Neural Networks on Embedded Systems. Proceedings of the ICC 2023-IEEE International Conference on Communications, Rome, Italy.","DOI":"10.1109\/ICC45041.2023.10278766"},{"key":"ref_17","unstructured":"Koller, O. (2020). Quantitative survey of the state of the art in sign language recognition. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1007\/s13042-017-0705-5","article-title":"A review of hand gesture and sign language recognition techniques","volume":"10","author":"Cheok","year":"2019","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1109\/THMS.2014.2318280","article-title":"Image-based and sensor-based approaches to Arabic sign language recognition","volume":"44","author":"Mohandes","year":"2014","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Aiouez, S., Hamitouche, A., Belmadoui, M.S., Belattar, K., and Souami, F. (2022, January 22\u201324). Real-time Arabic Sign Language Recognition based on YOLOv5. Proceedings of the IMPROVE, Online Streaming.","DOI":"10.5220\/0010979300003209"},{"key":"ref_21","first-page":"692","article-title":"Arabic sign language recognition using Faster R-CNN","volume":"12","author":"Alawwad","year":"2021","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_22","unstructured":"Elhagry, A., and Elrayes, R.G. (2021). Egyptian sign language recognition using cnn and lstm. arXiv."},{"key":"ref_23","first-page":"424","article-title":"A Deep Learning based Approach for Recognition of Arabic Sign Language Letters","volume":"14","author":"Hdioud","year":"2023","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Rivera-Acosta, M., Ruiz-Varela, J.M., Ortega-Cisneros, S., Rivera, J., Parra-Michel, R., and Mejia-Alvarez, P. (2021). Spelling correction real-time American sign language alphabet translation system based on YOLO network and LSTM. Electronics, 10.","DOI":"10.3390\/electronics10091035"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Dutta, K.K., and Bellary, S.A.S. (2017, January 8\u20139). Machine learning techniques for Indian sign language recognition. Proceedings of the 2017 International Conference on Current Trends in Computer, Electrical, Electronics, and Communication (CTCEEC), Mysore, India.","DOI":"10.1109\/CTCEEC.2017.8454988"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Sako, S., Hatano, M., and Kitamura, T. (2016, January 17\u201322). Real-time Japanese sign language recognition based on three phonological elements of sign. Proceedings of the HCI International 2016\u2013Posters\u2019 Extended Abstracts: 18th International Conference, HCI International 2016, Toronto, ON, Canada. Proceedings, Part II 18.","DOI":"10.1007\/978-3-319-40542-1_21"},{"key":"ref_27","first-page":"1198","article-title":"Sign Language Recognition Using Convolutional Neural Networks","volume":"14","author":"Uyyala","year":"2022","journal-title":"J. Interdiscip. Cycle Res."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"132","DOI":"10.56578\/jisc020302","article-title":"Detection and interpretation of Indian Sign Language using LSTM networks","volume":"2","author":"Vyavahare","year":"2023","journal-title":"J. Intell Syst. Control"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Shurid, S.A., Amin, K.H., Mirbahar, M.S., Karmaker, D., Mahtab, M.T., Khan, F.T., Alam, M.G.R., and Alam, M.A. (2020, January 16\u201318). Bangla sign language recognition and sentence building using deep learning. Proceedings of the 2020 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE), Gold Coast, Australia.","DOI":"10.1109\/CSDE50874.2020.9411523"},{"key":"ref_30","unstructured":"Daniels, S., Suciati, N., and Fathichah, C. (2020, January 13\u201314). Indonesian sign language recognition using YOLO method. Proceedings of the IOP Conf. on Materials Science and Engineering, Yogyakarta, Indonesia."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ko, S.K., Son, J.G., and Jung, H. (2018, January 9\u201312). Sign language recognition with recurrent neural network using human keypoint detection. Proceedings of the 2018 Conference on Research in Adaptive and Convergent Systems, Honolulu, HI, USA.","DOI":"10.1145\/3264746.3264805"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"127194","DOI":"10.1016\/j.neucom.2023.127194","article-title":"TMS-Net: A multi-feature multi-stream multi-level information sharing network for skeleton-based sign language recognition","volume":"572","author":"Deng","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zuo, R., Wei, F., and Mak, B. (2023, January 18\u201322). Natural language-assisted sign language recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.01430"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ryumin, D., Ivanko, D., and Ryumina, E. (2023). Audio-visual speech and gesture recognition by sensors of mobile devices. Sensors, 23.","DOI":"10.3390\/s23042284"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Qi, W., Fan, H., Xu, Y., Su, H., and Aliverti, A. (2022, January 22\u201324). A 3d-CLDNN based multiple data fusion framework for finger gesture recognition in human-robot interaction. Proceedings of the 2022 4th International Conference on Control and Robotics (ICCR), Guangzhou, China.","DOI":"10.1109\/ICCR55715.2022.10053856"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1384","DOI":"10.1016\/j.procs.2023.01.117","article-title":"Real-time assamese sign language recognition using mediapipe and deep learning","volume":"218","author":"Bora","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Eunice, J., Sei, Y., and Hemanth, D.J. (2023). Sign2Pose: A Pose-Based Approach for Gloss Prediction Using a Transformer Model. Sensors, 23.","DOI":"10.3390\/s23052853"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"181340","DOI":"10.1109\/ACCESS.2020.3028072","article-title":"Autsl: A large scale multi-modal turkish sign language dataset and baseline methods","volume":"8","author":"Sincan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1162\/neco_a_01199","article-title":"A review of recurrent neural networks: LSTM cells and network architectures","volume":"31","author":"Yu","year":"2019","journal-title":"Neural Comput."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"132306","DOI":"10.1016\/j.physd.2019.132306","article-title":"Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network","volume":"404","author":"Sherstinsky","year":"2020","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"419","DOI":"10.3844\/jcssp.2024.419.430","article-title":"Measuring Customers\u2019 Satisfaction Using Sentiment Analysis: Model and Tool","volume":"20","author":"Alqurafi","year":"2024","journal-title":"J. Comput. Sci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"5929","DOI":"10.1007\/s10462-020-09838-1","article-title":"A review on the long short-term memory model","volume":"53","author":"Mosquera","year":"2020","journal-title":"Artif. Intell. Rev."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"100071","DOI":"10.1016\/j.dajour.2022.100071","article-title":"A comparative analysis of K-nearest neighbor, genetic, support vector machine, decision tree, and long short term memory algorithms in machine learning","volume":"3","author":"Bansal","year":"2022","journal-title":"Decis. Anal. J."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3683\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:54:37Z","timestamp":1760108077000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/11\/3683"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,6]]},"references-count":43,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,6]]}},"alternative-id":["s24113683"],"URL":"https:\/\/doi.org\/10.3390\/s24113683","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,6]]}}}