{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T06:23:05Z","timestamp":1774074185100,"version":"3.50.1"},"reference-count":53,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T00:00:00Z","timestamp":1691971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face\u2013hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron \u201cSelfMLP\u201d is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face\u2013hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy.<\/jats:p>","DOI":"10.3390\/s23167156","type":"journal-article","created":{"date-parts":[[2023,8,14]],"date-time":"2023-08-14T11:07:10Z","timestamp":1692011230000},"page":"7156","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":44,"title":["Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4687-5801","authenticated-orcid":false,"given":"Kanchon Kanti","family":"Podder","sequence":"first","affiliation":[{"name":"Department of Biomedical Physics & Technology, University of Dhaka, Dhaka 1000, Bangladesh"}]},{"given":"Maymouna","family":"Ezeddin","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Hamad Bin Khalifa University, Doha 34110, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0744-8206","authenticated-orcid":false,"given":"Muhammad E. H.","family":"Chowdhury","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Qatar University, Doha 2713, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5839-2826","authenticated-orcid":false,"given":"Md. Shaheenur Islam","family":"Sumon","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, Military Institute of Science and Technology (MIST), Dhaka 1216, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5018-0626","authenticated-orcid":false,"given":"Anas M.","family":"Tahir","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Qatar University, Doha 2713, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8663-886X","authenticated-orcid":false,"given":"Mohamed Arselene","family":"Ayari","sequence":"additional","affiliation":[{"name":"Department of Civil and Architectural Engineering, Qatar University, Doha 2713, Qatar"}]},{"given":"Proma","family":"Dutta","sequence":"additional","affiliation":[{"name":"Department of Electrical& Electronic Engineering, Chittagong University of Engineering & Technology, Chittagong 4349, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7068-9112","authenticated-orcid":false,"given":"Amith","family":"Khandakar","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Qatar University, Doha 2713, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3877-0177","authenticated-orcid":false,"given":"Zaid Bin","family":"Mahbub","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Physics, North South University, Dhaka 1229, Bangladesh"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4535-9215","authenticated-orcid":false,"given":"Muhammad Abdul","family":"Kadir","sequence":"additional","affiliation":[{"name":"Department of Biomedical Physics & Technology, University of Dhaka, Dhaka 1000, Bangladesh"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e20200291","DOI":"10.1590\/0034-7167-2020-0291","article-title":"Information about COVID-19 for deaf people: An analysis of Youtube videos in Brazilian sign language","volume":"74","author":"Galindo","year":"2021","journal-title":"Rev. Bras. Enferm."},{"key":"ref_2","unstructured":"Makhashen, G.M.B., Luqman, H.A., and El-Alfy, E.S.M. (2019, January 24\u201326). Using Gabor filter bank with downsampling and SVM for visual sign language alphabet recognition. Proceedings of the 2nd Smart Cities Symposium (SCS 2019), Bahrain, Bahrain."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.procs.2017.10.087","article-title":"Transform-based Arabic sign language recognition","volume":"117","author":"Luqman","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"294","DOI":"10.3390\/agriengineering3020020","article-title":"Automatic and Reliable Leaf Disease Detection Using Deep Learning Techniques","volume":"3","author":"Chowdhury","year":"2021","journal-title":"AgriEngineering"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Podder, K.K., Chowdhury, M.E.H., Tahir, A.M., Mahbub, Z.B., Khandakar, A., Hossain, M.S., and Kadir, M.A. (2022). Bangla Sign Language (BdSL) Alphabets and Numerals Classification Using a Deep Learning Model. Sensors, 22.","DOI":"10.3390\/s22020574"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Podder, K.K., Tabassum, S., Khan, L.E., Salam, K.M.A., Maruf, R.I., and Ahmed, A. (2021, January 17\u201320). Design of a Sign Language Transformer to Enable the Participation of Persons with Disabilities in Remote Healthcare Systems for Ensuring Universal Healthcare Coverage. Proceedings of the 2021 IEEE Technology Engineering Management Conference\u2013Europe (TEMSCON-EUR), Dubrovnik, Croatia.","DOI":"10.1109\/TEMSCON-EUR52034.2021.9488605"},{"key":"ref_7","first-page":"20","article-title":"Bangla Sign Language Alphabet Recognition Using Transfer Learning Based Convolutional Neural Network","volume":"31\u201333","author":"Podder","year":"2020","journal-title":"Bangladesh J. Sci. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1007\/s10209-019-00695-6","article-title":"A machine translation system from Arabic sign language to Arabic","volume":"19","author":"Luqman","year":"2019","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1016\/j.cviu.2015.08.004","article-title":"Recent methods and databases in vision-based hand gesture recognition: A review","volume":"141","author":"Pisharady","year":"2015","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1007\/s10209-018-0622-8","article-title":"Automatic translation of Arabic text-to-Arabic sign language","volume":"18","author":"Luqman","year":"2019","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_11","unstructured":"Falco, C.M., and Jiang, X. (2017, January 19\u201322). Continuous Chinese sign language recognition with CNN-LSTM. Proceedings of the Ninth International Conference on Digital Image Processing (ICDIP 2017), Hong Kong, China."},{"key":"ref_12","first-page":"33","article-title":"A review on Indian sign language recognition","volume":"73","author":"Nair","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_13","unstructured":"Gupta, P., Agrawal, A.K., and Fatima, S. (2004, January 13\u201316). Sign Language Problem and Solutions for Deaf and Dumb People. Proceedings of the 3rd International Conference on System Modeling & Advancement in Research Trends (SMART), Sicily, Italy."},{"key":"ref_14","unstructured":"Luqman, H., Mahmoud, S., and Sidig, A. (2017). Proceedings of the International Conference of Reliable Information and Communication Technology, Johor, Malaysia, 23\u201324 April 2017, Springer."},{"key":"ref_15","unstructured":"Educational, A.L. (2000). Cultural and Scientific Organization. LAS: First Part of the Unified Arabic Sign Dictionary, Cultural and Scientific Organization."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Luqman, H., and El-Alfy, E.S.M. (2021). Towards Hybrid Multimodal Manual and Non-Manual Arabic Sign Language Recognition: mArSL Database and Pilot Study. Electronics, 10.","DOI":"10.3390\/electronics10141739"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"83199","DOI":"10.1109\/ACCESS.2020.2990699","article-title":"DeepArSLR: A Novel Signer-Independent Deep Learning Framework for Isolated Arabic Sign Language Gestures Recognition","volume":"8","author":"Aly","year":"2020","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/S0004-3702(01)00141-2","article-title":"Recognition of gestures in Arabic sign language using neuro-fuzzy systems","volume":"133","author":"Halawani","year":"2001","journal-title":"Artif. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"507614","DOI":"10.1155\/ASP.2005.2136","article-title":"Recognition of Arabic Sign Language Alphabet Using Polynomial Classifiers","volume":"2005","author":"Assaleh","year":"2005","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hemayed, E.E., and Hassanien, A.S. (2010, January 27\u201328). Edge-based recognizer for Arabic sign language alphabet (ArS2V-Arabic sign to voice). Proceedings of the 2010 International Computer Engineering Conference (ICENCO), Cairo, Egypt.","DOI":"10.1109\/ICENCO.2010.5720438"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"422","DOI":"10.1016\/j.compeleceng.2011.10.013","article-title":"A signer-independent Arabic Sign Language recognition system using face detection, geometric features, and a Hidden Markov Model","volume":"38","author":"Mohandes","year":"2012","journal-title":"Comput. Electr. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1007\/s12652-019-01209-1","article-title":"Recognition of basketball referee signals from real-time videos","volume":"11","author":"Raudonis","year":"2020","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Vaitkevi\u010dius, A., Taroza, M., Bla\u017eauskas, T., Dama\u0161evi\u010dius, R., Maskeli\u016bnas, R., and Wo\u017aniak, M. (2019). Recognition of American Sign Language Gestures in a Virtual Reality Using Leap Motion. Appl. Sci., 9.","DOI":"10.3390\/app9030445"},{"key":"ref_24","unstructured":"Krnoul, Z., Hruz, M., and Campr, P. (2010). Proceedings of the IEEE 10th International Conference on Signal Processing Proceedings, Beijing, China, 24\u201328 October 2010, IEEE."},{"key":"ref_25","unstructured":"Sabyrov, A., Mukushev, M., and Kimmelman, V. (2019, January 16\u201320). Towards Real-time Sign Language Interpreting Robot: Evaluation of Non-manual Components on Recognition Accuracy. Proceedings of the CVPR Workshops, Long Beach, CA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Elons, A., Ahmed, M., Shedid, H., and Tolba, M. (2014, January 22\u201323). Arabic sign language recognition using leap motion sensor. Proceedings of the 2014 9th International Conference on Computer Engineering Systems (ICCES), Cairo, Egypt.","DOI":"10.1109\/ICCES.2014.7030987"},{"key":"ref_27","unstructured":"Elhagry, A., and Elrayes, R.G. (2021). Egyptian Sign Language Recognition Using CNN and LSTM. arXiv."},{"key":"ref_28","unstructured":"Basnin, N., Nahar, L., and Hossain, M.S. (2020). Advances in Intelligent Systems and Computing, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"107395","DOI":"10.1016\/j.compeleceng.2021.107395","article-title":"Intelligent real-time Arabic sign language classification using attention-based inception and BiLSTM","volume":"95","author":"Abdul","year":"2021","journal-title":"Comput. Electr. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"79491","DOI":"10.1109\/ACCESS.2020.2990434","article-title":"Hand Gesture Recognition for Sign Language Using 3DCNN","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"59612","DOI":"10.1109\/ACCESS.2021.3069714","article-title":"Arabic Sign Language Recognition System Using 2D Hands and Body Skeleton Data","volume":"9","author":"Bencherif","year":"2021","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2187","DOI":"10.1007\/s13369-021-06167-5","article-title":"Isolated Video-Based Arabic Sign Language Recognition Using Convolutional and Recursive Neural Networks","volume":"47","author":"Boukdir","year":"2021","journal-title":"Arab. J. Sci. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Boukdir, A., Benaddy, M., Ellahyani, A., Meslouhi, O.E., and Kardouchi, M. (2022). 3D gesture segmentation for word-level Arabic sign language using large-scale RGB video sequences and autoencoder convolutional networks. Signal Image Video Process.","DOI":"10.1007\/s11760-022-02167-6"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Luqman, H. (2023, January 5\u20138). ArabSign: A Multi-modality Dataset and Benchmark for Continuous Arabic Sign Language Recognition. Proceedings of the 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG), Waikoloa Beach, HI, USA.","DOI":"10.1109\/FG57933.2023.10042720"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Alyami, S., Luqman, H., and Hammoudeh, M. (ACM Trans. Asian Low-Resour. Lang. Inf. Process., 2023). Isolated Arabic Sign Language Recognition Using A Transformer-Based Model and Landmark Keypoints, ACM Trans. Asian Low-Resour. Lang. Inf. Process., accepted.","DOI":"10.1145\/3584984"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"13070","DOI":"10.3390\/rs71013070","article-title":"Assessment and Calibration of a RGB-D Camera (Kinect v2 Sensor) Towards a Potential Use for Close-Range 3D Modeling","volume":"7","author":"Lachat","year":"2015","journal-title":"Remote Sens."},{"key":"ref_37","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.L., Yong, M.G., and Lee, J. (2019). MediaPipe: A Framework for Building Perception Pipelines. arXiv."},{"key":"ref_38","unstructured":"Columbia Regional Inclusive Services (2021, December 31). Basic Sign Language for Caregivers of the Deaf\/Hard of Hearing. Available online: https:\/\/youtu.be\/rkQZQhloXuE."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2019). MobileNetV2: Inverted Residuals and Linear Bottlenecks. arXiv.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_43","unstructured":"Ahmed, H.F.T., Ahmad, H., Phang, S.K., Harkat, H., and Narasingamurthi, K. (2021, January 27\u201328). Wi-Fi CSI Based Human Sign Language Recognition using LSTM Network. Proceedings of the 2021 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT), Bandung, Indonesia."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"6645","DOI":"10.1007\/s00521-020-04780-3","article-title":"Operational neural networks","volume":"32","author":"Kiranyaz","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1016\/j.neunet.2021.02.028","article-title":"Self-organized Operational Neural Networks with Generative Neurons","volume":"140","author":"Kiranyaz","year":"2021","journal-title":"Neural Netw."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1109\/TMM.2021.3070438","article-title":"A Comprehensive Study on Deep Learning-based Methods for Sign Language Recognition","volume":"24","author":"Adaloglou","year":"2021","journal-title":"IEEE Trans. Multimed."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1880","DOI":"10.1109\/TMM.2018.2889563","article-title":"A Deep Neural Framework for Continuous Sign Language Recognition by Iterative Training","volume":"21","author":"Cui","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"von Agris, U., Knorr, M., and Kraiss, K.F. (2008, January 17\u201319). The significance of facial features for automatic sign language recognition. Proceedings of the 2008 8th IEEE International Conference on Automatic Face Gesture Recognition, Amsterdam, The Netherlands.","DOI":"10.1109\/AFGR.2008.4813472"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Santa, U., Tazreen, F., and Chowdhury, S.A. (2017, January 22\u201324). Bangladeshi hand sign language recognition from video. Proceedings of the 2017 20th International Conference of Computer and Information Technology (ICCIT), Dhaka, Bangladesh.","DOI":"10.1109\/ICCITECHN.2017.8281818"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Rastgoo, R., Kiani, K., and Escalera, S. (2018). Multi-Modal Deep Hand Sign Language Recognition in Still Images Using Restricted Boltzmann Machine. Entropy, 20.","DOI":"10.3390\/e20110809"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"192527","DOI":"10.1109\/ACCESS.2020.3032140","article-title":"Deep Learning-Based Approach for Sign Language Gesture Recognition With Efficient Hand Gesture Representation","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"105238","DOI":"10.1016\/j.compbiomed.2022.105238","article-title":"Robust biometric system using session invariant multimodal EEG and keystroke dynamics by the ensemble of self-ONNs","volume":"142","author":"Rahman","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"104882","DOI":"10.1016\/j.bspc.2023.104882","article-title":"Deep learning-based middle cerebral artery blood flow abnormality detection using flow velocity waveform derived from transcranial Doppler ultrasound","volume":"85","author":"Podder","year":"2023","journal-title":"Biomed. Signal Process. Control"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7156\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:32:58Z","timestamp":1760128378000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/16\/7156"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,14]]},"references-count":53,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["s23167156"],"URL":"https:\/\/doi.org\/10.3390\/s23167156","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,14]]}}}