{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T11:01:48Z","timestamp":1775041308833,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,6,16]],"date-time":"2022-06-16T00:00:00Z","timestamp":1655337600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Targeted Research Grant Program\u2014The National Transformation Program in King Abdulaziz City for Science and Technology\u2014Kingdom of Saudi Arabia","award":["5-18-03-001-0003"],"award-info":[{"award-number":["5-18-03-001-0003"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Sign language is the main channel for hearing-impaired people to communicate with others. It is a visual language that conveys highly structured components of manual and non-manual parameters such that it needs a lot of effort to master by hearing people. Sign language recognition aims to facilitate this mastering difficulty and bridge the communication gap between hearing-impaired people and others. This study presents an efficient architecture for sign language recognition based on a convolutional graph neural network (GCN). The presented architecture consists of a few separable 3DGCN layers, which are enhanced by a spatial attention mechanism. The limited number of layers in the proposed architecture enables it to avoid the common over-smoothing problem in deep graph neural networks. Furthermore, the attention mechanism enhances the spatial context representation of the gestures. The proposed architecture is evaluated on different datasets and shows outstanding results.<\/jats:p>","DOI":"10.3390\/s22124558","type":"journal-article","created":{"date-parts":[[2022,6,19]],"date-time":"2022-06-19T21:19:26Z","timestamp":1655673566000},"page":"4558","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Spatial Attention-Based 3D Graph Convolutional Neural Network for Sign Language Recognition"],"prefix":"10.3390","volume":"22","author":[{"given":"Muneer","family":"Al-Hammadi","sequence":"first","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Department of Civil and Environmental Engineering, Faculty of Engineering, Norwegian University of Science and Technology, H\u00f8gskoleringen 1, 7034 Trondheim, Norway"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8147-8679","authenticated-orcid":false,"given":"Mohamed A.","family":"Bencherif","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Mansour","family":"Alsulaiman","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9781-3969","authenticated-orcid":false,"given":"Ghulam","family":"Muhammad","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Mohamed Amine","family":"Mekhtiche","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6871-6633","authenticated-orcid":false,"given":"Wadood","family":"Abdul","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Yousef A.","family":"Alohali","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Tareq S.","family":"Alrayes","sequence":"additional","affiliation":[{"name":"Department of Special Education, College of Education, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Hassan","family":"Mathkour","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7720-0076","authenticated-orcid":false,"given":"Mohammed","family":"Faisal","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Center of AI & Robotics, Kuwait College of Science and Technology (KCST), Kuwait City 35004, Kuwait"}]},{"given":"Mohammed","family":"Algabri","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Science Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1780-6388","authenticated-orcid":false,"given":"Hamdi","family":"Altaheri","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0366-5932","authenticated-orcid":false,"given":"Taha","family":"Alfakih","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"}]},{"given":"Hamid","family":"Ghaleb","sequence":"additional","affiliation":[{"name":"Centre of Smart Robotics Research (CS2R), King Saud University, Riyadh 11543, Saudi Arabia"},{"name":"Software Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1504\/IJAPR.2016.079048","article-title":"A survey on manual and non-manual sign language recognition for isolated and continuous sign","volume":"3","author":"Agrawal","year":"2016","journal-title":"Int. J. Appl. Pattern Recognit."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-012-9356-9","article-title":"Vision based hand gesture recognition for human computer interaction: A survey","volume":"43","author":"Rautaray","year":"2015","journal-title":"Artif. Intell. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1064","DOI":"10.1109\/TSMCA.2011.2116004","article-title":"A framework for hand gesture recognition based on accelerometer and EMG sensors","volume":"41","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. A Syst. Hum."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"760","DOI":"10.1109\/TNSRE.2019.2896269","article-title":"Deep learning for electromyographic hand gesture signal classification using transfer learning","volume":"27","author":"Fall","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"79491","DOI":"10.1109\/ACCESS.2020.2990434","article-title":"Hand gesture recognition for sign language using 3DCNN","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Altaheri, H., Muhammad, G., Alsulaiman, M., Amin, S.U., Altuwaijri, G.A., Abdul, W., Bencherif, M.A., and Faisal, M. (2021). Deep learning techniques for classification of electroencephalogram (EEG) motor imagery (MI) signals: A review. Neural Comput. Appl., 1\u201342.","DOI":"10.1007\/s00521-021-06352-5"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1109\/TII.2018.2875149","article-title":"Automatic fruit classification using deep learning for industrial applications","volume":"15","author":"Hossain","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","first-page":"1864","article-title":"An incremental learning framework for human-like redundancy optimization of anthropomorphic manipulators","volume":"13","author":"Su","year":"2020","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Qi, W., and Su, H. (2022). A cybertwin based multimodal network for ecg patterns monitoring using deep learning. IEEE Trans. Ind. Inform.","DOI":"10.1109\/TII.2022.3159583"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21\u201326). OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Selvaraj, P., NC, G., Kumar, P., and Khapra, M. (2021). OpenHands: Making Sign Language Recognition Accessible with Pose-based Pretrained Models across Languages. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.150"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yasir, F., Prasad, P.W.C., Alsadoon, A., and Elchouemi, A. (2015, January 6\u20137). Sift based approach on bangla sign language recognition. Proceedings of the IEEE 8th International Workshop on Computational Intelligence and Applications (IWCIA), Hiroshima, Japan.","DOI":"10.1109\/IWCIA.2015.7449458"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1007\/978-3-319-13572-4_30","article-title":"Sift-based arabic sign language recognition system","volume":"334","author":"Thrwat","year":"2015","journal-title":"Adv. Intell. Syst. Comput."},{"key":"ref_14","unstructured":"Liwicki, S., and Everingham, M. (June, January USA). Automatic recognition of fingerspelled words in british sign language. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Miami, FL."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Buehler, P., Zisserman, A., and Everingham, M. (2009, January 20\u201325). Learning sign language by watching tv (using weakly aligned subtitles). Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206523"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Badhe, P., and Kulkarni, V. (2015, January 2\u20133). Indian sign language translator using gesture recognition algorithm. Proceedings of the IEEE International Conference on Computer Graphics, Bhubaneswar, India.","DOI":"10.1109\/CGVIS.2015.7449921"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1371","DOI":"10.1109\/34.735811","article-title":"Real-time american sign language recognition using desk and wearable computer based video","volume":"20","author":"Starner","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2040","DOI":"10.1109\/TPAMI.2008.123","article-title":"Sign language recognition by combining statistical DTW and independent classification","volume":"30","author":"Lichtenauer","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","first-page":"4","article-title":"Static hand gesture recognition for sign language alphabets using edge oriented histogram and multi class SVM","volume":"82","author":"Nagarajan","year":"2013","journal-title":"Int. J. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tornay, S., Razavi, M., and Doss, M. (2020, January 4\u20138). Towards multilingual sign language recognition. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054631"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1007\/s11263-016-0957-7","article-title":"Beyond temporal pooling: Recurrence and temporal convolutions for gesture recognition in video","volume":"126","author":"Pigou","year":"2018","journal-title":"Int. J. Comput. Vis."},{"key":"ref_22","unstructured":"Chen, X., and Gao, K. (2018). DenseImage network: Video spatial-temporal evolution encoding and understanding. arXiv."},{"key":"ref_23","first-page":"95","article-title":"Hand gesture recognition using 3D-CNN model","volume":"9","author":"Muhammad","year":"2019","journal-title":"IEEE Consum. Electron. Mag."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4828102","DOI":"10.1155\/2021\/4828102","article-title":"Dynamic gesture recognition algorithm based on 3D convolutional neural network","volume":"2021","author":"Liu","year":"2021","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"192527","DOI":"10.1109\/ACCESS.2020.3032140","article-title":"Deep learning-based approach for sign language gesture recognition with efficient hand gesture representation","volume":"8","author":"Muhammad","year":"2020","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Qin, W., Mei, X., Chen, Y., Zhang, Q., Yao, Y., and Hu, S. (2021, January 3\u20134). Sign Language Recognition and Translation Method based on VTN. Proceedings of the International Conference on Digital Society and Intelligent Systems, Chengdu, China.","DOI":"10.1109\/DSInS54396.2021.9670588"},{"key":"ref_27","first-page":"2398","article-title":"Hear sign language: A real-time end-to-end sign language recognition system","volume":"21","author":"Wang","year":"2022","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6039","DOI":"10.1109\/LRA.2021.3089999","article-title":"Multi-Sensor Guided Hand Gesture Recognition for a Teleoperated Robot Using a Recurrent Neural Network","volume":"6","author":"Qi","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yan, S., Xiong, Y., and Lin, D. (2018, January 2\u20137). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12328"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Shahroudy, A., Liu, J., Ng, T., and Wang, G. (2016, January 27\u201330). Ntu rgb+d: A large scale dataset for 3d human activity analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.115"},{"key":"ref_31","unstructured":"Amorim, C.C.D., Mac\u00eado, D., and Zanchettin, C. (2019, January 17\u201319). Spatial-temporal graph convolutional networks for sign language recognition. Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Jiang, S., Sun, J., Wang, L., Bai, Y., Li, K., and Fu, Y. (2021, January 21\u201325). Skeleton aware multi-modal sign language recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00380"},{"key":"ref_33","unstructured":"Jiang, S., Sun, B., Wang, L., Bai, Y., Li, K., and Fu, Y. (2021). Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble. arXiv."},{"key":"ref_34","unstructured":"Coster, M.D., Herreweghe, M.V., and Dambre, J. (2021, January 21\u201325). Isolated sign recognition from rgb video using pose flow and self-attention. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA."},{"key":"ref_35","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, L. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_36","unstructured":"Zhou, K., Huang, X., Li, Y., Zha, D., Chen, R., and Hu, X. (2020). Towards deeper graph neural networks with differentiable group normalization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1016\/j.inffus.2021.06.007","article-title":"A comprehensive survey on multimodal medical signals fusion for smart healthcare systems","volume":"76","author":"Muhammad","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.inffus.2021.02.013","article-title":"COVID-19 and non-COVID-19 classification using multi-layers fusion from lung ultrasound images","volume":"72","author":"Muhammad","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Altuwaijri, G., Muhammad, G., Altaheri, H., and Alsulaiman, M. (2022). A Multi-Branch Convolutional Neural Network with Squeeze-and-Excitation Attention Blocks for EEG-Based Motor Imagery Signals Classification. Diagnostics, 12.","DOI":"10.3390\/diagnostics12040995"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"5412","DOI":"10.1109\/TII.2021.3132340","article-title":"Attention-Inception and Long Short-Term Memory-based Electroencephalography Classification for Motor Imagery Tasks in Rehabilitation","volume":"18","author":"Amin","year":"2022","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"181340","DOI":"10.1109\/ACCESS.2020.3028072","article-title":"Autsl: A large scale multi-modal turkish sign language dataset and baseline methods","volume":"8","author":"Sincan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_42","unstructured":"Ronchetti, F., Quiroga, F., Estrebou, C., Lanzarini, L., and Rosete, A. (2016, January 3\u20137). LSA64: An Argentinian sign language dataset. Proceedings of the XXII Congreso Argentino de Ciencias de la Computaci\u00f3n, San Luis, Argentina."},{"key":"ref_43","unstructured":"Neidle, C., Thangali, A., and Sclaroff, S. (2012, January 21\u201327). Challenges in Development of the American Sign Language Lexicon Video Dataset (ASLLVD) Corpus. Proceedings of the Conference Language Resources and Evaluation Conference (LREC), Istanbul, Turkey."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Materzynska, J., Berger, G., Bax, I., and Memisevic, R. (2019, January 27\u201328). The jester dataset: A large-scale video dataset of human gestures. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea.","DOI":"10.1109\/ICCVW.2019.00349"},{"key":"ref_45","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C., Yong, M., and Lee, J. (2019, January 16\u201320). MediaPipe: A Framework for Perceiving and Processing Reality. Proceedings of the Third Workshop on Computer Vision for AR\/VR at IEEE Computer Vision and Pattern Recognition, Long Beach, CA, USA."},{"key":"ref_46","unstructured":"Google Research Team (2022, February 18). MediaPipe. Available online: https:\/\/google.github.io\/mediapipe\/solutions\/hands.html."},{"key":"ref_47","unstructured":"Kipf, T., and Welling, M. (2016). Semi-supervised classification with graph convolutional networks. arXiv."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4558\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:33:30Z","timestamp":1760139210000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/12\/4558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,16]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,6]]}},"alternative-id":["s22124558"],"URL":"https:\/\/doi.org\/10.3390\/s22124558","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,16]]}}}