{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T22:08:20Z","timestamp":1785362900354,"version":"3.55.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2022,11,25]],"date-time":"2022-11-25T00:00:00Z","timestamp":1669334400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Department of Science & Technology (DST), India","award":["INT\/RUS\/RFBR\/393"],"award-info":[{"award-number":["INT\/RUS\/RFBR\/393"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2023,1,31]]},"abstract":"<jats:p>The Sign Language Recognition system intends to recognize the Sign language used by the hearing and vocally impaired populace. The interpretation of isolated sign language from static and dynamic gestures is a difficult study field in machine vision. Managing quick hand movement, facial expression, illumination variations, signer variation, and background complexity are amongst the most serious challenges in this arena. While deep learning-based models have been used to accomplish the entirety of the field's state-of-the-art outcomes, the previous issues have not been fully addressed. To overcome these issues, we propose a Hybrid Neural Network Architecture for the recognition of Isolated Indian and Russian Sign Language. In the case of static gesture recognition, the proposed framework deals with the 3D Convolution Net with an atrous convolution mechanism for spatial feature extraction. For dynamic gesture recognition, the proposed framework is an integration of semantic spatial multi-cue feature detection, extraction, and Temporal-Sequential feature extraction. The semantic spatial multi-cue feature detection and extraction module help in the generation of feature maps for Full-frame, pose, face, and hand. For face and hand detection, GradCam and Camshift algorithm have been used. The temporal and sequential module consists of a modified auto-encoder with a GELU activation function for abstract high-level feature extraction and a hybrid attention layer. The hybrid attention layer is an integration of segmentation and spatial attention mechanism. The proposed work also involves creating a novel multi-signer, single, and double-handed Isolated Sign representation dataset for Indian and Russian Sign Language. The experimentation was done on the novel dataset created. The accuracy obtained for Static Isolated Sign Recognition was 99.76%, and the accuracy obtained for Dynamic Isolated Sign Recognition was 99.85%. We have also compared the performance of our proposed work with other baseline models with benchmark datasets, and our proposed work proved to have better performance in terms of Accuracy metrics.<\/jats:p>","DOI":"10.1145\/3530989","type":"journal-article","created":{"date-parts":[[2022,4,20]],"date-time":"2022-04-20T12:00:42Z","timestamp":1650456042000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":43,"title":["Static and Dynamic Isolated Indian and Russian Sign Language Recognition with Spatial and Temporal Feature Detection Using Hybrid Neural Network"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8472-7933","authenticated-orcid":false,"given":"E.","family":"Rajalakshmi","sequence":"first","affiliation":[{"name":"School of Computing, SASTRA Deemed University, Thanjavur, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2257-0640","authenticated-orcid":false,"given":"R.","family":"Elakkiya","sequence":"additional","affiliation":[{"name":"School of Computing, SASTRA Deemed University, Thanjavur, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9063-5362","authenticated-orcid":false,"given":"Alexey L.","family":"Prikhodko","sequence":"additional","affiliation":[{"name":"Novosibirsk State Technical University, Russian Federation"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3016-3647","authenticated-orcid":false,"given":"M. G.","family":"Grif","sequence":"additional","affiliation":[{"name":"Novosibirsk State Technical University, Russian Federation"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1889-0692","authenticated-orcid":false,"given":"Maxim A.","family":"Bakaev","sequence":"additional","affiliation":[{"name":"Novosibirsk State Technical University, Russian Federation"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5205-5263","authenticated-orcid":false,"given":"Jatinderkumar R.","family":"Saini","sequence":"additional","affiliation":[{"name":"Symbiosis Institute of Computer Studies and Research, Symbiosis International (Deemed University), Pune, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2653-3780","authenticated-orcid":false,"given":"Ketan","family":"Kotecha","sequence":"additional","affiliation":[{"name":"Symbiosis Centre for Applied Artificial Intelligence, Symbiosis International (Deemed University), Pune, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5328-7672","authenticated-orcid":false,"given":"V.","family":"Subramaniyaswamy","sequence":"additional","affiliation":[{"name":"School of Computing, SASTRA Deemed University, Thanjavur, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,25]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Y. Saleh and G. Issa. 2020. Arabic sign language recognition through deep neural networks fine-tuning. https:\/\/www.learntechlib.org\/p\/217934\/.","DOI":"10.3991\/ijoe.v16i05.13087"},{"key":"e_1_3_1_3_2","unstructured":"K. Wangchuk K. Wangchuk and P. Riyamongkol. 2020. Bhutanese sign language hand-shaped alphabets and digits detection and recognition (doctoral dissertation naresuan university). http:\/\/nuir.lib.nu.ac.th\/dspace\/handle\/123456789\/2491."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-019-08345-y"},{"issue":"3","key":"e_1_3_1_5_2","doi-asserted-by":"crossref","first-page":"200","DOI":"10.35860\/iarej.700564","article-title":"Turkish sign language digits classification with CNN using different optimizers","volume":"4","author":"Sevli O.","year":"2020","unstructured":"O. Sevli and N. Kemalo\u011flu. 2020. Turkish sign language digits classification with CNN using different optimizers. Int. Adv. Res. Eng. J. 4, 3 (2020), 200\u2013207.","journal-title":"Int. Adv. Res. Eng. J."},{"key":"e_1_3_1_6_2","unstructured":"R. Elakkiya and E. Rajalakshmi Islan. Mendeley Data Vol. 1. https:\/\/data.mendeley.com\/datasets\/rc349j45m5\/1."},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.112"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308561.3353774"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2018.2889563"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11903"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2911077"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2915032"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00429"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01004"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2015.09.013"},{"key":"e_1_3_1_16_2","unstructured":"D. Hendrycks and K. Gimpel. 2016. Gaussian error linear units (gelus). Retrieved from https:\/\/arXiv:1606.08415."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1002\/lnc3.326"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12652-020-02396-y"},{"key":"e_1_3_1_19_2","first-page":"57","article-title":"A multilingual multimedia Indian sign language dictionary tool","author":"Diwakar S.","year":"2008","unstructured":"S. Diwakar and A. Basu. 2008. A multilingual multimedia Indian sign language dictionary tool. In Proceedings of the International Joint Conference on Natural language Processing (IJCNLP\u201908). 57.","journal-title":"Proceedings of the International Joint Conference on Natural language Processing (IJCNLP\u201908)"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1353\/sls.1989.0027"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.lingua.2005.04.006"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2013.2246148"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2014.2374357"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2017.361"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.456"},{"key":"e_1_3_1_26_2","first-page":"422","volume-title":"Proceedings of the 3rd International Conference on Recent Advances in Information Technology (RAIT\u201916).","author":"Kumar A.","year":"2016","unstructured":"A. Kumar, K. Thankachan, and M. M. Dominic. 2016. Sign language recognition. In Proceedings of the 3rd International Conference on Recent Advances in Information Technology (RAIT\u201916). IEEE, 422\u2013428."},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2009.17"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/NCM.2009.357"},{"key":"e_1_3_1_29_2","first-page":"1537","volume-title":"Proceedings of the 5th IEEE Conference on Industrial Electronics and Applications.","author":"Yang Q.","year":"2010","unstructured":"Q. Yang. 2010. Chinese sign language recognition based on video sequence appearance modeling. In Proceedings of the 5th IEEE Conference on Industrial Electronics and Applications. IEEE, 1537\u20131542."},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1007\/978-3-319-38771-0_54","volume-title":"Information Technology and Intelligent Transportation Systems.","author":"Hore S.","year":"2017","unstructured":"S. Hore, S. Chatterjee, V. Santhi, N. Dey, A. S. Ashour, V. E. Balas, and F. Shi. 2017. Indian sign language recognition using optimized neural networks. In Information Technology and Intelligent Transportation Systems. Springer, Cham, 553\u2013563."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1088\/1757-899X\/1116\/1\/012126"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/WiSPNET.2017.8299784"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijleo.2017.11.158"},{"key":"e_1_3_1_34_2","first-page":"1","volume-title":"Proceedings of the 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT\u201918).","author":"Shenoy K.","year":"2018","unstructured":"K. Shenoy, T. Dastane, V. Rao, and D. Vyavaharkar. 2018. Real-time indian sign language (ISL) recognition. In Proceedings of the 9th International Conference on Computing, Communication and Networking Technologies (ICCCNT\u201918). IEEE, 1\u20139."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-6876-3_4"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1088\/1757-899X\/1116\/1\/012126"},{"key":"e_1_3_1_37_2","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association (ELRA\u201920)","author":"Mukushev M.","year":"2020","unstructured":"M. Mukushev, A. Sabyrov, A. Imashev, K. Koishibay, V. Kimmelman, and A. Sandygulova. 2020. Evaluation of manual and non-manual components for sign language recognition. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association (ELRA\u201920)."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACVW52041.2021.00008"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00812"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-019-7263-7"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2990699"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00624"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-020-09048-5"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00380"},{"key":"e_1_3_1_45_2","first-page":"1","volume-title":"Wireless Personal Commun","author":"Sharma S.","year":"2021","unstructured":"S. Sharma and S. Singh. 2021. Recognition of Indian sign language (ISL) using deep learning model. Wireless Personal Commun. (2021), 1\u201322."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1088\/1742-6596\/1873\/1\/012009"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21062227"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.17212\/2782-2001-2021-3-53-74"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2019.103777"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093512"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACVW52041.2021.00008"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00347"},{"key":"e_1_3_1_53_2","doi-asserted-by":"crossref","unstructured":"M. Maruyama S. Ghose K. Inoue P. P. Roy M. Iwamura and M. Yoshioka. 2021. Word-level sign language recognition with multi-stream neural networks focusing on local regions. Retrieved from https:\/\/arXiv:2106.15989.","DOI":"10.2139\/ssrn.4263878"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3092743"},{"issue":"2","key":"e_1_3_1_55_2","first-page":"1","article-title":"Word reordering for translation into korean sign language using syntactically-guided classification","volume":"19","author":"Jung H. Y.","year":"2019","unstructured":"H. Y. Jung, J. H. Lee, E. Min, and S. H. Na. 2019. Word reordering for translation into korean sign language using syntactically-guided classification. ACM Trans. Asian Low-Res. Lang. Info. Process. 19, 2 (2019), 1\u201320.","journal-title":"ACM Trans. Asian Low-Res. Lang. Info. Process."},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387632"},{"key":"e_1_3_1_57_2","doi-asserted-by":"crossref","unstructured":"J. Singha and K. Das. 2013. Recognition of indian sign language in live video. Retrieved from https:\/\/arXiv:1306.1301.","DOI":"10.5120\/12174-7306"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3423420"},{"key":"e_1_3_1_59_2","first-page":"44","article-title":"Deep multi-model fusion for human activity recognition using evolutionary algorithms","volume":"7","author":"Verma K. K.","year":"2021","unstructured":"K. K. Verma and B. M. Singh. 2021. Deep multi-model fusion for human activity recognition using evolutionary algorithms. Int. J. Interact. Multimedia Artific. Intell. 7 (2021), 44\u201358.","journal-title":"Int. J. Interact. Multimedia Artific. Intell."},{"key":"e_1_3_1_60_2","first-page":"11.","article-title":"Two-stage human activity recognition using 2D-ConvNet","volume":"6","author":"Verma K. K.","year":"2020","unstructured":"K. K. Verma, B. M. Singh, H. L. Mandoria, and P. Chauhan. 2020. Two-stage human activity recognition using 2D-ConvNet. Int. J. Interact. Multimedia Artific. Intell. 6 (2020), 11.","journal-title":"Int. J. Interact. Multimedia Artific. Intell."},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACVW54805.2022.00024"},{"key":"e_1_3_1_62_2","doi-asserted-by":"crossref","unstructured":"S. Srivastava A. Gangwar R. Mishra and S. Singh. 2022. Sign language recognition system using tensorflow object detection API. Retrieved from https:\/\/arXiv:2201.01486.","DOI":"10.1007\/978-3-030-96040-7_48"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530989","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3530989","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:27Z","timestamp":1750183767000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3530989"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,25]]},"references-count":61,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,31]]}},"alternative-id":["10.1145\/3530989"],"URL":"https:\/\/doi.org\/10.1145\/3530989","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,25]]},"assertion":[{"value":"2022-01-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}