{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T15:43:21Z","timestamp":1781019801690,"version":"3.54.1"},"reference-count":27,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2022,8,25]],"date-time":"2022-08-25T00:00:00Z","timestamp":1661385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Language plays a vital role in the communication of ideas, thoughts, and information to others. Hearing-impaired people also understand our thoughts using a language known as sign language. Every country has a different sign language which is based on their native language. In our research paper, our major focus is on Indian Sign Language, which is mostly used by hearing- and speaking-impaired communities in India. While communicating our thoughts and views with others, one of the most essential factors is listening. What if the other party is not able to hear or grasp what you are talking about? This situation is faced by nearly every hearing-impaired person in our society. This led to the idea of introducing an audio to Indian Sign Language translation system which can erase this gap in communication between hearing-impaired people and society. The system accepts audio and text as input and matches it with the videos present in the database created by the authors. If matched, it shows corresponding sign movements based on the grammar rules of Indian Sign Language as output; if not, it then goes through the processes of tokenization and lemmatization. The heart of the system is natural language processing which equips the system with tokenization, parsing, lemmatization, and part-of-speech tagging.<\/jats:p>","DOI":"10.3390\/fi14090253","type":"journal-article","created":{"date-parts":[[2022,8,25]],"date-time":"2022-08-25T21:28:12Z","timestamp":1661462892000},"page":"253","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Translating Speech to Indian Sign Language Using Natural Language Processing"],"prefix":"10.3390","volume":"14","author":[{"given":"Purushottam","family":"Sharma","sequence":"first","affiliation":[{"name":"Amity School of Engineering & Technology, Amity University, Noida 201301, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Devesh","family":"Tulsian","sequence":"additional","affiliation":[{"name":"Amity School of Engineering & Technology, Amity University, Noida 201301, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9925-112X","authenticated-orcid":false,"given":"Chaman","family":"Verma","sequence":"additional","affiliation":[{"name":"Department of Media and Educational Informatics, Faculty of Informatics, Eotvos Lorand University, 1053 Budapest, Hungary"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pratibha","family":"Sharma","sequence":"additional","affiliation":[{"name":"Amity School of Engineering & Technology, Amity University, Noida 201301, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1264-6899","authenticated-orcid":false,"given":"Nancy","family":"Nancy","sequence":"additional","affiliation":[{"name":"Amity School of Engineering & Technology, Amity University, Noida 201301, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,25]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"HamNoSys-Representing Sign Language Data in Language Resources and Language Processing Contexts","volume":"4","author":"Hanke","year":"2004","journal-title":"LREC"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Prikhodko, A., Grif, M., and Bakaev, M. (2020, January 25\u201327). Sign Language Recognition Based on Notations and Neural Networks. Proceedings of the Communications in Computer and Information Science, Valletta, Malta.","DOI":"10.1007\/978-3-030-65218-0_34"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sonawane, P., Shah, K., Patel, P., Shah, S., and Shah, J. (2021, January 4\u20135). Speech to Indian Sign Language (ISL) Translation System. Proceedings of the IEEE 2021 International Conference on Computing, Communication, and Intelligent Systems, ICCCIS, Greater Noida, India.","DOI":"10.1109\/ICCCIS51004.2021.9397097"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Tewari, Y., Soni, P., Singh, S., Turlapati, M.S., and Bhuva, A. (2021, January 25\u201327). Real Time Sign Language Recognition Framework for Two Way Communication. Proceedings of the International Conference on Communication, Information and Computing Technology, ICCICT, Mumbai, India.","DOI":"10.1109\/ICCICT50803.2021.9510094"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kunjumon, J., and Megalingam, R.K. (2019, January 27\u201329). Hand gesture recognition system for translating indian sign language into text and speech. Proceedings of the 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India.","DOI":"10.1109\/ICSSIT46314.2019.8987762"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Gangadia, D., Chamaria, V., Doshi, V., and Gandhi, J. (2020, January 16\u201318). Indian Sign Language Interpretation and Sentence Formation. Proceedings of the 2020 IEEE Pune Section International Conference, PuneCon 2020, Pune, India.","DOI":"10.1109\/PuneCon50868.2020.9362383"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shangeetha, R.K., Valliammai, V., and Padmavathi, S. (2012, January 14\u201315). Computer vision based approach for Indian sign language character recognition. Proceedings of the 2012 International Conference on Machine Vision and Image Processing, MVIP, Coimbatore, India.","DOI":"10.1109\/MVIP.2012.6428790"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sawant, S.N., and Kumbhar, M.S. (2014, January 8\u201310). Real time sign language recognition using PCA. Proceedings of the 2014 IEEE International Conference on Advanced Communication, Control and Computing Technologies, ICACCCT 2014, Ramanathapuram, India.","DOI":"10.1109\/ICACCCT.2014.7019333"},{"key":"ref_9","unstructured":"Papadogiorgaki, M., Grammalidis, N., Tzovaras, D., and Strintzis, M.G. (2005, January 4\u20138). Text-to-sign language synthesis tool; Text-to-sign language synthesis tool. Proceedings of the 2005 13th European Signal Processing Conference, Antalya, Turkey."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sonare, B., Padgal, A., Gaikwad, Y., and Patil, A. (2021, January 21). Video-based sign language translation system using machine learning. Proceedings of the 2021 2nd International Conference for Emerging Technology, INCET, Belagavi, Inida.","DOI":"10.1109\/INCET51464.2021.9456176"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kanvinde, A., Revadekar, A., Tamse, M., Kalbande, D.R., and Bakereywala, N. (2021, January 25\u201327). Bidirectional Sign Language Translation. Proceedings of the International Conference on Communication, Information and Computing Technology, ICCICT, Mumbai, India.","DOI":"10.1109\/ICCICT50803.2021.9510146"},{"key":"ref_12","unstructured":"Qi, J., Wang, D., Jiang, Y., and Liu, R. (2013, January 19\u201323). Auditory features based on gammatone filters for robust speech recognition. Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nair, M.S., Nimitha, A.P., and Idicula, S.M. (2016, January 1\u20133). Conversion of Malayalam text to Indian sign language using synthetic animation. Proceedings of the 2016 International Conference on Next Generation Intelligent Systems (ICNGIS), Kottayam, India.","DOI":"10.1109\/ICNGIS.2016.7854002"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Qi, J., Wang, D., Xu, J., and Tejedor Noguerales, J. (2013, January 5\u201329). Bottleneck features based on gammatone frequency cepstral coefficients. Proceedings of the International Speech Communication Association, Lyon, France.","DOI":"10.21437\/Interspeech.2013-435"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Grif, M., and Manueva, Y. (2016, January 1\u20133). Semantic analyses of text to translate to Russian sign language. Proceedings of the 2016 11th Inter-national Forum on Strategic Technology, IFOST, Novosibirsk, Russia.","DOI":"10.1109\/IFOST.2016.7884107"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dhanjal, A.S., and Singh, W. (2019, January 25\u201328). Comparative Analysis of Sign Language Notation Systems for Indian Sign Language. Proceedings of the 2019 Second International Conference on Advanced Computational and Communication Paradigms (ICACCP), Sikkim, India.","DOI":"10.1109\/ICACCP.2019.8883009"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Varghese, M., and Nambiar, S.K. (2018, January 21\u201322). English to SiGML conversion for sign language generation. Proceedings of the 2018 Interna-tional Conference on Circuits and Systems in Digital Enterprise Technology (ICCSDET), Kottayam, India.","DOI":"10.1109\/ICCSDET.2018.8821212"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Raghavan, R.J., Prasad, K.A., and Muraleedharan, R. (2013, January 10\u201311). Animation System for Indian Sign Language Communication using LOTS Notation. Proceedings of the 2013 International Conference on Emerging Trends in Communication, Control, Signal Processing and Computing Applications (C2SPCA), Bangalore, India.","DOI":"10.1109\/C2SPCA.2013.6749444"},{"key":"ref_19","unstructured":"Dhivyasri, S., Krishnaa Hari, K.B., Akash, M., Sona, M., Divyapriya, S., and Krishnaveni, V. (2021, January 13\u201314). An efficient approach for interpretation of Indian sign language using machine learning. Proceedings of the 2021 3rd International Conference on Signal Processing and Communication, ICPSC, Coimbatore, India."},{"key":"ref_20","unstructured":"Allen, J.M., and Foulds, R.A. (2015, January 7\u201319). An approach to animating sign language: A spoken english to sign english translator system. Proceedings of the IEEE Annual Northeast Bioengineering Conference, NEBEC, Troy, NY, USA."},{"key":"ref_21","unstructured":"Patel, B.D., Patel, H.B., Khanvilkar, M.A., Patel, N.R., and Akilan, T. (September, January 30). ES2ISL: An Advancement in Speech to Sign Language Trans-lation using 3D Avatar Animator. Proceedings of the Canadian Conference on Electrical and Computer Engineering, London, ON, Canada."},{"key":"ref_22","unstructured":"Agarwal, A. (2015, January 4\u20135). Generating HamNoSys signs from the user\u2019s input. Proceedings of the 2015 1st International Conference on Next Generation Computing Technologies (NGCT), Dehradun, India."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Priya, L., Sathya, A., and Raja, S.K.S. (2020, January 10). Indian and English Language to Sign Language Translator-an Automated Portable Two Way Communicator for Bridging Normal and Deprived Ones. Proceedings of the ICPECTS 2020\u2013IEEE 2nd International Con-ference on Power, Energy, Control and Transmission Systems, Proceedings, Chennai, India.","DOI":"10.1109\/ICPECTS49113.2020.9336983"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Saija, K., Sangeetha, S., and Shah, V. (2019, January 13\u201315). WordNet Based Sign Language MachineTranslation: From English Voice to ISL Gloss. Proceedings of the 2019 IEEE 16th India Council International Conference (INDICON), Rajkot, India.","DOI":"10.1109\/INDICON47234.2019.9029074"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ahire, P.G., Tilekar, K.B., Jawake, T.A., and Warale, P.B. (2015, January 26\u201327). Two-way communicator between deaf and dumb people and normal people. Proceedings of the 1st International Conference on Computing, Communication, Control and Automation, IC-CUBEA 2015, Pune, India.","DOI":"10.1109\/ICCUBEA.2015.131"},{"key":"ref_26","unstructured":"Jamil, T. (September, January 30). Design and Implementation of an Intelligent System to translate Arabic Text into Arabic Sign Language. Proceedings of the Canadian Conference on Electrical and Computer Engineering, London, ON, Canada."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1016\/j.procs.2017.08.363","article-title":"The Use of Hidden Markov Model in Natural ARABIC Language Processing: A survey","volume":"113","author":"Suleiman","year":"2017","journal-title":"Procedia Comput. Sci."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/9\/253\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:15:12Z","timestamp":1760141712000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/14\/9\/253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,25]]},"references-count":27,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["fi14090253"],"URL":"https:\/\/doi.org\/10.3390\/fi14090253","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,25]]}}}