{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T05:04:39Z","timestamp":1784264679471,"version":"3.55.0"},"reference-count":39,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T00:00:00Z","timestamp":1784073600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Around 1.5% of the global population is suffering from speech impairments; the major causes for this are cerebral palsy and ALS, and the only way for these individuals to communicate is through Augmentative and Alternative Communication (AAC). These systems are either electronic or non-electronic. Based on new study developments, electronic methods, such as Brain\u2013Computer Interaction (BCI) and eye-gaze-based communication, are assessed as the best choices, but they have their own limitations, incorporating limited adaptability to changing conditions, such as setup variations and user fatigue, which reduces the system\u2018s robustness. Our previous study, Netravad, shows potential for addressing these gaps, but it lacks multilingual support and will not yield the same results under changing lighting conditions. This study, Thrivaad, provides multilingual support and text prediction and integrates optimized deep learning to accurately capture eye movements even in varying environmental lighting conditions. Thrivaad uses eye movements as input from a webcam, and the Optuna-optimized YOLOv5 model is used to detect the eye direction accurately. Then communication is established in English, Malayalam, and Hindi. The text-prediction feature of this system improves communication by reducing the number of eye gestures required to form a message. This study included a total of 60 participants across three age groups with 35,263 eye-sign images collected. With this data, the YOLOv5 model is trained and then optimized by Optuna. The proposal system provides accurate eye direction, text prediction, multilingual support, and improved adaptability to changing conditions for eye-based AAC.<\/jats:p>","DOI":"10.3390\/s26144503","type":"journal-article","created":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T03:34:23Z","timestamp":1784172863000},"page":"4503","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Thrivaad: A Multilingual, Predictive Eye-Sign-Based AAC System Powered by Optimized Deep Learning"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1097-5576","authenticated-orcid":false,"given":"Rajesh Kannan","family":"Megalingam","sequence":"first","affiliation":[{"name":"Humanitarian Technology (HuT) Labs, Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kollam 690525, Kerrla, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sakthiprasad","family":"Kuttankulangara Manoharan","sequence":"additional","affiliation":[{"name":"Humanitarian Technology (HuT) Labs, Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kollam 690525, Kerrla, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dhilna","family":"Cheriyan Manjooran","sequence":"additional","affiliation":[{"name":"Humanitarian Technology (HuT) Labs, Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kollam 690525, Kerrla, India"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dhanaraj","family":"Kamble","sequence":"additional","affiliation":[{"name":"Humanitarian Technology (HuT) Labs, Department of Electronics and Communication Engineering, Amrita Vishwa Vidyapeetham, Amritapuri, Kollam 690525, Kerrla, India"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,15]]},"reference":[{"key":"ref_1","unstructured":"GBD 2021 Nervous System Disorders Collaborators (2024). Global, regional, and national burden of disorders affecting the nervous system, 1990\u20132021: A systematic analysis for the Global Burden of Disease Study 2021. Lancet Neurol., 23, 344\u2013381."},{"key":"ref_2","first-page":"424","article-title":"Estimation of prevalence and screening of communication disorders in North and Middle Andaman and Nicobar Islands","volume":"8","author":"Prabhu","year":"2020","journal-title":"Clin. Epidemiol. Glob. Health"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Elsahar, Y., Hu, S., Bouazza-Marouf, K., Kerr, D., and Mansor, A. (2019). Augmentative and Alternative Communication (AAC) Advances: A Review of Configurations for Individuals with a Speech Disability. Sensors, 19.","DOI":"10.20944\/preprints201903.0033.v1"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Rega, A., Somma, F., and Simeoli, R. (2018). A review of scientific studies on the effectiveness of speech-generating devices to stimulate communication in people with autism. Proceedings of the ICERI2018 Proceedings, Seville, Spain, 12\u201314 November 2018, IATED Publications.","DOI":"10.21125\/iceri.2018.2182"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5130","DOI":"10.1109\/TNNLS.2022.3202643","article-title":"Eye-LRCN: A Long-Term Recurrent Convolutional Network for Eye Blink Completeness Detection","volume":"35","author":"Lira","year":"2024","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kunjumon, J., and Megalingam, R.K. (2019). Hand Gesture Recognition System For Translating Indian Sign Language Into Text And Speech. Proceedings of the 2019 International Conference on Smart Systems and Inventive Technology (ICSSIT), Tirunelveli, India, 27\u201329 November 2019, IEEE.","DOI":"10.1109\/ICSSIT46314.2019.8987762"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"185","DOI":"10.3233\/TAD-180197","article-title":"Low-cost hand orthotic device to improve hand function of stroke patients","volume":"30","author":"Megalingam","year":"2019","journal-title":"Technol. Disabil."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1007\/s40474-023-00271-x","article-title":"A Scoping Review of Eye Tracking Technology for Communication: Current Progress and Next Steps","volume":"10","author":"Vessoyan","year":"2023","journal-title":"Curr. Dev. Disord. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Megalingam, R.K., and Menon, G.G. (2023). Face Detection and Adaptive Zooming for Intelligent Lecturer Tracking System. Proceedings of the 2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN), Salem, India, 19\u201320 June 2023, IEEE.","DOI":"10.1109\/ICPCSN58827.2023.00064"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1007\/s12311-024-01675-0","article-title":"Optimizing Communication in Ataxia: A Multifaceted Approach to Alternative and Augmentative Communication (AAC)","volume":"23","author":"Vogel","year":"2024","journal-title":"Cerebellum"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, Y.-J., Aryanti, A., and Hong, Q.-B. (2026). A Multi-Model CNN Approach Using Pre-Trained Network for Improved Hand Gesture Recognition. Appl. Syst. Innov., 9.","DOI":"10.3390\/asi9050100"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019, January 4\u20138). Optuna: A Next-generation Hyperparameter Optimization Framework. Proceedings of the 25th ACM SIGKDD, Anchorage, AK, USA.","DOI":"10.1145\/3292500.3330701"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2759","DOI":"10.1109\/TNSRE.2024.3435016","article-title":"A Hybrid BCI Integrating EEG and Eye-Tracking for Assisting Clinical Communication in Patients With Disorders of Consciousness","volume":"32","author":"Yi","year":"2024","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"74499","DOI":"10.1109\/ACCESS.2024.3405954","article-title":"Smarter Aging: Developing a Foundational Elderly Activity Monitoring System with AI and GUI Interface","volume":"12","author":"Htet","year":"2024","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2007","DOI":"10.1109\/TLA.2018.8447369","article-title":"Equivalence Relation Between Widgets for GUIs in Mobile Applications","volume":"16","author":"Meneses","year":"2018","journal-title":"IEEE Lat. Am. Trans."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Chintala, S., Teja, S., Sunku, V., and Megalingam, R. (2025). Interprocess communication through System V queues: A multi-terminal and GUI-based demonstration. Proceedings of the 2025 International Conference on Robotics and Mechatronics (ICRM), IEEE.","DOI":"10.1109\/ICRM66809.2025.11349051"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Paing, M.P., Juhong, A., and Pintavirooj, C. (2022). Design and Development of an Assistive System Based on Eye Tracking. Electronics, 11.","DOI":"10.3390\/electronics11040535"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2900","DOI":"10.1044\/2022_AJSLP-21-00396","article-title":"Developing Augmentative and Alternative Communication Systems in Languages Other Than English: A Scoping Review","volume":"31","author":"Amery","year":"2022","journal-title":"Am. J. Speech Lang. Pathol."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Destyanto, T.Y.R., and Lin, R.F. (2022). Evaluating the Effectiveness of Complexity Features of Eye Movement on Computer Activities Detection. Healthcare, 10.","DOI":"10.3390\/healthcare10061016"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Suzuki, K., Miah, A.S.M., and Shin, J. (2025). Deep Learning-Based Eye-Writing Recognition with Improved Preprocessing and Data Augmentation Techniques. Sensors, 25.","DOI":"10.3390\/s25206325"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"69838","DOI":"10.1109\/ACCESS.2024.3402334","article-title":"Netravaad: Interactive Eye Based Communication System for People With Speech Issues","volume":"12","author":"Megalingam","year":"2024","journal-title":"IEEE Access"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Megalingam, R.K., Manoharan, S.K., and Riju, G. (2024). Accurate gaze estimation: An innovative eye movement prediction architecture. Proceedings of the 2024 7th International Conference on Circuit Power and Computing Technologies (ICCPCT), IEEE.","DOI":"10.1109\/ICCPCT61902.2024.10673363"},{"key":"ref_23","unstructured":"Megalingam, R.K., and Manoharan, S.K. (2024). Eye Sign Language Communication System. (20240264666), U.S. Patent Application No., Available online: https:\/\/patents.justia.com\/patent\/20240264666."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Trnka, K., Yarrington, D., McCaw, J., McCoy, K.F., and Pennington, C. (2007). The effects of word prediction on communication rate for AAC. Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, Association for Computational Linguistics. Companion Volume, Short Papers (NAACL-Short \u201807).","DOI":"10.3115\/1614108.1614152"},{"key":"ref_25","first-page":"1727","article-title":"Research on Web GUI Image Recognition System Based on Improved Convolutional Neural Network","volume":"21","author":"Zhao","year":"2022","journal-title":"J. Web Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"41952","DOI":"10.1109\/ACCESS.2022.3164075","article-title":"Eye-Tracking Assistive Technologies for Individuals With Amyotrophic Lateral Sclerosis","volume":"10","author":"Edughele","year":"2022","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1299","DOI":"10.1109\/TBCAS.2020.3027242","article-title":"Spintronic Sensors Based on Magnetic Tunnel Junctions for Wireless Eye Movement Gesture Control","volume":"14","author":"Tanwear","year":"2020","journal-title":"IEEE Trans. Biomed. Circuits Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, T., and Mo, H. (2023). Research on Perception and Control Technology for Dexterous Robot Operation. Electronics, 12.","DOI":"10.3390\/electronics12143065"},{"key":"ref_29","unstructured":"Harikrishnan, B.A., Pradeep, K., Kushal, K., Kumar, S.S., and Mohan, M. (2023). Blinkify\u2014A IoT Based Device for Speech Impaired. Proceedings of the 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), Delhi, India, 6\u20138 July 2023, IEEE."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/s11554-023-01336-1","article-title":"Development of a Real-Time Eye Movement-Based Computer Interface for Communication with Improved Accuracy for Disabled People Under Natural Head Movements","volume":"20","author":"Chhimpa","year":"2023","journal-title":"J. Real-Time Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Vasiljevas, M., Dama\u0161evi\u010dius, R., and Maskeli\u016bnas, R. (2023). A Human-Adaptive Model for User Performance and Fatigue Evaluation during Gaze-Tracking Tasks. Electronics, 12.","DOI":"10.3390\/electronics12051130"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Bissoli, A., Lavino-Junior, D., Sime, M., Encarna\u00e7\u00e3o, L., and Bastos-Filho, T. (2019). A Human\u2013Machine Interface Based on Eye Tracking for Controlling and Monitoring a Smart Home Using the Internet of Things. Sensors, 19.","DOI":"10.3390\/s19040859"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1109\/TNSRE.2021.3083815","article-title":"A Data-Driven Framework for Intention Prediction via Eye Movement With Applications to Assistive Systems","volume":"29","author":"Koochaki","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Abbas, M.R., Mutlag, A.H., and Gharghan, S.K. (2023). Eye-to-Text Communication Based on Human-Computer Interface Method. Proceedings of the 2023 IEEE 13th International Conference on System Engineering and Technology (ICSET), Shah Alam, Malaysia, 2 October 2023, IEEE.","DOI":"10.1109\/ICSET59111.2023.10295157"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Saleh, N., and Tarek, A. (2021). Vision-Based Communication System for Patients with Amyotrophic Lateral Sclerosis. Proceedings of the 3rd Novel Intelligent and Leading Emerging Sciences Conference (NILES), Giza, Egypt, 23\u201325 October 2021, IEEE.","DOI":"10.1109\/NILES53778.2021.9600487"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1784","DOI":"10.1017\/S0033291720003608","article-title":"Prediction of depression symptoms in individual subjects with face and eye movement tracking","volume":"52","author":"Stolicyn","year":"2022","journal-title":"Psychol. Med."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Futami, K., Tabuchi, Y., Murao, K., and Terada, T. (2022). Exploring Gaze Movement Gesture Recognition Method for Eye-Based Interaction Using Eyewear with Infrared Distance Sensor Array. Electronics, 11.","DOI":"10.3390\/electronics11101637"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"7961","DOI":"10.1038\/s41598-023-34310-9","article-title":"Blink-To-Live Eye-Based Communication System for Users with Speech Impairments","volume":"13","author":"Ezzat","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ramprasad, R., and Manne, D. (2024). Eyeball Tracking Based Communication System for Disabled People. Proceedings of the 2024 10th International Conference on Advanced Computing and Communication Systems (ICACCS), Coimbatore, India, 14\u201315 March 2024, IEEE.","DOI":"10.1109\/ICACCS60874.2024.10717256"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4503\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T04:19:03Z","timestamp":1784261943000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4503"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,15]]},"references-count":39,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["s26144503"],"URL":"https:\/\/doi.org\/10.3390\/s26144503","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,15]]}}}