{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T11:10:28Z","timestamp":1788952228147,"version":"build-2803163510"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T00:00:00Z","timestamp":1788912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>\n                    Online voice-based applications and speech communication have grown as a result of the revolutionary rise of smart gadgets and social media. The rapid advancement of deep learning (DL) has transformed the field of audio processing, enabling smooth human-computer interaction. DL approaches have been used to develop speech-to-text (STT) systems across various languages and topics. These models require a large amount of training data: extensive corpora of continuous speech utterances collected from numerous speakers, along with their corresponding transcripts. In this study, we explore the use of state-of-the-art pre-trained models like Wav2Vec2 XLSR-53 and Whisper-small for developing STT systems in the Telugu language, addressing the challenge of limited data availability and demonstrating satisfactory results [23.62% Word Error Rate (WER), 4.12% Character Error Rate (CER)] even when fine-tuned on a smaller dataset. To evaluate model performance, we employed a k-fold cross-validation approach with values of\n                    <jats:italic>k<\/jats:italic>\n                    = 2 to\n                    <jats:italic>k<\/jats:italic>\n                    = 5, and compared the results with the conventional train-test split method. The results indicate that at\n                    <jats:italic>k<\/jats:italic>\n                    = 5, the Wav2Vec2 XLSR-53 model achieved a cross-validation Character Error Rate(WER) of 23.62% and a Character Error Rate (CER) of 4.12%, and the Whisper-small model yielded a cross-validation WER of 28.67% and a CER of 5.48%. These findings suggest that the k-fold cross-validation strategy, at k = 5, enhances the robustness of Wav2Vec2 XLSR-53 in low-resource language scenarios where training data is limited. For Whisper-small, however, the baseline train-test split (WER: 27.76%) outperformed all k-fold configurations tested (k = 2: 32.05%, k = 3: 29.72%, k = 4: 29.40%, k = 5: 28.67%), indicating that the benefit of k-fold cross-validation observed for Wav2Vec2 XLSR-53 does not generalize across model architectures. Additionally, the models were evaluated on unseen noisy data, and both models demonstrated satisfactory performance.\n                  <\/jats:p>","DOI":"10.3389\/frai.2026.1883049","type":"journal-article","created":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T10:17:20Z","timestamp":1788949040000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring Wav2Vec2 XLSR-53 and Whisper-small for Telugu language speech-to-text: a fine-tuning approach"],"prefix":"10.3389","volume":"9","author":[{"given":"Muzaffar Ahmad","family":"Dar","sequence":"first","affiliation":[{"name":"Department of Mathematics, School of Advanced Sciences, Vellore Institute of Technology","place":["Vellore, Tamil Nadu, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sri Gani Kaarthikeya","family":"Kammula","sequence":"additional","affiliation":[{"name":"Department of Quantum AI, School of Computer Science and Engineering, Vellore Institute of Technology, Katpadi","place":["Vellore, Tamil Nadu, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jagalingam","family":"Pushparaj","sequence":"additional","affiliation":[{"name":"Department of Quantum AI, School of Computer Science and Engineering, Vellore Institute of Technology, Katpadi","place":["Vellore, Tamil Nadu, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Venkata Sai","family":"Santhosh","sequence":"additional","affiliation":[{"name":"Space Inventive, Hyderabad","place":["Telangana, India"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2504.18582","article-title":"Speaker diarization for low resource languages through wav2vec fine tuning","author":"Abdullah","year":"2025","journal-title":"arXiv preprint, abs\/2504.18582"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2410.16330","article-title":"End to end transformer based automatic speech recognition for northern kurdish: a pioneering approach","author":"Abdullah","year":"2024","journal-title":"arXiv preprint, abs\/2410.16330"},{"key":"B3","doi-asserted-by":"publisher","first-page":"108","DOI":"10.37652\/juaps.2022.176500","article-title":"Central kurdish automatic speech recognition using deep learning","volume":"16","author":"Abdullah","year":"2022","journal-title":"J. Univ. Anbar Pure Sci"},{"key":"B4","doi-asserted-by":"publisher","first-page":"51527","DOI":"10.1109\/ACCESS.2025.3552935","article-title":"Machine learning and deep learning approaches for accent recognition: a review","volume":"13","author":"Ahmad Dar","year":"2025","journal-title":"IEEE Access"},{"key":"B5","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1049\/sil2.12057","article-title":"Arabic speech recognition using end-to-end deep learning","volume":"15","author":"Alsayadi","year":"2021","journal-title":"IET Signal Process"},{"key":"B6","first-page":"4218","article-title":"\u201cCommon voice: a massively-multilingual speech corpus,\u201d","volume-title":"Proceedings of the twelfth language resources and evaluation conference","author":"Ardila","year":"2020"},{"key":"B7","doi-asserted-by":"publisher","author":"Baevski","year":"2020","DOI":"10.48550\/ARXIV.2006.11477"},{"key":"B8","doi-asserted-by":"publisher","author":"Chadha","year":"2022","DOI":"10.48550\/ARXIV.2203.16578"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1002\/9781394214624.ch13","author":"Chowdary","year":"2024","journal-title":"Transformer-Based Multilingual Automatic Speech Recognition (ASR) Model for Dravidian Languages"},{"key":"B10","doi-asserted-by":"publisher","author":"Conneau","year":"2020","DOI":"10.48550\/ARXIV.2006.13979"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/AIIoT58432.2024.10574740","article-title":"\u201cHybrid architecture cnn-blstm for automatic speech recognition,\u201d","author":"Dar","year":"2024","journal-title":"2024 3rd International conference on artificial intelligence for internet of things (AIIoT)"},{"key":"B12","doi-asserted-by":"publisher","first-page":"110505","DOI":"10.1016\/j.apacoust.2024.110505","article-title":"Bi-directional lstm-based isolated spoken word recognition for kashmiri language utilizing mel-spectrogram feature","volume":"231","author":"Dar","year":"2025","journal-title":"Appl. Acoust"},{"key":"B13","doi-asserted-by":"crossref","DOI":"10.21437\/Interspeech.2021-1339","article-title":"\u201cMucs 2021: multilingual and code-switching asr challenges for low resource indian languages,\u201d","volume-title":"Proceedings of interspeech 2021","author":"Diwan","year":"2021"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3197","DOI":"10.21437\/Interspeech.2018-2117","article-title":"\u201cTdnn-based multilingual speech recognition system for low resource indian languages,\u201d","author":"Fathima","year":"2018","journal-title":"Proceedings of interspeech 2018"},{"key":"B15","doi-asserted-by":"publisher","first-page":"020004","DOI":"10.1063\/5.0178629","article-title":"\u201cAsr for indian regional language using nvidia's nemo toolkit,\u201d","author":"Ghadekar","year":"2023","journal-title":"WOMEN IN PHYSICS: 7th IUPAP international conference on women in physics"},{"key":"B16","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1145\/1143844.1143891","article-title":"\u201cConnectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,\u201d","author":"Graves","year":"2006","journal-title":"Proceedings of the 23rd international conference on machine learning (ICML '06)"},{"key":"B17","first-page":"6494","article-title":"\u201cOpen-source multi-speaker speech corpora for building Gujarati, Kannada, Malayalam, Marathi, Tamil and Telugu speech synthesis systems,\u201d","volume-title":"Proceedings of the twelfth language resources and evaluation conference","author":"He","year":"2020"},{"key":"B18","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1109\/CCOMS.2019.8821629","article-title":"\u201cA speech recognition system for bengali language using recurrent neural network,\u201d","author":"Islam","year":"2019","journal-title":"2019 IEEE 4th international conference on computer and communication systems (ICCCS)"},{"key":"B19","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s13636-025-00395-5","article-title":"Comparative performance analysis of end-to-end asr models on Indo-Aryan and Dravidian languages within India's linguistic landscape","volume":"2025","author":"Jain","year":"2025","journal-title":"EURASIP J. Audio, Speech Music Process"},{"key":"B20","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1109\/ICMOCE.2015.7489765","article-title":"\u201cAutomatic speech recognition based odia system,\u201d","author":"Karan","year":"2015","journal-title":"2015 International conference on microwave, optical and communication engineering (ICMOCE)"},{"key":"B21","doi-asserted-by":"publisher","first-page":"372","DOI":"10.1109\/ICATCCT.2016.7912026","article-title":"\u201cPower-normalized cepstral coefficients (pncc) for punjabi automatic speech recognition using phone-based modelling in htk,\u201d","author":"Kaur","year":"2016","journal-title":"Proceedings of the 2016 2nd international conference on applied and theoretical computing and communication technology (iCATccT)"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2109.03969","article-title":"Multilingual speech recognition for low-resource indian languages using multi-task conformer","author":"Krishna","year":"2021"},{"key":"B23","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1007\/s10772-020-09757-0","article-title":"Hindi speech recognition using time delay neural network acoustic modeling with i-vector adaptation","volume":"25","author":"Kumar","year":"2020","journal-title":"Int. J. Speech Technol"},{"key":"B24","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1007\/s41870-020-00586-7","article-title":"Hindi speech recognition in noisy environment using hybrid technique","volume":"13","author":"Kumar","year":"2021","journal-title":"Int. J. Inform. Technol"},{"key":"B25","doi-asserted-by":"publisher","first-page":"050001","DOI":"10.1063\/5.0214118","article-title":"\u201cAn end-to-end deep learning based continuous speech recognition in Kannada,\u201d","author":"Kumari","year":"2024","journal-title":"Proceedings of the TIM22 physics conference"},{"key":"B26","doi-asserted-by":"publisher","first-page":"717","DOI":"10.1007\/s10772-024-10132-6","article-title":"An automatic speech recognition system in odia language using attention mechanism and data augmentation","volume":"27","author":"Majhi","year":"2024","journal-title":"Int. J. Speech Technol"},{"key":"B27","doi-asserted-by":"publisher","first-page":"2965","DOI":"10.1016\/j.patcog.2008.05.008","article-title":"Invited paper: automatic speech recognition: history, methods and challenges","volume":"41","author":"O'Shaughnessy","year":"2008","journal-title":"Pattern Recogn"},{"key":"B28","doi-asserted-by":"publisher","first-page":"1015","DOI":"10.1145\/2733373.2806390","article-title":"\u201cESC: dataset for environmental sound classification,\u201d","author":"Piczak","year":"2015","journal-title":"Proceedings of the 23rd annual ACM conference on multimedia"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2202.04971","article-title":"Asrpu: a programmable accelerator for low-power automatic speech recognition","author":"Pinto","year":"2022"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2012.03411","article-title":"Mls: a large-scale multilingual dataset for speech research","author":"Pratap","year":"2020","journal-title":"arXiv, abs\/2012.03411"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2212.04356","article-title":"Robust speech recognition via large-scale weak supervision","author":"Radford","year":"2022"},{"key":"B32","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1007\/s10772-025-10186-0","article-title":"DNNT (deep neural network for Telugu): a framework for speech recognition of Telugu language with parallel computing approach","volume":"28","author":"Reddy","year":"2025","journal-title":"Int. J. Speech Technol"},{"key":"B33","doi-asserted-by":"publisher","author":"Salesky","year":"2021","DOI":"10.48550\/ARXIV.2102.01757"},{"key":"B34","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1007\/978-3-540-30176-9_22","article-title":"\u201cImplementation of Tamil speech recognition system using neural networks,\u201d","author":"Saraswathi","year":"2004","journal-title":"Advances in neural networks"},{"key":"B35","doi-asserted-by":"publisher","first-page":"6878","DOI":"10.1109\/ICASSP39728.2021.9413386","article-title":"\u201cThe accented english speech recognition challenge 2020: open datasets, tracks, baselines, results and methods,\u201d","author":"Shi","year":"2021","journal-title":"ICASSP 2021"},{"key":"B36","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1007\/s00607-019-00753-0","article-title":"English speech recognition based on deep learning with multiple features","volume":"102","author":"Song","year":"2019","journal-title":"Computing"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1474","DOI":"10.1109\/TASLP.2022.3166400","article-title":"Non-autoregressive asr modeling using pre-trained language models for chinese speech recognition","volume":"30","author":"Yu","year":"2022","journal-title":"IEEE\/ACM Trans. Audio Speech Lang. Process"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1883049\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,9]],"date-time":"2026-09-09T10:17:24Z","timestamp":1788949044000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1883049\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,9]]},"references-count":37,"alternative-id":["10.3389\/frai.2026.1883049"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1883049","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,9]]},"article-number":"1883049"}}