{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T17:44:40Z","timestamp":1792604680187,"version":"4.5.0"},"reference-count":50,"publisher":"Public Library of Science (PLoS)","issue":"10","license":[{"start":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T00:00:00Z","timestamp":1792540800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>Dysarthria is a common motor speech disorder affecting individuals with neurological conditions, with high prevalence among stroke survivors. Effective treatment of dysarthria requires accurate assessment of severity and the provision of speech exercises tailored to the degree of impairment. However, traditional speech intervention faces several limitations, including limited accessibility to therapists and low treatment adherence rates. AI-driven solutions have the potential to address these limitations by automating the detection of dysarthria severity and recommending personalized exercises. Although substantial progress has been made in the automated detection and classification of dysarthria severity, the performance of these approaches on less controlled, real-world speech remains insufficiently characterized. This proof-of-concept study describes the development and evaluation of Whisper-ST1, a novel framework for classifying dysarthric speech severity from audio files by fine-tuning a pretrained automatic speech recognition model. Our framework builds on previous studies by incorporating real-world data alongside traditional research datasets, adapting a model architecture pretrained on a large corpus of audio data, applying a progressive unfreezing strategy to prevent catastrophic forgetting of relevant speech features while promoting flexible adaptation to the new task, and implementing an ordinal regression loss to minimize the impact of misclassification errors. Ablation experiments demonstrate the performance contributions of individual pipeline components in isolation. Whisper-ST1 achieves 79% [67% \u2013 100%] per-speaker F1 in the classification of dysarthria severity when tested on real-world samples, and shows strong agreement with two speech therapists\u2019 ratings of dysarthria (Weighted Cohen\u2019s kappa of 0.814 [0.400 \u2013 1.000] and 0.878 [0.647 \u2013 1.000]), comparable to inter-therapist agreement (0.851 [0.550 \u2013 1.000]). Model errors are predominantly confined to adjacent severity levels, suggesting preservation of ordinal structure. These findings demonstrate the feasibility of leveraging pretrained speech models for dysarthria severity classification in heterogeneous audio conditions. and provide a foundational step toward AI-assisted tools for remote monitoring and support of dysarthria rehabilitation.<\/jats:p>","DOI":"10.1371\/journal.pdig.0001794","type":"journal-article","created":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T17:29:16Z","timestamp":1792603756000},"page":"e0001794","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["An artificial intelligence-based audio recognition system for dysarthria home-based rehabilitation: A proof-of-concept study"],"prefix":"10.1371","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8043-309X","authenticated-orcid":true,"given":"Vicente","family":"Sancenon","sequence":"first","affiliation":[{"name":"Synapxe","place":["Singapore, Singapore"],"department":["Department of Future Technology & Innovation - Data, AI & Innovation"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pamela Y. S.","family":"Oh","sequence":"additional","affiliation":[{"name":"National University Health System","place":["Singapore, Singapore"],"department":["Rehabilitation, Ng Teng Fong General Hospital and Jurong Community Hospital"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-4799-7200","authenticated-orcid":true,"given":"Elvina K. W.","family":"Tay","sequence":"additional","affiliation":[{"name":"Jurong Community Hospital, National University Health System","place":["Singapore, Singapore"],"department":["Department of Post-Acute & Continuing Care"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jarrod Z. X.","family":"Lee","sequence":"additional","affiliation":[{"name":"National University Health System","place":["Singapore, Singapore"],"department":["Rehabilitation, Ng Teng Fong General Hospital and Jurong Community Hospital"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Megan L. E.","family":"Soon","sequence":"additional","affiliation":[{"name":"National University Health System","place":["Singapore, Singapore"],"department":["Rehabilitation, Ng Teng Fong General Hospital and Jurong Community Hospital"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Han Leong","family":"Goh","sequence":"additional","affiliation":[{"name":"Synapxe","place":["Singapore, Singapore"],"department":["Department of Future Technology & Innovation - Data, AI & Innovation"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicodemus","family":"Oey","sequence":"additional","affiliation":[{"name":"National University Health System","place":["Singapore, Singapore"],"department":["Division of Rehabilitation Medicine, Department of Medicine, National University Hospital"]},{"name":"Nanyang Technological University","place":["Singapore, Singapore"],"department":["Lee Kong Chian School of Medicine"]},{"name":"National University of Singapore","place":["Singapore, Singapore"],"department":["Yong Loo Lin School of Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andy W. A.","family":"Ta","sequence":"additional","affiliation":[{"name":"Synapxe","place":["Singapore, Singapore"],"department":["Department of Future Technology & Innovation - Data, AI & Innovation"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,10,21]]},"reference":[{"key":"pdig.0001794.ref001","doi-asserted-by":"crossref","first-page":"1629640","DOI":"10.3389\/fneur.2025.1629640","article-title":"Stroke-associated dysarthria","volume":"16","author":"L-X Lin","year":"2025","journal-title":"Front Neurol"},{"key":"pdig.0001794.ref002","volume-title":"Motor speech disorders\u2005: substrates, differential diagnosis, and management","author":"JR Duffy","year":"2005","edition":"2"},{"issue":"5","key":"pdig.0001794.ref003","doi-asserted-by":"crossref","first-page":"547","DOI":"10.23736\/S1973-9087.20.06185-7","article-title":"Speech rehabilitation in dysarthria after stroke: a systematic review of the studies","volume":"56","author":"R Chiaramonte","year":"2020","journal-title":"Eur J Phys Rehabil Med"},{"key":"pdig.0001794.ref004","author":"Health M.o.","year":"2023"},{"issue":"8","key":"pdig.0001794.ref005","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.3390\/brainsci12081011","article-title":"The reliability and validity of speech-language pathologists\u2019 estimations of intelligibility in dysarthria","volume":"12","author":"ME Hirsch","year":"2022","journal-title":"Brain Sci"},{"issue":"1","key":"pdig.0001794.ref006","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1159\/000111799","article-title":"Living with dysarthria: evaluation of a self-report questionnaire","volume":"60","author":"L Hartelius","year":"2008","journal-title":"Folia Phoniatr Logop"},{"issue":"4","key":"pdig.0001794.ref007","first-page":"251","article-title":"Assessment of orofacial strength in patients with dysarthria","volume":"16","author":"NP Solomon","year":"2008","journal-title":"J Med Speech Lang Pathol"},{"issue":"13","key":"pdig.0001794.ref008","article-title":"Assessment of dysarthria using one-word speech recognition with hidden markov models","volume":"34","author":"SH Lee","year":"2019","journal-title":"J Korean Med Sci"},{"issue":"2","key":"pdig.0001794.ref009","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1044\/2015_JSLHR-S-15-0271","article-title":"Comparison of intelligibility measures for adults with parkinson\u2019s disease, adults with multiple sclerosis, and healthy controls","volume":"59","author":"KL Stipancic","year":"2016","journal-title":"J Speech Lang Hear Res"},{"issue":"1","key":"pdig.0001794.ref010","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1121\/10.0016806","article-title":"Intelligibility as a measure of speech perception: current approaches, challenges, and recommendations","volume":"153","author":"MM Baese-Berk","year":"2023","journal-title":"J Acoust Soc Am"},{"issue":"3","key":"pdig.0001794.ref011","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1044\/1058-0360(2008\/025)","article-title":"Principles of motor learning in treatment of motor speech disorders","volume":"17","author":"E Maas","year":"2008","journal-title":"Am J Speech Lang Pathol"},{"issue":"1","key":"pdig.0001794.ref012","doi-asserted-by":"crossref","DOI":"10.1044\/1092-4388(2008\/018)","article-title":"Principles of experience-dependent neural plasticity: implications for rehabilitation after brain damage","volume":"51","author":"JA Kleim","year":"2008","journal-title":"J Speech Lang Hear Res"},{"key":"pdig.0001794.ref013","doi-asserted-by":"crossref","first-page":"149643","DOI":"10.1016\/j.brainres.2025.149643","article-title":"The neuroplastic brain: current breakthroughs and emerging frontiers","volume":"1858","author":"P Gazerani","year":"2025","journal-title":"Brain Res"},{"issue":"2","key":"pdig.0001794.ref014","doi-asserted-by":"crossref","first-page":"197","DOI":"10.3390\/brainsci12020197","article-title":"Feasibility of a telerehabilitation adaptation of the be clear speech treatment program for non-progressive dysarthria","volume":"12","author":"B-M Whelan","year":"2022","journal-title":"Brain Sci"},{"key":"pdig.0001794.ref015","doi-asserted-by":"crossref","first-page":"100429","DOI":"10.1016\/j.eclinm.2020.100429","article-title":"The effects of intensive speech treatment on intelligibility in Parkinson\u2019s disease: a randomised controlled trial","volume":"24","author":"ES Levy","year":"2020","journal-title":"EClinicalMedicine"},{"issue":"3","key":"pdig.0001794.ref016","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.1044\/2022_AJSLP-21-00255","article-title":"A systematic review of adherence strategies for adult populations in speech-language pathology treatment","volume":"31","author":"RS Bartlett","year":"2022","journal-title":"Am J Speech Lang Pathol"},{"issue":"3","key":"pdig.0001794.ref017","doi-asserted-by":"crossref","first-page":"136","DOI":"10.47102\/annals-acadmedsg.V43N3p136","article-title":"Post community hospital discharge rehabilitation attendance: self-perceived barriers and participation over time","volume":"43","author":"AWL Chen","year":"2014","journal-title":"Ann Acad Med Singap"},{"key":"pdig.0001794.ref018","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TNSRE.2022.3169814","article-title":"Automated dysarthria severity classification: a study on acoustic features and deep learning techniques","volume":"30","author":"AA Joshy","year":"2022","journal-title":"IEEE Trans Neural Syst Rehabil Eng"},{"key":"pdig.0001794.ref019","doi-asserted-by":"crossref","first-page":"1346297","DOI":"10.3389\/fnhum.2024.1346297","article-title":"AFM signal model for dysarthric speech classification using speech biomarkers","volume":"18","author":"SM Shabber","year":"2024","journal-title":"Front Hum Neurosci"},{"key":"pdig.0001794.ref020","doi-asserted-by":"crossref","first-page":"103047","DOI":"10.1016\/j.specom.2024.103047","article-title":"Pre-trained models for detection and severity level classification of dysarthria from speech","volume":"158","author":"F Javanmardi","year":"2024","journal-title":"Speech Commun"},{"key":"pdig.0001794.ref021","unstructured":"Sanjay B, Priyadharshini MK, Vijayalakshmi P, Nagarajan T. Severity Classification and Dysarthric Speech Detection using Self-Supervised Representations. In: ICON; 2024."},{"key":"pdig.0001794.ref022","doi-asserted-by":"crossref","unstructured":"Mannepalli RS, Pusuluri A, A.Patil H. Dysarthria severity classification using phase based features of LP residual. In: 2024 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC); 2024. pp. 1\u20135. https:\/\/doi.org\/10.1109\/apsipaasc63619.2025.10848959","DOI":"10.1109\/APSIPAASC63619.2025.10848959"},{"issue":"22","key":"pdig.0001794.ref023","doi-asserted-by":"crossref","first-page":"25435","DOI":"10.1007\/s11042-024-20053-w","article-title":"DySARNet: a lightweight self-attention deep learning model for diagnosing dysarthria from speech recordings","volume":"84","author":"FJP Montalbo","year":"2024","journal-title":"Multimed Tools Appl"},{"issue":"1","key":"pdig.0001794.ref024","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1186\/s13636-024-00357-3","article-title":"Automatic dysarthria detection and severity level assessment using CWT-layered CNN model","author":"S Sajiha","year":"2024","journal-title":"Eurasip J Audio Speech Music Process"},{"issue":"8","key":"pdig.0001794.ref025","doi-asserted-by":"crossref","DOI":"10.1007\/s42452-025-07260-2","article-title":"Enhancing dysarthria severity classification: efficient audio based deep learning models","volume":"7","author":"VJ Varma","year":"2025","journal-title":"Discov Appl Sci"},{"issue":"1","key":"pdig.0001794.ref026","doi-asserted-by":"crossref","first-page":"42080","DOI":"10.1038\/s41598-025-26049-2","article-title":"A hybrid cross-attentive CNN-BiLSTM-transformer network for dysarthria severity classification","volume":"15","author":"MS Remya","year":"2025","journal-title":"Sci Rep"},{"issue":"1","key":"pdig.0001794.ref027","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.icte.2021.07.004","article-title":"Dysarthric-speech detection using transfer learning with convolutional neural networks","volume":"8","author":"SR Mani Sekhar","year":"2022","journal-title":"ICT Express"},{"issue":"11","key":"pdig.0001794.ref028","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0001076","article-title":"Speaker-independent dysarthria severity classification using self-supervised transformers and multi-task learning","volume":"4","author":"B Kadirvelu","year":"2025","journal-title":"PLOS Digit Health"},{"key":"pdig.0001794.ref029","doi-asserted-by":"crossref","unstructured":"Yeo EJ, Choi K, Kim S, Chung M. Automatic severity classification of dysarthric speech by using self-supervised model with multi-task learning. In: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2023. pp. 1\u20135. https:\/\/doi.org\/10.1109\/icassp49357.2023.10094605","DOI":"10.1109\/ICASSP49357.2023.10094605"},{"key":"pdig.0001794.ref030","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.specom.2022.12.004","article-title":"Dysarthria severity classification using multi-head attention and multi-task learning","volume":"147","author":"AA Joshy","year":"2023","journal-title":"Speech Commun"},{"key":"pdig.0001794.ref031","doi-asserted-by":"crossref","unstructured":"Kim H, Hasegawa-Johnson M, Perlman A, Gunderson J, Huang TS, Watkin K, et al., Dysarthric speech database for universal access research; 2008. pp. 1741\u20134.","DOI":"10.21437\/Interspeech.2008-480"},{"issue":"4","key":"pdig.0001794.ref032","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1007\/s10579-011-9145-0","article-title":"The TORGO database of acoustic and articulatory speech from speakers with dysarthria","volume":"46","author":"F Rudzicz","year":"2011","journal-title":"Lang Resources & Evaluation"},{"key":"pdig.0001794.ref033","article-title":"Smartphone-based speech therapy for poststroke dysarthria: pilot randomized controlled trial evaluating efficacy and feasibility","volume":"26","author":"Y Kim","year":"2024","journal-title":"J Med Internet Res"},{"key":"pdig.0001794.ref034","doi-asserted-by":"crossref","unstructured":"Suh H, Dangol A, Meadan H, Miller CA, Kientz JA. Opportunities and challenges for AI-based support for speech-language pathologists. In: Proceedings of the 3rd Annual Meeting of the Symposium on Human-Computer Interaction for Work; 2024. pp. 1\u201314. https:\/\/doi.org\/10.1145\/3663384.3663387","DOI":"10.1145\/3663384.3663387"},{"issue":"10","key":"pdig.0001794.ref035","doi-asserted-by":"crossref","DOI":"10.2196\/18858","article-title":"Mobile apps for speech-language therapy in adults with communication disorders: review of content and quality","volume":"8","author":"A Vaezipour","year":"2020","journal-title":"JMIR mHealth uHealth"},{"key":"pdig.0001794.ref036","unstructured":"Radford A, et al. Robust speech recognition via large-scale weak supervision. In: Proceedings of the 40th International Conference on Machine Learning; 2023. pp. 28492\u2013518."},{"issue":"10267","key":"pdig.0001794.ref037","doi-asserted-by":"crossref","first-page":"2006","DOI":"10.1016\/S0140-6736(20)32340-0","article-title":"Global estimates of the need for rehabilitation based on the Global Burden of Disease study 2019: a systematic analysis for the Global Burden of Disease Study 2019","volume":"396","author":"A Cieza","year":"2021","journal-title":"Lancet"},{"issue":"1","key":"pdig.0001794.ref038","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"EJ Topol","year":"2019","journal-title":"Nat Med"},{"issue":"3","key":"pdig.0001794.ref039","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1093\/jamia\/ocz192","article-title":"A governance model for the application of AI in health care","volume":"27","author":"S Reddy","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"pdig.0001794.ref040","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1080\/0269920031000086248","article-title":"Toward an acoustic typology of motor speech disorders","volume":"17","author":"RD Kent","year":"2003","journal-title":"Clin Linguist Phon"},{"key":"pdig.0001794.ref041","unstructured":"Baevski A, et al. Wav2vec 2.0: a framework for self-supervised learning of speech representations. In: Proceedings of the 34th International Conference on Neural Information Processing Systems; 2020. Article 1044."},{"key":"pdig.0001794.ref042","doi-asserted-by":"crossref","unstructured":"Panayotov V, Chen G, Povey D, Khudanpur S. Librispeech: an ASR corpus based on public domain audio books. In: 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 2015. pp. 5206\u201310. https:\/\/doi.org\/10.1109\/icassp.2015.7178964","DOI":"10.1109\/ICASSP.2015.7178964"},{"key":"pdig.0001794.ref043","doi-asserted-by":"crossref","unstructured":"Christensen H, Cunningham S, Fox C, Green P, Hain T. A comparative study of adaptive, automatic recognition of disordered speech; 2012.","DOI":"10.21437\/Interspeech.2012-484"},{"issue":"6","key":"pdig.0001794.ref044","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1016\/j.csl.2012.11.001","article-title":"Adjusting dysarthric speech signals to be more intelligible","volume":"27","author":"F Rudzicz","year":"2013","journal-title":"Comput Speech Lang"},{"key":"pdig.0001794.ref045","article-title":"Classification of dysarthria based on the levels of severity: a systematic review.","author":"A Al-Ali","year":"2023","journal-title":"ArXiv"},{"issue":"4","key":"pdig.0001794.ref046","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1159\/000450548","article-title":"Speech characteristics and intelligibility in adults with mild and moderate intellectual disabilities","volume":"68","author":"MC Coppens-Hofman","year":"2016","journal-title":"Folia Phoniatr Logop"},{"key":"pdig.0001794.ref047","doi-asserted-by":"crossref","first-page":"106301","DOI":"10.1016\/j.jcomdis.2023.106301","article-title":"Assessing speech intelligibility of pathological speech in sentences and word lists: the contribution of phoneme-level measures","volume":"102","author":"W Xue","year":"2023","journal-title":"J Commun Disord"},{"issue":"12","key":"pdig.0001794.ref048","doi-asserted-by":"crossref","first-page":"4718","DOI":"10.1044\/2021_JSLHR-21-00197","article-title":"\u201cYou Say Severe, I Say Mild\u201d: toward an empirical classification of dysarthria severity","volume":"64","author":"KL Stipancic","year":"2021","journal-title":"J Speech Lang Hear Res"},{"key":"pdig.0001794.ref049","volume-title":"Sortformer: a novel approach for permutation-resolved speaker supervision in speech-to-text systems.","author":"TJ Park","year":"2024"},{"issue":"10","key":"pdig.0001794.ref050","doi-asserted-by":"crossref","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"A survey on transfer learning","volume":"22","author":"SJ Pan","year":"2010","journal-title":"IEEE Trans Knowl Data Eng"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001794","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,21]],"date-time":"2026-10-21T17:29:22Z","timestamp":1792603762000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001794"}},"subtitle":[],"editor":[{"given":"Pengxu","family":"Wei","sequence":"first","affiliation":[{"name":"National Research Center for Rehabilitation Technical Aids","place":["CHINA"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,10,21]]},"references-count":50,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,10,21]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001794","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,21]]}}}