{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T22:52:52Z","timestamp":1770591172343,"version":"3.49.0"},"reference-count":41,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T00:00:00Z","timestamp":1617753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"2020 Li Ka Shing Foundation Cross-disciplinary Research Grant","award":["2020LKSFG04C"],"award-info":[{"award-number":["2020LKSFG04C"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Speech assessment is an essential part of the rehabilitation procedure for patients with aphasia (PWA). It is a comprehensive and time-consuming process that aims to discriminate between healthy individuals and aphasic patients, determine the type of aphasia syndrome, and determine the patients\u2019 impairment severity levels (these are referred to here as aphasia assessment tasks). Hence, the automation of aphasia assessment tasks is essential. In this study, the performance of three automatic speech assessment models based on the speech dataset-type was investigated. Three types of datasets were used: healthy subjects\u2019 dataset, aphasic patients\u2019 dataset, and a combination of healthy and aphasic datasets. Two machine learning (ML)-based frameworks, classical machine learning (CML) and deep neural network (DNN), were considered in the design of the proposed speech assessment models. In this paper, the DNN-based framework was based on a convolutional neural network (CNN). Direct or indirect transformation of these models to achieve the aphasia assessment tasks was investigated. Comparative performance results for each of the speech assessment models showed that quadrature-based high-resolution time-frequency images with a CNN framework outperformed all the CML frameworks over the three dataset-types. The CNN-based framework reported an accuracy of 99.23 \u00b1 0.003% with the healthy individuals\u2019 dataset and 67.78 \u00b1 0.047% with the aphasic patients\u2019 dataset. Moreover, direct or transformed relationships between the proposed speech assessment models and the aphasia assessment tasks are attainable, given a suitable dataset-type, a reasonably sized dataset, and appropriate decision logic in the ML framework.<\/jats:p>","DOI":"10.3390\/s21082582","type":"journal-article","created":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T11:31:59Z","timestamp":1617795119000},"page":"2582","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Performance Evaluation of Machine Learning Frameworks for Aphasia Assessment"],"prefix":"10.3390","volume":"21","author":[{"given":"Seedahmed S.","family":"Mahmoud","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou 515041, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3684-8498","authenticated-orcid":false,"given":"Akshay","family":"Kumar","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou 515041, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Youcun","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou 515041, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yiting","family":"Tang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou 515041, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiang","family":"Fang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, College of Engineering, Shantou University, Shantou 515041, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.specom.2018.04.001","article-title":"Automatic quantitative analysis of spontaneous aphasic speech","volume":"100","author":"Le","year":"2018","journal-title":"Speech Commun."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Qin, Y., Lee, T., Feng, S., and Kong, A.P.H. (2018). Automatic Speech Assessment for People with Aphasia Using TDNN-BLSTM with Multi-Task Learning. Interspeech 2018, 3418\u20133422.","DOI":"10.21437\/Interspeech.2018-1630"},{"key":"ref_3","unstructured":"Le, D. (2017). Towards Automatic Speech-Language Assessment for Aphasia Rehabilitation. [Ph.D. Thesis, University of Michigan]."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1264","DOI":"10.1109\/TBME.2012.2183367","article-title":"Novel Speech Signal Processing Algorithms for High-Accuracy Classification of Parkinson\u2019s Disease","volume":"59","author":"Tsanas","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","first-page":"1583","article-title":"A comparison of GMM-HMM and DNN-HMM based pronunciation verification techniques for use in the assessment of childhood apraxia of speech","volume":"1","author":"Shahin","year":"2014","journal-title":"Interspeech"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1016\/j.compeleceng.2016.08.021","article-title":"An incremental method combining density clustering and support vector machines for voice pathology detection","volume":"57","author":"Amami","year":"2017","journal-title":"Comput. Electr. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"16246","DOI":"10.1109\/ACCESS.2018.2816338","article-title":"Voice Disorder Identification by Using Machine Learning Techniques","volume":"6","author":"Verde","year":"2018","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2775","DOI":"10.1007\/s11517-020-02260-3","article-title":"Evaluation of train and test performance of machine learning algorithms and Parkinson diagnosis with statistical measurements","volume":"58","author":"Elen","year":"2020","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1007\/s11265-019-01511-3","article-title":"An End-to-End Approach to Automatic Speech Assessment for Cantonese-speaking People with Aphasia","volume":"92","author":"Qin","year":"2020","journal-title":"J. Signal Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3191","DOI":"10.1109\/JBHI.2020.3011104","article-title":"An efficient deep learning based method for speech as-sessment of Mandarin-speaking aphasic patients","volume":"24","author":"Mahmoud","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_11","unstructured":"Mahmoud, S.S., Fang, Q., Tang, Y., Alsulami, M., and Alotaibi, M. (2019, January 28\u201329). Automatic Mandarin Vowels Recognition Framework for Aphasic Patients Rehabilitation. Proceedings of the 13th IEEE-EMBS International Summer School and Symposium on Medical Devices and Biosensors (MDBS2019), Chengdu, China."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2187","DOI":"10.1109\/TASLP.2016.2598428","article-title":"Automatic Assessment of Speech Intelligibility for Individuals with Aphasia","volume":"24","author":"Le","year":"2016","journal-title":"IEEE ACM Trans. Audio Speech Lang. Process."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kohlschein, C., Schmitt, M., Schuller, B., Jeschke, S., and Werner, C.J. (2017, January 12\u201315). A machine learning based system for the automatic evaluation of aphasia speech. Proceedings of the 2017 IEEE 19th International Conference on e-Health Networking, Applications and Services (Healthcom), Dalian, China.","DOI":"10.1109\/HealthCom.2017.8210766"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.cortex.2012.12.006","article-title":"Automated classification of primary progressive aphasia subtypes from narrative speech transcripts","volume":"55","author":"Fraser","year":"2014","journal-title":"Cortex"},{"key":"ref_15","unstructured":"Li, S. (2014). Speech Therapy, Huaxia Press."},{"key":"ref_16","first-page":"291","article-title":"The Aachen aphasia test","volume":"Volume 42","author":"Rose","year":"1984","journal-title":"Advances in Neurology. Progress in Aphasiology"},{"key":"ref_17","unstructured":"Goodglass, H., and Kaplan, E. (1983). The Assessment of Aphasia and Related Disorders, Williams & Wilkins."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.cmpb.2011.02.015","article-title":"Comparison of machine learning methods for classifying aphasic and non-aphasic speakers","volume":"104","author":"Juhola","year":"2011","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_19","unstructured":"Gu, Y., Bahrani, M., Billot, A., Lai, S., Braun, E.J., Varkanitsa, M., Bighetto, J., Rapp, B., Parrish, T.B., and Caplan, D. (July, January 30). A machine learning approach for predicting post-stroke aphasia recovery: A pilot study. Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments, Corfu, Greece."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1682","DOI":"10.1002\/hbm.25321","article-title":"Machine learning-based multimodal prediction of language outcomes in chronic aphasia","volume":"42","author":"Kristinsson","year":"2021","journal-title":"Hum. Brain Mapp."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.cortex.2019.05.007","article-title":"Machine learning in the clinical and language characterization of primary progressive aphasia variants","volume":"119","author":"Cuetos","year":"2019","journal-title":"Cortex"},{"key":"ref_22","unstructured":"Rabiner, L.R., and Schafer, R.W. (2010). Theory and Application of Digital Speech Processing, Pearson."},{"key":"ref_23","unstructured":"Slaney, M. (1998). Auditory toolbox. Interval Research Corporation, Interval Research Corporation. Technical Report No. 10."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ryant, N., Slaney, M., Liberman, M., Shriberg, E., and Yuan, J. (2014, January 20\u201323). Highly Accurate Mandarin Tone Classification in the Absence of Pitch Information. Proceedings of the 7th International Conference on Speech Prosody 2014, Dublin, Ireland.","DOI":"10.21437\/SpeechProsody.2014-123"},{"key":"ref_25","unstructured":"Wainer, J. (2016). Comparison of 14 different families of classification algorithms on 115 binary datasets. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1016\/j.ins.2018.05.053","article-title":"Multi- granulation Rough-Fuzzy Clustering Based on Shadowed Sets","volume":"507","author":"Zhou","year":"2020","journal-title":"Inf. Sci."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3457","DOI":"10.1109\/TCOMM.2019.2894711","article-title":"Resource Assignment Based on Dynamic Fuzzy Clustering in Elastic Optical Networks with Multi-Core Fibers","volume":"67","author":"Yang","year":"2019","journal-title":"IEEE Trans. Commun."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2014\/201243","article-title":"A Geometric Fuzzy-Based Approach for Airport Clustering","volume":"2014","author":"Postorino","year":"2014","journal-title":"Adv. Fuzzy Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1109\/TSP.2002.800406","article-title":"Adaptive instantaneous frequency estimation of multicomponent FM signals using quadratic time-frequency distributions","volume":"50","author":"Hussain","year":"2002","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.bspc.2006.02.001","article-title":"Time-frequency analysis of normal and abnormal biological signals","volume":"1","author":"Mahmoud","year":"2006","journal-title":"Biomed. Signal Process. Control."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dodge, S., and Karam, L. (2016, January 6\u20138). Understanding how image quality affects deep neural networks. Proceedings of the 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX), Lisbon, Portugal.","DOI":"10.1109\/QoMEX.2016.7498955"},{"key":"ref_32","first-page":"451","article-title":"Prasad Kannojia, Effects of Varying Resolution on Performance of CNN based Image Classification: An Experi-mental Study","volume":"6","author":"Suresh","year":"2018","journal-title":"Int. J. Comput. Sci. Eng."},{"key":"ref_33","first-page":"040901","article-title":"Development of convolutional neural network and its application in image classification: A survey","volume":"58","author":"Wang","year":"2019","journal-title":"Opt. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep residual learning for image recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Smith, L.N. (2017, January 24\u201331). Cyclical learning rates for training neural networks. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017, Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.58"},{"key":"ref_36","unstructured":"Kingma, D.P., and Ba, J. (2015, January 5\u20138). Adam: A method for stochastic optimization. Proceedings of the International Conference Learning Representation (ICLR), San Diego, CA, USA."},{"key":"ref_37","unstructured":"Krogh, A., and Hertz, J.A. (1991, January 2\u20135). A simple weight decay can improve generalization. Proceedings of the 4th International Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_38","unstructured":"Howard, J. (2020, June 07). fastai, Available online: https:\/\/github.com\/fastai\/fastai."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Christensen, H., Cunningham, S., Fox, C., Green, P., and Hain, T. (2012, January 9\u201313). A comparative study of adaptive, automatic recognition of disordered speech. Proceedings of the Thirteenth Annual Conference of the International Speech Communication Association, Portland, OR, USA.","DOI":"10.21437\/Interspeech.2012-484"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Mengistu, K.T., and Rudzicz, F. (2011, January 5\u20138). Comparing Humans and Automatic Speech Recognition Systems in Recognizing Dysarthric Speech. Proceedings of the Advances in Artificial Intelligence, Perth, Australia.","DOI":"10.1007\/978-3-642-21043-3_36"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"3924","DOI":"10.1016\/j.eswa.2015.01.033","article-title":"Exploring the influence of general and specific factors on the recognition accuracy of an ASR system for dysarthric speaker","volume":"42","author":"Mustafa","year":"2015","journal-title":"Expert Syst. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2582\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:11:30Z","timestamp":1760364690000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/8\/2582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,7]]},"references-count":41,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["s21082582"],"URL":"https:\/\/doi.org\/10.3390\/s21082582","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,7]]}}}