{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T06:40:10Z","timestamp":1784875210958,"version":"3.55.0"},"reference-count":40,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T00:00:00Z","timestamp":1721174400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"NRF (National Research Foundation) of Korea","doi-asserted-by":"publisher","award":["RS-2024-00340935"],"award-info":[{"award-number":["RS-2024-00340935"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Patients suffering from Parkinson\u2019s disease suffer from voice impairment. In this study, we introduce models to classify normal and Parkinson\u2019s patients using their speech. We used an AST (audio spectrogram transformer), a transformer-based speech classification model that has recently outperformed CNN-based models in many fields, and a CNN-based PSLA (pretraining, sampling, labeling, and aggregation), a high-performance model in the existing speech classification field, for the study. This study compares and analyzes the models from both quantitative and qualitative perspectives. First, qualitatively, PSLA outperformed AST by more than 4% in accuracy, and the AUC was also higher, with 94.16% for AST and 97.43% for PSLA. Furthermore, we qualitatively evaluated the ability of the models to capture the acoustic features of Parkinson\u2019s through various CAM (class activation map)-based XAI (eXplainable AI) models such as GradCAM and EigenCAM. Based on PSLA, we found that the model focuses well on the muffled frequency band of Parkinson\u2019s speech, and the heatmap analysis of false positives and false negatives shows that the speech features are also visually represented when the model actually makes incorrect predictions. The contribution of this paper is that we not only found a suitable model for diagnosing Parkinson\u2019s through speech using two different types of models but also validated the predictions of the model in practice.<\/jats:p>","DOI":"10.3390\/s24144625","type":"journal-article","created":{"date-parts":[[2024,7,17]],"date-time":"2024-07-17T15:15:19Z","timestamp":1721229319000},"page":"4625","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Exploring Spectrogram-Based Audio Classification for Parkinson\u2019s Disease: A Study on Speech Classification and Qualitative Reliability Verification"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6408-3701","authenticated-orcid":false,"given":"Seung-Min","family":"Jeong","sequence":"first","affiliation":[{"name":"Department of AI & Informatics, Graduate School, Sangmyung University, Hongjimun 2-gil 20, Jongno-gu, Seoul 03016, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6209-4570","authenticated-orcid":false,"given":"Seunghyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of AI & Informatics, Graduate School, Sangmyung University, Hongjimun 2-gil 20, Jongno-gu, Seoul 03016, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6504-3333","authenticated-orcid":false,"given":"Eui Chul","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Human-Centered Artificial Intelligence, Sangmyung University, Hongjimun 2-gil 20, Jongno-gu, Seoul 03016, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8219-9663","authenticated-orcid":false,"given":"Han Joon","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Neurology, Seoul National University College of Medicine, Seoul National University Hospital, Daehak-ro 101, Jongno-gu, Seoul 03080, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"17013","DOI":"10.1038\/nrdp.2017.13","article-title":"Parkinson disease","volume":"3","author":"Poewe","year":"2017","journal-title":"Nat. Rev. Dis. Primers"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1038\/s41531-022-00410-y","article-title":"Incidence of Parkinson disease in North America","volume":"8","author":"Willis","year":"2022","journal-title":"NPJ Park. Dis."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Park, J.H., Kim, D.H., Kwon, D.Y., Choi, M., Kim, S., Jung, J.H., Han, K., and Park, Y.G. (2019). Trends in the incidence and prevalence of Parkinson\u2019s disease in Korea: A nationwide, population-based study. BMC Geriatr., 19.","DOI":"10.1186\/s12877-019-1332-7"},{"key":"ref_4","unstructured":"(2024, April 12). Per-Patient Health Insurance Treatment Costs for Parkinson\u2019s Disease Increase by 8.9% in Five Years. Available online: http:\/\/www.doctorstimes.com\/news\/articleView.html?idxno=218383."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1080\/14737175.2019.1649142","article-title":"Motor speech and non-motor language endophenotypes of Parkinson\u2019s disease","volume":"19","author":"Magee","year":"2019","journal-title":"Expert Rev. Neurother."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1038\/s41598-017-00047-5","article-title":"Automated analysis of connected speech reveals early biomarkers of Parkinson\u2019s disease in patients with rapid eye movement sleep behaviour disorder","volume":"7","author":"Rusz","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"649","DOI":"10.3233\/NRE-130887","article-title":"Acoustic characteristics of vowel sounds in patients with Parkinson disease","volume":"32","author":"Bang","year":"2013","journal-title":"NeuroRehabilitation"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1080\/136828200410654","article-title":"Voice characteristics in the progression of Parkinson\u2019s disease","volume":"35","author":"Holmes","year":"2000","journal-title":"Int. J. Lang. Commun. Disord."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1121\/10.0000581","article-title":"Assessment of speech impairment in patients with Parkinson\u2019s disease from acoustic quantifications of oral diadochokinetic sequences","volume":"147","author":"Karlsson","year":"2020","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1329","DOI":"10.1212\/WNL.0000000000002550","article-title":"Two different phenomena in basic motor speech performance in premanifest Huntington disease","volume":"86","author":"Skodda","year":"2016","journal-title":"Neurology"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1159\/000498851","article-title":"How well does diadochokinetic task performance predict articulatory imprecision? Differentiating individuals with Parkinson\u2019s disease from control subjects","volume":"71","author":"Karlsson","year":"2019","journal-title":"Folia Phoniatr. Logop."},{"key":"ref_12","first-page":"796205","article-title":"A preliminary report on disordered speech with deep brain stimulation in individuals with Parkinson\u2019s disease","volume":"2011","author":"Dromey","year":"2011","journal-title":"Park. Dis."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Godino-Llorente, J.I., Shattuck-Hufnagel, S., Choi, J.Y., Moro-Vel\u00e1zquez, L., and G\u00f3mez-Garc\u00eda, J.A. (2017). Towards the identification of Idiopathic Parkinson\u2019s Disease from the speech. New articulatory kinetic biomarkers. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0189583"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mei, J., Desrosiers, C., and Frasnelli, J. (2021). Machine learning for the diagnosis of Parkinson\u2019s disease: A review of literature. Front. Aging Neurosci., 13.","DOI":"10.3389\/fnagi.2021.633752"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"20615","DOI":"10.1038\/s41598-023-47568-w","article-title":"A machine learning method to process voice samples for identification of Parkinson\u2019s disease","volume":"13","author":"Iyer","year":"2023","journal-title":"Sci. Rep."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2079","DOI":"10.1038\/s41591-021-01577-2","article-title":"Rising to the challenge of bias in health care AI","volume":"27","author":"Cho","year":"2021","journal-title":"Nat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1038\/s41746-023-00858-z","article-title":"Bias in AI-based models for medical applications: Challenges and mitigation strategies","volume":"6","author":"Mittermaier","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1038\/s42256-022-00516-1","article-title":"Advances, challenges and opportunities in creating data for trustworthy AI","volume":"4","author":"Liang","year":"2022","journal-title":"Nat. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1146\/annurev-biodatasci-092820-114757","article-title":"Ethical machine learning in healthcare","volume":"4","author":"Chen","year":"2021","journal-title":"Annu. Rev. Biomed. Data Sci."},{"key":"ref_20","unstructured":"Arrieta, A.B., D\u00edaz-Rodr\u00edguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garc\u00eda, S., Gil-L\u00f3pez, S., Molina, D., and Benjamins, R. (2019). Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Gong, Y., Chung, Y.A., and Glass, J. (September, January 30). AST: Audio Spectrogram Transformer. Proceedings of the Interspeech 2021, ISCA, Brno, Czechia.","DOI":"10.21437\/Interspeech.2021-698"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3292","DOI":"10.1109\/TASLP.2021.3120633","article-title":"PSLA: Improving audio tagging with pretraining, sampling, labeling, and aggregation","volume":"29","author":"Gong","year":"2021","journal-title":"IEEE ACM Trans. Audio Speech Lang. Process."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., and Torralba, A. (July, January 26). Learning deep features for discriminative localization. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.319"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual explanations from deep networks via gradient-based localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Muhammad, M.B., and Yeasin, M. (2020, January 19\u201324). Eigen-CAM: Class Activation Map using Principal Components. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9206626"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wodzinski, M., Skalski, A., Hemmerling, D., Orozco-Arroyave, J.R., and N\u00f6th, E. (2019, January 23\u201327). Deep learning approach to Parkinson\u2019s disease detection using voice recordings and convolutional neural network dedicated to image classification. Proceedings of the 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC), Berlin, Germany.","DOI":"10.1109\/EMBC.2019.8856972"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ma, J., Zhang, Y., Li, Y., Zhou, L., Qin, L., Zeng, Y., Wang, P., and Lei, Y. (2021). Deep dual-side learning ensemble model for Parkinson speech recognition. Biomed. Signal Process. Control, 69.","DOI":"10.1016\/j.bspc.2021.102849"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"10239","DOI":"10.1109\/ACCESS.2021.3051432","article-title":"A deep learning based method for Parkinson\u2019s disease detection using dynamic features of speech","volume":"9","author":"Quan","year":"2021","journal-title":"IEEE Access"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1016\/j.bbe.2022.04.002","article-title":"End-to-end deep learning approach for Parkinson\u2019s disease detection from speech signals","volume":"42","author":"Quan","year":"2022","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Costantini, G., Cesarini, V., Leo, P.D., Amato, F., Suppa, A., Asci, F., Pisani, A., Calculli, A., and Saggio, G. (2023). Artificial Intelligence-Based Voice Assessment of Patients with Parkinson\u2019s Disease Off and On Treatment: Machine vs. Deep-Learning Comparison. Sensors, 23.","DOI":"10.3390\/s23042293"},{"key":"ref_31","first-page":"3204","article-title":"Parkinson\u2019s diagnosis hybrid system based on deep learning classification with imbalanced dataset","volume":"13","author":"Ouhmida","year":"2023","journal-title":"Int. J. Electr. Comput. Eng."},{"key":"ref_32","unstructured":"Tsanas, A. (2014). LSVT Voice Rehabilitation, UCI Machine Learning Repository."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1109\/JBHI.2013.2245674","article-title":"Collection and analysis of a Parkinson speech dataset with multiple types of sound recordings","volume":"17","author":"Sakar","year":"2013","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1591","DOI":"10.1002\/mds.26424","article-title":"MDS clinical diagnostic criteria for Parkinson\u2019s disease: MDS-PD Clinical Diagnostic Criteria","volume":"30","author":"Postuma","year":"2015","journal-title":"Mov. Disord."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1417","DOI":"10.1121\/10.0009411","article-title":"A model of speech recognition for hearing-impaired listeners based on deep learning","volume":"151","author":"Kollmeier","year":"2022","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_36","unstructured":"Touvron, H., Cord, M., Douze, M., Massa, F., Sablayrolles, A., and J\u00e9gou, H. (2021, January 18\u201324). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_38","unstructured":"Chaudhuri, K., and Salakhutdinov, R. (2019, January 10\u201315). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Park, D.S., Chan, W., Zhang, Y., Chiu, C.C., Zoph, B., Cubuk, E.D., and Le, Q.V. (2019). Specaugment: A simple data augmentation method for automatic speech recognition. arXiv.","DOI":"10.21437\/Interspeech.2019-2680"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (2017). mixup: Beyond empirical risk minimization. arXiv.","DOI":"10.1007\/978-1-4899-7687-1_79"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4625\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:18:17Z","timestamp":1760109497000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4625"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,17]]},"references-count":40,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["s24144625"],"URL":"https:\/\/doi.org\/10.3390\/s24144625","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,17]]}}}