{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T18:32:45Z","timestamp":1790620365016,"version":"4.1.0"},"reference-count":45,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T00:00:00Z","timestamp":1763078400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Millions worldwide are affected by Parkinson\u2019s disease, with the World Health Organization highlighting its growing prevalence. Early neuromotor speech impairments make voice analysis a promising tool for detecting Parkinson\u2019s, aided by advances in deep speech embeddings. However, existing approaches often rely on either handcrafted acoustic features or opaque deep representations, limiting diagnostic performance and interoperability. To address this, we propose a multi-modal decentralized hybrid learning framework that combines structured voice biomarkers from the UCI Parkinson\u2019s dataset (195 sustained-phonation samples from 31 subjects) with contrastive speech embeddings derived from the DAIC-WOZ corpus (189 interview recordings originally collected for depression detection) using Wav2Vec 2.0. This system employs an early fusion strategy followed by a dense neural classifier optimized for binary classification. By integrating both clinically interpretable and semantically rich features, the model captures complementary phonatory and affective patterns relevant to early-stage Parkinson\u2019s detection. Extensive evaluation demonstrates that the proposed method achieves an accuracy of 96.2% and an AUC of 97.1%, outperforming unimodal and baseline fusion models. SHAP-based analysis confirms that a subset of features have disproportionately high discriminative value, enhancing interpretability. Overall, the proposed framework establishes a promising pathway toward data-driven, non-invasive screening for neurodegenerative conditions through voice analysis.<\/jats:p>","DOI":"10.3390\/s25226959","type":"journal-article","created":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T14:37:52Z","timestamp":1763131072000},"page":"6959","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multi-Modal Decentralized Hybrid Learning for Early Parkinson\u2019s Detection Using Voice Biomarkers and Contrastive Speech Embeddings"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-4529-1301","authenticated-orcid":false,"given":"Khaled M.","family":"Alhawiti","sequence":"first","affiliation":[{"name":"Faculty of Computers and Information Technology, University of Tabuk, Tabuk 47512, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.bandl.2018.08.002","article-title":"Communication impairment in Parkinson\u2019s disease: Impact of motor and cognitive symptoms on speech and language","volume":"185","author":"Smith","year":"2018","journal-title":"Brain Lang."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1007\/s00702-024-02838-3","article-title":"Diagnostic and therapeutic challenges in PD-associated non-motor symptoms: The roles of neurologists and consultant physicians","volume":"131","author":"Ip","year":"2024","journal-title":"J. Neural Transm."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5571","DOI":"10.1109\/TMC.2023.3314837","article-title":"Accuth Accelerometer-Based Anti-Spoofing voice authentication on Wrist-Worn wearables","volume":"23","author":"Han","year":"2023","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1007\/s10462-024-11009-5","article-title":"Digital phenotypes and digital biomarkers for health and diseases: A systematic review of machine learning approaches utilizing passive non-invasive signals collected via wearable devices and smartphones","volume":"58","author":"Sameh","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Costantini, G., Cesarini, V., Di Leo, P., Amato, F., Suppa, A., Asci, F., Pisani, A., Calculli, A., and Saggio, G. (2023). Artificial intelligence-based voice assessment of patients with Parkinson\u2019s disease off and on treatment: Machine vs. deep-learning comparison. Sensors, 23.","DOI":"10.3390\/s23042293"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1634","DOI":"10.1109\/TAFFC.2021.3114365","article-title":"Survey of deep representation learning for speech emotion recognition","volume":"14","author":"Latif","year":"2021","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shou, Y., Meng, T., Ai, W., Yin, N., and Li, K. (2023). A comprehensive survey on multi-modal conversational emotion recognition with deep learning. arXiv.","DOI":"10.2139\/ssrn.5017731"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1109\/TAFFC.2025.3551330","article-title":"Identifying stable EEG patterns in manipulation task for negative emotion recognition","volume":"16","author":"Pei","year":"2025","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., and Zong, Y. (2023). A survey of deep learning-based multimodal emotion recognition: Speech, text, and face. Entropy, 25.","DOI":"10.3390\/e25101440"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/TAFFC.2024.3521327","article-title":"Speech-based depression assessment: A comprehensive survey","volume":"16","author":"Leal","year":"2024","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., and Pei, Y. (2024). A comprehensive review on synergy of multi-modal data and ai technologies in medical diagnosis. Bioengineering, 11.","DOI":"10.3390\/bioengineering11030219"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4682","DOI":"10.1109\/TVCG.2024.3409568","article-title":"TalkingStyle: Personalized speech-driven 3D facial animation with style preservation","volume":"31","author":"Song","year":"2024","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Suppa, A., Costantini, G., Asci, F., Di Leo, P., Al-Wardat, M.S., Di Lazzaro, G., Scalise, S., Pisani, A., and Saggio, G. (2022). Voice in Parkinson\u2019s Disease: A Machine Learning Study. Front. Neurol., 13.","DOI":"10.3389\/fneur.2022.831428"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"7397","DOI":"10.1109\/TVCG.2024.3456213","article-title":"Expressive 3d facial animation generation based on local-to-global latent diffusion","volume":"30","author":"Song","year":"2024","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Srinivasan, S., Ramadass, P., Mathivanan, S.K., Panneer Selvam, K., Shivahare, B.D., and Shah, M.A. (2024). Detection of Parkinson disease using multiclass machine learning approach. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-64004-9"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"116267","DOI":"10.1016\/j.chaos.2025.116267","article-title":"A study of bidirectional control of Parkinson\u2019s beta oscillations by basal ganglia","volume":"195","author":"Hu","year":"2025","journal-title":"Chaos Solitons Fractals"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1179","DOI":"10.1109\/JSTSP.2022.3207050","article-title":"Self-supervised speech representation learning: A review","volume":"16","author":"Mohamed","year":"2022","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"124396","DOI":"10.1109\/ACCESS.2022.3225198","article-title":"Evaluating self-supervised speech representations for speech emotion recognition","volume":"10","author":"Atmaja","year":"2022","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"107483","DOI":"10.1016\/j.radmeas.2025.107483","article-title":"Neutron-Gamma Discrimination Method Based on Voiceprint Identification","volume":"187","author":"Zhao","year":"2025","journal-title":"Radiat. Meas."},{"key":"ref_20","first-page":"183","article-title":"Deep learning for multi-modal medical imaging fusion: Enhancing diagnostic accuracy in complex disease detection","volume":"6","author":"Kumar","year":"2022","journal-title":"Int. J. Eng. Technol. Res. Manag."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kumar, S., Rani, S., Sharma, S., and Min, H. (2024). Multimodality fusion aspects of medical diagnosis: A comprehensive review. Bioengineering, 11.","DOI":"10.3390\/bioengineering11121233"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Harry, M., Vijula, V., and Princess, J.P. (2024, January 28\u201329). Parkinson\u2019s Disease Prediction Using Machine Learning Techniques. Proceedings of the 2024 IEEE International Conference on Information Technology, Electronics and Intelligent Communication Systems (ICITEICS), Bangalore, India.","DOI":"10.1109\/ICITEICS61368.2024.10625564"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"9250066","DOI":"10.26599\/FSHW.2024.9250066","article-title":"Effect of purine diet on prognosis of deep brain stimulation for Parkinson\u2019s disease","volume":"14","author":"Chang","year":"2025","journal-title":"Food Sci. Hum. Wellness"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"114798","DOI":"10.1109\/ACCESS.2025.3585099","article-title":"Multi-Modal Feature Set-based Detection of Freezing of Gait in Parkinson\u2019s Disease Patients Using SVM","volume":"13","author":"Murtaza","year":"2025","journal-title":"IEEE Access"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3716818","article-title":"Facial Expression Analysis in Parkinsons\u2019s Disease Using Machine Learning: A Review","volume":"57","author":"Oliveira","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_26","first-page":"7527","article-title":"Protective effect of gastrodin on dopaminergic neurons by miR-24 targeting of the DJ-1\/Nrf2 pathway in Parkinson\u2019s disease","volume":"13","author":"Li","year":"2020","journal-title":"Int. J. Clin. Exp. Med."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shin, D., Cho, W.I., Park, C.H.K., Rhee, S.J., Kim, M.J., Lee, H., Kim, N.S., and Ahn, Y.M. (2021). Detection of minor and major depression through voice as a biomarker using machine learning. J. Clin. Med., 10.","DOI":"10.3390\/jcm10143046"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Naeem, I., Ditta, A., Mazhar, T., Anwar, M., Saeed, M.M., and Hamam, H. (2025). Voice biomarkers as prognostic indicators for Parkinson\u2019s disease using machine learning techniques. Sci. Rep., 15.","DOI":"10.1038\/s41598-025-96950-3"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Guti\u00e9rrez-Seraf\u00edn, B., Andreu-Perez, J., P\u00e9rez-Espinosa, H., Paulmann, S., and Ding, W. (2024). Toward assessment of human voice biomarkers of brain lesions through explainable deep learning. Biomed. Signal Process. Control, 87.","DOI":"10.1016\/j.bspc.2023.105457"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yao, D., Koivu, A., and Simonyan, K. (World J. Otorhinolaryngol. Head Neck Surg., 2025). Applications of Artificial Intelligence in Neurological Voice Disorders, World J. Otorhinolaryngol. Head Neck Surg., early view.","DOI":"10.1002\/wjo2.70017"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Malekroodi, H.S., Madusanka, N., Lee, B.i., and Yi, M. (2024). Leveraging deep learning for fine-grained categorization of Parkinson\u2019s disease progression levels through analysis of vocal acoustic patterns. Bioengineering, 11.","DOI":"10.3390\/bioengineering11030295"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"56355","DOI":"10.1109\/ACCESS.2025.3555176","article-title":"A Review of Machine Learning Approaches for Non-Invasive Cognitive Impairment Detection","volume":"13","author":"Alsuhaibani","year":"2025","journal-title":"IEEE Access"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nagrani, A., Chung, J.S., Albanie, S., and Zisserman, A. (2020, January 4\u20138). Disentangled speech embeddings using cross-modal self-supervision. Proceedings of the ICASSP 2020\u20132020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054057"},{"key":"ref_34","first-page":"12449","article-title":"wav2vec 2.0: A framework for self-supervised learning of speech representations","volume":"33","author":"Baevski","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gimeno-G\u00f3mez, D., Botelho, C., Pompili, A., Abad, A., and Mart\u00ednez-Hinarejos, C.D. (2025). Unveiling Interpretability in Self-Supervised Speech Representations for Parkinson\u2019s Diagnosis. IEEE J. Sel. Top. Signal Process., 1\u201314.","DOI":"10.1109\/JSTSP.2025.3539845"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lee, J., Sim, Y., Kim, J., and Suh, Y.J. (2025). EmoSDS: Unified Emotionally Adaptive Spoken Dialogue System Using Self-Supervised Speech Representations. Future Internet, 17.","DOI":"10.3390\/fi17040143"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/TAFFC.2024.3418415","article-title":"Image encoding and fusion of multi-modal data enhance depression diagnosis in Parkinson\u2019s disease patients","volume":"16","author":"Li","year":"2024","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"109019","DOI":"10.1016\/j.jneumeth.2020.109019","article-title":"Classification of Parkinson\u2019s disease based on multi-modal features and stacking ensemble learning","volume":"350","author":"Yang","year":"2021","journal-title":"J. Neurosci. Methods"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Dai, Y., Song, Y., Liu, W., Bai, W., Gao, Y., Dong, X., and Lv, W. (2021). Multi-focus image fusion based on convolution neural network for Parkinson\u2019s disease image classification. Diagnostics, 11.","DOI":"10.3390\/diagnostics11122379"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"81252","DOI":"10.1109\/ACCESS.2025.3563177","article-title":"HAMF: A novel Hierarchical Attention based Multi-modal Fusion model for Parkinson\u2019s Disease Classification and Severity Prediction","volume":"13","author":"Palakayala","year":"2025","journal-title":"IEEE Access"},{"key":"ref_41","unstructured":"Little, M.A., McSharry, P.E., Hunter, E.J., and Ramig, L.O. (2025, June 20). Parkinsons [Dataset]. UCI Machine Learning Repository. Available online: https:\/\/archive.ics.uci.edu\/ml\/datasets\/parkinsons."},{"key":"ref_42","unstructured":"Sakar, C.O., Serbes, G., Gunduz, A., Nizam, H., Sakar, B., Tutuncu, M., Aydin, T., Isenkul, M.E., and Apaydin, H. (2025, June 16). Parkinson\u2019s Disease Classification Dataset. UCI Machine Learning Repository. Available online: https:\/\/archive.ics.uci.edu\/dataset\/470\/parkinson+s+disease+classification."},{"key":"ref_43","unstructured":"Gratch, J., Artstein, R., Lucas, G.M., Stratou, G., Scherer, S., Nazarian, A., Wood, R., Boberg, J., DeVault, D., and Marsella, S. (2014, January 26\u201331). The Distress Analysis Interview Corpus of human and computer interviews. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC), Reykjavik, Iceland."},{"key":"ref_44","unstructured":"DeVault, D., Artstein, R., Benn, G., Dey, T., Fast, E., Gainer, A., Georgila, K., Gratch, J., Hartholt, A., and Lhommet, M. (2014, January 5\u20139). SimSensei: A Virtual Human Interviewer for Healthcare Decision Support. Proceedings of the 13th International Conference on Autonomous Agents and Multi-Agent Systems, Paris, France."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Yafooz, W.M.S. (2024). Enhancing Arabic dialect detection on social media: A hybrid model with an attention mechanism. Information, 15.","DOI":"10.3390\/info15060316"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/22\/6959\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,14]],"date-time":"2025-11-14T14:42:13Z","timestamp":1763131333000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/22\/6959"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,14]]},"references-count":45,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["s25226959"],"URL":"https:\/\/doi.org\/10.3390\/s25226959","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,14]]}}}