{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T04:49:24Z","timestamp":1793335764955,"version":"4.5.0"},"reference-count":56,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,4,10]],"date-time":"2025-04-10T00:00:00Z","timestamp":1744243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"The National Centre for Research and Development, Poland","award":["LIDER\/6\/0049\/L-12\/20\/NCBIR\/2021"],"award-info":[{"award-number":["LIDER\/6\/0049\/L-12\/20\/NCBIR\/2021"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study explores an innovative approach to early Parkinson\u2019s disease (PD) detection by analyzing speech data collected using a mixed reality (MR) system. A total of 57 Polish participants, including PD patients and healthy controls, performed five speech tasks while using an MR head-mounted display (HMD). Speech data were recorded and analyzed to extract acoustic and linguistic features, which were then evaluated using machine learning models, including logistic regression, support vector machines (SVMs), random forests, AdaBoost, and XGBoost. The XGBoost model achieved the best performance, with an F1-score of 0.90 \u00b1 0.05 in the story-retelling task. Key features such as MFCCs (mel-frequency cepstral coefficients), spectral characteristics, RASTA-filtered auditory spectrum, and local shimmer were identified as significant in detecting PD-related speech alterations. Additionally, state-of-the-art deep learning models (wav2vec2, HuBERT, and WavLM) were fine-tuned for PD detection. HuBERT achieved the highest performance, with an F1-score of 0.94 \u00b1 0.04 in the diadochokinetic task, demonstrating the potential of deep learning to capture complex speech patterns linked to neurodegenerative diseases. This study highlights the effectiveness of combining MR technology for speech data collection with advanced machine learning (ML) and deep learning (DL) techniques, offering a non-invasive and high-precision approach to PD diagnosis. The findings hold promise for broader clinical applications, advancing the diagnostic landscape for neurodegenerative disorders.<\/jats:p>","DOI":"10.3390\/s25082405","type":"journal-article","created":{"date-parts":[[2025,4,10]],"date-time":"2025-04-10T07:41:51Z","timestamp":1744270911000},"page":"2405","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Analysis of Voice, Speech, and Language Biomarkers of Parkinson\u2019s Disease Collected in a Mixed Reality Setting"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-4299-9162","authenticated-orcid":false,"given":"Milosz","family":"Dudek","sequence":"first","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2193-7690","authenticated-orcid":false,"given":"Daria","family":"Hemmerling","sequence":"additional","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-7676-1991","authenticated-orcid":false,"given":"Marta","family":"Kaczmarska","sequence":"additional","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joanna","family":"Stepien","sequence":"additional","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2363-7912","authenticated-orcid":false,"given":"Mateusz","family":"Daniol","sequence":"additional","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8076-6246","authenticated-orcid":false,"given":"Marek","family":"Wodzinski","sequence":"additional","affiliation":[{"name":"Department of Measurement and Electronics, AGH University of Krakow, 30-059 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Magdalena","family":"Wojcik-Pedziwiatr","sequence":"additional","affiliation":[{"name":"Department of Neurology, Andrzej Frycz Modrzewski Krakow University, 30-705 Krakow, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,10]]},"reference":[{"key":"ref_1","first-page":"504","article-title":"Parkinson\u2019s Disease and Its Management: Part 1: Disease Entity, Risk Factors, Pathophysiology, Clinical Presentation, and Diagnosis","volume":"40","author":"DeMaagd","year":"2015","journal-title":"Pharm. Ther."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1044\/jshd.4301.47","article-title":"Frequency and cooccurrence of vocal tract dysfunctions in the speech of a large sample of Parkinson patients","volume":"43","author":"Logemann","year":"1978","journal-title":"J. Speech Hear. Disord."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1136\/jnnp.54.12.1093","article-title":"Articulatory deficits in parkinsonian dysarthria: An acoustic analysis","volume":"54","author":"Ackermann","year":"1991","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1093\/ageing\/afj053","article-title":"Life with communication changes in Parkinson\u2019s disease","volume":"35","author":"Miller","year":"2006","journal-title":"Age Ageing"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Biomarkers Definitions Working Group, Atkinson, A.J., Colburn, W.A., DeGruttola, V.G., DeMets, D.L., Downing, G.J., Hoth, D.F., Oates, J.A., Peck, C.C., and Schooley, R.T. (2001). Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework. Clin. Pharmacol. Ther., 69, 89\u201395.","DOI":"10.1067\/mcp.2001.113989"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/1535370217750088","article-title":"Biomarker definitions and their applications","volume":"243","author":"Califf","year":"2018","journal-title":"Exp. Biol. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1159\/000515346","article-title":"Voice for Health: The Use of Vocal Biomarkers from Research to Clinical Practice","volume":"5","author":"Fagherazzi","year":"2021","journal-title":"Digit. Biomark."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1353","DOI":"10.1016\/j.mayocp.2023.03.007","article-title":"Guess What We Can Hear\u2014Novel Voice Biomarkers for the Remote Detection of Disease","volume":"98","author":"Sara","year":"2023","journal-title":"Mayo Clin. Proc."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Godino-Llorente, J.I., Shattuck-Hufnagel, S., J.Y., C., Moro-Vel\u00e1zquez, L., and G\u00f3mez-Garc\u00eda, J.A. (2017). Towards the identification of Idiopathic Parkinson\u2019s Disease from the speech. New articulatory kinetic biomarkers. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0189583"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Pascal, H., Nico, S., Florian, E., Bj\u00f6rn, W.S., and Bert, A. (2022). Voice Analysis for Neurological Disorder Recognition\u2014A Systematic Review and Perspective on Emerging Trends. Front. Digit. Health, 4.","DOI":"10.3389\/fdgth.2022.842301"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"884","DOI":"10.1109\/TBME.2009.2036000","article-title":"Accurate Telemonitoring of Parkinson\u2019s Disease Progression by Noninvasive Speech Tests","volume":"57","author":"Tsanas","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"337","DOI":"10.2217\/nmt-2018-0021","article-title":"Speech disorders in Parkinson\u2019s disease: Pathophysiology, medical management and surgical approaches","volume":"8","author":"Dashtipour","year":"2018","journal-title":"Neurodegener. Dis. Manag."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1109\/TASL.2010.2072499","article-title":"Articulatory Knowledge in the Recognition of Dysarthric Speech","volume":"19","author":"Rudic","year":"2011","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"134","DOI":"10.3109\/02699206.2012.751624","article-title":"Speech and pause characteristics in multiple sclerosis: A preliminary study of speakers with high and low neuropsychological test performance","volume":"27","author":"Feenaughty","year":"2013","journal-title":"Clin. Linguist. Phon."},{"key":"ref_15","unstructured":"Antonio, S., Giovanni, C., Francesco, A., Pietro, D.L., Sami, A.W.M., Giulia, D.L., Simona, S., Antonio, P., and Giovanni, S. (2022). Voice in Parkinson\u2019s Disease: A Machine Learning Study. Front. Neurol., 13."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"142","DOI":"10.32996\/jcsts.2023.5.4.14","article-title":"Parkinson\u2019s Disease Detection through Vocal Biomarkers and Advanced Machine Learning Algorithms","volume":"5","author":"Sayed","year":"2023","journal-title":"J. Comput. Sci. Technol. Stud."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Giovanni, C., Cesarini, V., Leo, P.D., Amato, F., Suppa, A., Asci, F., Pisani, A., Calculli, A., and Saggio, G. (2023). Artificial Intelligence-Based Voice Assessment of Patients with Parkinson\u2019s Disease Off and On Treatment: Machine vs. Deep-Learning Comparison. Sensors, 23.","DOI":"10.3390\/s23042293"},{"key":"ref_18","unstructured":"Orozco-Arroyave, J., Arias-Londono, J., Vargas-Bonilla, J., Gonzalez-Rativa, M.C., and Noeth, E. (2014, January 26\u201331). New Spanish speech corpus database for the analysis of people suffering from Parkinson\u2019s disease. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC\u201914), Reykjavik, Iceland."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"20","DOI":"10.24018\/ejece.2023.7.2.488","article-title":"Classification of Parkinson\u2019s Disease Using Speech Signal With Machine Learning and Deep Learning Approaches","volume":"7","author":"Senjuti","year":"2023","journal-title":"Eur. J. Electr. Eng. Comput. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cesare, M.D.D., Perpetuini, D., Cardone, D., and Merla, A. (2024). Machine Learning-Assisted Speech Analysis for Early Detection of Parkinson\u2019s Disease: A Study on Speaker Diarization and Classification Techniques. Sensors, 24.","DOI":"10.3390\/s24051499"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Salman, A.J.M.K., and Stamatescu, G. (2023, January 7\u20139). Daily Monitoring of Speech Impairment for Early Parkinson\u2019s Disease Detection. Proceedings of the 2023 IEEE 12th International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), Dortmund, Germany.","DOI":"10.1109\/IDAACS58523.2023.10348855"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Naanoue, J., Ayoub, R., Sayyadi, F.E., Hamawy, L., Hage-Diab, A., and Sbeity, F. (2023, January 12\u201313). Parkinson\u2019s disease detection from speech analysis using deep learning. Proceedings of the 2023 Seventh International Conference on Advances in Biomedical Engineering (ICABME), Beirut, Lebanon.","DOI":"10.1109\/ICABME59496.2023.10293142"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kumar, S.A., Sasikala, S., Arthiya, K.B., Sathika, J., and Karishma, V. (2023, January 16\u201317). Parkinson\u2019s Speech Detection Using YAMNet. Proceedings of the 2023 2nd International Conference on Advancements in Electrical, Electronics, Communication, Computing and Automation (ICAECA), Coimbatore, India.","DOI":"10.1109\/ICAECA56562.2023.10200704"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"La Quatra, M., Turco, M.F., Svendsen, T., Salvi, G., Orozco-Arroyave, J.R., and Siniscalchi, S.M. (2024, January 1\u20135). Exploiting Foundation Models and Speech Enhancement for Parkinson\u2019s Disease Detection from Speech in Real-World Operative Conditions. Proceedings of the Interspeech 2024, Kos, Greece.","DOI":"10.21437\/Interspeech.2024-522"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Klempir, O., and Krupicka, R. (2024). Analyzing Wav2Vec 1.0 Embeddings for Cross-Database Parkinson\u2019s Disease Detection and Speech Features Extraction. Sensors, 24.","DOI":"10.3390\/s24175520"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Amirov, A., Castegnaro, A., Rudzka, K., Burgess, N., Chan, D., and Pan, X. (2024, January 7\u20139). Exploring Augmented Reality as a Tool for Early Detection of Alzheimer\u2019s Disease. Proceedings of the 2024 IEEE International Conference on Serious Games and Applications for Health (SeGAH), Funchal, Portugal.","DOI":"10.1109\/SeGAH61285.2024.10639570"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1111\/ene.14108","article-title":"Parkinson\u2019s Disease: An Updated Overview","volume":"27","author":"Balestrino","year":"2020","journal-title":"Eur. J. Neurol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1109\/THMS.2020.3016082","article-title":"The HoloLens Augmented Reality System Provides Valid Measures of Gait Performance in Healthy Adults","volume":"50","author":"Koop","year":"2020","journal-title":"IEEE Trans.-Hum.-Mach. Syst."},{"key":"ref_29","unstructured":"Thomos, M., Charisis, V., Hadjidimitriou, S., Iakovakis, D., Bostantzopoulou, S., and Hadjileontiadis, L. (2020). Holoprognosis: Augmented Reality Exercises for Early-Stage Parkinson\u2019s Patients. Universal Access in Human\u2013Computer Interaction, Springer."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Favaro, A., Moro-Vel\u00e1zquez, L., Butala, A., Motley, C., Cao, T., Stevens, R.D., Villalba, J., and Dehak, N. (2023). Multilingual evaluation of interpretable biomarkers to represent language and speech patterns in Parkinson\u2019s disease. Front. Neurol., 14.","DOI":"10.3389\/fneur.2023.1142642"},{"key":"ref_31","unstructured":"Microsoft (2024, December 15). Microsoft HoloLens 2. Available online: https:\/\/www.microsoft.com\/pl-pl\/hololens."},{"key":"ref_32","unstructured":"Kipman, A., and Pollefeys, M. (2025, April 05). Global Lecture 2019\u2014HoloLens 2: Unpacked, 2019. Available online: https:\/\/mspoweruser.com\/microsofts-alex-kipman-explains-the-groundbreaking-technology-behind-the-hololens-2\/."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2129","DOI":"10.1002\/mds.22340","article-title":"Movement Disorder Society-Sponsored Revision of the Unified Parkinson\u2019s Disease Rating Scale (MDS-UPDRS)","volume":"23","author":"Goetz","year":"2008","journal-title":"Mov. Disord."},{"key":"ref_34","unstructured":"The Movement Disorder Society (2024, December 20). MDS-UPDRS. Available online: https:\/\/www.movementdisorders.org\/MDS-Files1\/PDFs\/Rating-Scales\/MDS-UPDRS_English_FINAL.pdf."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1044\/1058-0360.0603.66","article-title":"The Voice Handicap Index (VHI): Development and Validation","volume":"6","author":"Jacobson","year":"1997","journal-title":"Am. J.-Speech-Lang. Pathol."},{"key":"ref_36","first-page":"1","article-title":"Zastosowanie wskaznika niepelnosprawnosci glosowej (Voice Handicap Index\u2014VHI) w ocenie efektywnosci terapii g\u0142osu u nauczycieli","volume":"58","author":"Kuzniarska","year":"2007","journal-title":"Medycyna Pracy"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1016\/j.jvoice.2007.09.001","article-title":"How do individuals with voice disorders? Introducing the Voice Disability Coping Questionnaire","volume":"23","author":"Epstein","year":"2009","journal-title":"J. Voice"},{"key":"ref_38","first-page":"101","article-title":"Validation of the Polish Version of Voice Disability Coping Questionnaire (PL-VDCQ)","volume":"18","author":"Morawska","year":"2019","journal-title":"Otolaryngol.-Prz. Klin."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"695","DOI":"10.1111\/j.1532-5415.2005.53221.x","article-title":"The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment","volume":"53","author":"Nasreddine","year":"2005","journal-title":"J. Am. Geriatr. Soc."},{"key":"ref_40","unstructured":"McEwen, B. (2025, April 05). Administering the MoCA: What Does It Mean and Why Is It Important? Presentation, Riverside University Medical Center, 2023. Available online: https:\/\/rivcodpss.org\/sites\/g\/files\/aldnop276\/files\/2023-04\/MoCA%20Presentation.FULL%20PAGE.pdf."},{"key":"ref_41","unstructured":"Sainburg, T. (2024, December 20). Tinsamb\/Noisereduce: v1.0.0. 2019. Available online: https:\/\/zenodo.org\/records\/3243139."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Eyben, F., Wollmer, M., and Schuller, B. (2010, January 25\u201329). openSMILE\u2014The Munich Versatile and Fast Open-Source Audio Feature Extractor. Proceedings of the 18th ACM International Conference on Multimedia, Florence, Italy.","DOI":"10.1145\/1873951.1874246"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.wocn.2018.07.001","article-title":"Introducing Parselmouth: A Python interface to Praat","volume":"71","author":"Jadoul","year":"2018","journal-title":"J. Phon."},{"key":"ref_44","unstructured":"Boersma, P., and Weenink, D. (2024, December 20). Praat: Doing Phonetics by Computer [Computer Program], 2021. Version 6.1.38. Available online: http:\/\/www.praat.org\/."},{"key":"ref_45","unstructured":"Tadeusiewicz, R. (1988). Sygnal Mowy, Wydawnictwo Komunikacji i Lacznosci."},{"key":"ref_46","unstructured":"Cleveland Clinic Tips for Improving Communication in People with Parkinson\u2019s Disease, 2020."},{"key":"ref_47","unstructured":"Wiseman, J. (2024, December 20). Python Interface to the WebRTC Voice Activity Detector, 2021. Available online: https:\/\/github.com\/wiseman\/py-webrtcvad."},{"key":"ref_48","first-page":"51","article-title":"Prosodic characteristics of Parkinsonian speech: The effect of levodopa-based medication","volume":"13","author":"Goberman","year":"2005","journal-title":"J. Med.-Speech-Lang. Pathol."},{"key":"ref_49","unstructured":"Microsoft (2024, December 20). Microsoft\u2014Azure Speech SDK. Available online: https:\/\/learn.microsoft.com\/en-us\/azure\/ai-services\/speech-service\/speech-sdk."},{"key":"ref_50","unstructured":"spaCy (2024, December 20). spaCy\u2014Industrial-Strength Natural Language Processing in Python. Available online: https:\/\/spacy.io\/."},{"key":"ref_51","unstructured":"Citizenlab (2024, December 20). Citizenlab\/Twitter-Xlm-Roberta-Base-Sentiment-Finetuned. Available online: https:\/\/huggingface.co\/citizenlab\/twitter-xlm-roberta-base-sentiment-finetuned."},{"key":"ref_52","unstructured":"Facebook (2024, December 20). Facebook\/Hubert-Base-ls960v. Available online: https:\/\/huggingface.co\/facebook\/hubert-base-ls960."},{"key":"ref_53","unstructured":"Microsoft (2024, December 20). Microsoft\/Wavlm-Base. Available online: https:\/\/huggingface.co\/microsoft\/wavlm-base."},{"key":"ref_54","unstructured":"Meta (2024, December 20). Wav2vec 2.0: Learning the Structure of Speech from Raw Audio. Available online: https:\/\/huggingface.co\/docs\/transformers\/en\/model_doc\/wav2vec2."},{"key":"ref_55","unstructured":"Putzer, M., and Barry, W.J. (2024, December 20). Saarbruecken Voice Database. Available online: https:\/\/uni-saarland.academia.edu\/ManfredP%C3%BCtzer."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"597","DOI":"10.22237\/jmasm\/1257035100","article-title":"New Effect Size Rules of Thumb","volume":"8","author":"Sawilowsky","year":"2009","journal-title":"J. Mod. Appl. Stat. Methods"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/8\/2405\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:12:06Z","timestamp":1760029926000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/25\/8\/2405"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,10]]},"references-count":56,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["s25082405"],"URL":"https:\/\/doi.org\/10.3390\/s25082405","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,10]]}}}