{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T02:21:22Z","timestamp":1794882082219,"version":"4.9.1"},"reference-count":64,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T00:00:00Z","timestamp":1783987200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T00:00:00Z","timestamp":1794873600000},"content-version":"vor","delay-in-days":126,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"CUHK Vice-Chancellor Early Career Professorship Scheme","award":["ref. no. 441"],"award-info":[{"award-number":["ref. no. 441"]}]},{"name":"Research Grants Council Collaborative Research Fund","award":["RGC-CRF ref. no. C4044-21GF"],"award-info":[{"award-number":["RGC-CRF ref. no. C4044-21GF"]}]},{"DOI":"10.13039\/501100005847","name":"Health and Medical Research Fund","doi-asserted-by":"publisher","award":["HMRF ref. no. 18190931"],"award-info":[{"award-number":["HMRF ref. no. 18190931"]}],"id":[{"id":"10.13039\/501100005847","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    REM sleep behavior disorder (RBD) is a robust prodromal marker of \u03b1-synucleinopathies: idiopathic RBD carries a 10\u201315-year phenoconversion risk of 80\u201390% to Parkinson\u2019s disease (PD) and related disorders. In major depressive disorder (MDD), comorbid RBD marks a subgroup at elevated prodromal PD risk, yet is frequently missed in psychiatric practice. Here, we developed a multimodal AI framework to detect comorbid RBD in MDD. From a clinical cohort of 329 patients, we obtained 261 video clips in 31 patients during reading and spontaneous-speech tasks, including 19 patients with MDD-RBD and 12 demographically and medication-matched MDD-only controls that, to our knowledge, formed the largest cohort of its kind worldwide. We used a dual-stream multimodal model that learned facial dynamics from video and vocal features from speech, and then combined both signals to predict comorbid RBD. In 5-fold cross-validation, our best model achieved 80.5% accuracy and 0.848 F1-score. Explainability analysis highlighted lower-face tension and variability, together with brow lowering, as candidate biomarkers requiring further validation. Predicted risk correlated with RBDQ score (\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u20090.53,\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.005) and weakly with UPDRS motor score (\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u20090.34,\n                    <jats:italic>p<\/jats:italic>\n                    \u2009=\u20090.067). Out-of-distribution evaluation showed broadly similar patterns, supporting the promise of multimodal AI for predicting RBD in MDD and identifying interpretable potential digital markers of prodromal synucleinopathy.\n                  <\/jats:p>","DOI":"10.1038\/s41746-026-03001-w","type":"journal-article","created":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T14:43:41Z","timestamp":1784040221000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multimodal AI for predicting comorbid REM sleep behavior disorder in major depressive disorder"],"prefix":"10.1038","volume":"9","author":[{"given":"Lizhou","family":"Fan","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"},{"role":"corresponding-author","vocabulary":"crossref"}]},{"given":"Xiang","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinze","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianzhang","family":"Ni","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhixuan","family":"He","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuhua","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huizi","family":"Yu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhiying","family":"Liang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shi","family":"Tang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Siyi","family":"Gong","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ningning","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinxin","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xin","family":"Ma","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaping","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jihui","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Wang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"},{"role":"corresponding-author","vocabulary":"crossref"}]},{"given":"Joey W. Y.","family":"Chan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ngan Yin","family":"Chan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tim M. H.","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bei","family":"Huang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yun Kwok","family":"Wing","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"},{"role":"corresponding-author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,7,14]]},"reference":[{"key":"3001_CR1","doi-asserted-by":"crossref","unstructured":"Barber, T. R. et al. Prodromal parkinsonism and neurodegenerative risk stratification in REM sleep behavior disorder. Sleep 40, zsx071 (2017).","DOI":"10.1093\/sleep\/zsx071"},{"key":"3001_CR2","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1016\/S1474-4422(21)00176-9","volume":"20","author":"MG Miglis","year":"2021","unstructured":"Miglis, M. G. et al. Biomarkers of conversion to \u03b1-synucleinopathy in isolated rapid-eye-movement sleep behaviour disorder. Lancet Neurol. 20, 671\u2013684 (2021).","journal-title":"Lancet Neurol."},{"key":"3001_CR3","doi-asserted-by":"publisher","first-page":"627481","DOI":"10.3389\/fbioe.2021.627481","volume":"9","author":"L Ma","year":"2021","unstructured":"Ma, L. et al. Detection of motor dysfunction with wearable sensors in patients with idiopathic rapid eye movement disorder. Front. Bioeng. Biotechnol. 9, 627481 (2021).","journal-title":"Front. Bioeng. Biotechnol."},{"key":"3001_CR4","doi-asserted-by":"publisher","first-page":"1711","DOI":"10.1002\/mds.26665","volume":"31","author":"L Alibiglou","year":"2016","unstructured":"Alibiglou, L., Videnovic, A., Planetta, P. J., Vaillancourt, D. E. & MacKinnon, C. D. Subliminal gait initiation deficits in rapid eye movement sleep behavior disorder: a harbinger of freezing of gait? Mov. Disord. 31, 1711\u20131719 (2016).","journal-title":"Mov. Disord."},{"key":"3001_CR5","doi-asserted-by":"publisher","first-page":"1464","DOI":"10.1002\/mds.27802","volume":"34","author":"S Heinzel","year":"2019","unstructured":"Heinzel, S. et al. Update of the MDS research criteria for prodromal Parkinson\u2019s disease. Mov. Disord. 34, 1464\u20131470 (2019).","journal-title":"Mov. Disord."},{"key":"3001_CR6","doi-asserted-by":"publisher","first-page":"1600","DOI":"10.1002\/mds.26431","volume":"30","author":"D Berg","year":"2015","unstructured":"Berg, D. et al. MDS research criteria for prodromal Parkinson\u2019s disease. Mov. Disord. 30, 1600\u20131611 (2015).","journal-title":"Mov. Disord."},{"key":"3001_CR7","doi-asserted-by":"publisher","first-page":"710","DOI":"10.1136\/jnnp-2015-311680","volume":"87","author":"KL Adams-Carr","year":"2016","unstructured":"Adams-Carr, K. L. et al. Constipation preceding Parkinson\u2019s disease: a systematic review and meta-analysis. J. Neurol. Neurosurg. Psychiatry 87, 710\u2013716 (2016).","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"3001_CR8","doi-asserted-by":"publisher","first-page":"3753786","DOI":"10.1155\/2019\/3753786","volume":"2019","author":"X Sui","year":"2019","unstructured":"Sui, X. et al. Hyposmia as a predictive marker of Parkinson\u2019s disease: a systematic review and meta-analysis. Biomed. Res. Int. 2019, 3753786 (2019).","journal-title":"Biomed. Res. Int."},{"key":"3001_CR9","doi-asserted-by":"publisher","first-page":"2013","DOI":"10.1093\/brain\/awz113","volume":"142","author":"TS Lysen","year":"2019","unstructured":"Lysen, T. S., Darweesh, S. K. L., Ikram, M. K., Luik, A. I. & Ikram, M. A. Sleep and risk of Parkinsonism and Parkinson\u2019s disease: a population-based study. Brain 142, 2013\u20132022 (2019).","journal-title":"Brain"},{"key":"3001_CR10","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.sleep.2007.11.019","volume":"10","author":"PT Teman","year":"2009","unstructured":"Teman, P. T., Tippmann-Peikert, M., Silber, M. H., Slocumb, N. L. & Auger, R. R. Idiopathic rapid-eye-movement sleep disorder: associations with antidepressants, psychiatric diagnoses, and other factors, in relation to age of onset. Sleep. Med. 10, 60\u201365 (2009).","journal-title":"Sleep. Med."},{"key":"3001_CR11","doi-asserted-by":"publisher","first-page":"1374","DOI":"10.4088\/JCP.v69n0904","volume":"69","author":"SP Lam","year":"2008","unstructured":"Lam, S. P., Fong, S. Y. Y., Ho, C. K. W., Yu, M. W. M. & Wing, Y. K. Parasomnia among psychiatric outpatients: a clinical, epidemiologic, cross-sectional study. J. Clin. Psychiatry 69, 1374\u20131382 (2008).","journal-title":"J. Clin. Psychiatry"},{"key":"3001_CR12","doi-asserted-by":"publisher","first-page":"1010","DOI":"10.1136\/jnnp-2021-327460","volume":"93","author":"J Wang","year":"2022","unstructured":"Wang, J. et al. Prevalence and correlates of REM sleep behaviour disorder in patients with major depressive disorder: a two-phase study. J. Neurol. Neurosurg. Psychiatry 93, 1010\u20131017 (2022).","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"3001_CR13","doi-asserted-by":"publisher","first-page":"893","DOI":"10.1136\/jnnp-2022-330922","volume":"94","author":"J Wang","year":"2023","unstructured":"Wang, J. et al. Familial \u03b1-synucleinopathy spectrum features in patients with psychiatric REM sleep behaviour disorder. J. Neurol. Neurosurg. Psychiatry 94, 893\u2013903 (2023).","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"3001_CR14","doi-asserted-by":"publisher","first-page":"1579","DOI":"10.5665\/sleep.3102","volume":"36","author":"RB Postuma","year":"2013","unstructured":"Postuma, R. B. et al. Antidepressants and REM sleep behavior disorder: isolated side effect or neurodegenerative signal? Sleep 36, 1579\u20131585 (2013).","journal-title":"Sleep"},{"key":"3001_CR15","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.sleep.2024.11.032","volume":"126","author":"Y Yang","year":"2025","unstructured":"Yang, Y. et al. Sleep related injury and its correlates in isolated rapid eye movement sleep behavior disorder. Sleep. Med. 126, 9\u201318 (2025).","journal-title":"Sleep. Med."},{"key":"3001_CR16","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1038\/s41531-025-01032-w","volume":"11","author":"K Colman","year":"2025","unstructured":"Colman, K. et al. Novel technologies for REM sleep behavior disorder detection for home screening in Parkinson\u2019s disease and related alpha-synucleinopathies. NPJ Parkinsons Dis. 11, 196 (2025).","journal-title":"NPJ Parkinsons Dis."},{"key":"3001_CR17","doi-asserted-by":"publisher","DOI":"10.1038\/s41398-024-02873-4","volume":"14","author":"J Chen","year":"2024","unstructured":"Chen, J. et al. Multimodal digital assessment of depression with actigraphy and app in Hong Kong Chinese. Transl. Psychiatry 14, 150 (2024).","journal-title":"Transl. Psychiatry"},{"key":"3001_CR18","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-70165-4","volume":"14","author":"V Dentamaro","year":"2024","unstructured":"Dentamaro, V., Impedovo, D., Musti, L., Pirlo, G. & Taurisano, P. Enhancing early Parkinson\u2019s disease detection through multimodal deep learning and explainable AI: insights from the PPMI database. Sci. Rep. 14, 20941 (2024).","journal-title":"Sci. Rep."},{"key":"3001_CR19","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s41746-023-00751-9","volume":"6","author":"DW Joyce","year":"2023","unstructured":"Joyce, D. W., Kormilitzin, A., Smith, K. A. & Cipriani, A. Explainable artificial intelligence for mental health through transparency and interpretability for understandability. NPJ Digit. Med. 6, 6 (2023).","journal-title":"NPJ Digit. Med."},{"key":"3001_CR20","first-page":"4765","volume":"30","author":"SM Lundberg","year":"2017","unstructured":"Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. Neural Inf. Process Syst. 30, 4765\u20134774 (2017).","journal-title":"Neural Inf. Process Syst."},{"key":"3001_CR21","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-96575-6","volume":"15","author":"M Shen","year":"2025","unstructured":"Shen, M., Mortezaagha, P. & Rahgozar, A. Explainable artificial intelligence to diagnose early Parkinson\u2019s disease via voice analysis. Sci. Rep. 15, 11687 (2025).","journal-title":"Sci. Rep."},{"key":"3001_CR22","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1002\/ana.26517","volume":"93","author":"F Raschell\u00e0","year":"2023","unstructured":"Raschell\u00e0, F., Scafa, S., Puiatti, A., Martin Moraud, E. & Ratti, P.-L. Actigraphy enables home screening of rapid eye movement behavior disorder in Parkinson\u2019s disease. Ann. Neurol. 93, 317\u2013329 (2023).","journal-title":"Ann. Neurol."},{"key":"3001_CR23","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1016\/j.clinph.2019.01.011","volume":"130","author":"N Cooray","year":"2019","unstructured":"Cooray, N. et al. Detection of REM sleep behaviour disorder by automated polysomnography analysis. Clin. Neurophysiol. 130, 505\u2013514 (2019).","journal-title":"Clin. Neurophysiol."},{"key":"3001_CR24","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1038\/s41746-025-01999-z","volume":"8","author":"L Zhou","year":"2025","unstructured":"Zhou, L. et al. Actigraphy-based detection of isolated REM sleep behavior disorder: multicenter validation across devices and populations. NPJ Digit. Med. 8, 634 (2025).","journal-title":"NPJ Digit. Med."},{"key":"3001_CR25","doi-asserted-by":"publisher","first-page":"1219479","DOI":"10.3389\/fpsyt.2023.1219479","volume":"14","author":"E Kerz","year":"2023","unstructured":"Kerz, E., Zanwar, S., Qiao, Y. & Wiechmann, D. Toward explainable AI (XAI) for mental health detection based on language behavior. Front. Psychiatry 14, 1219479 (2023).","journal-title":"Front. Psychiatry"},{"key":"3001_CR26","unstructured":"THE CHINESE UNIVERSITY OF HONG KONG. SmartBrainApp. HK Brand at https:\/\/esearch.ipd.gov.hk\/nis-pos-view\/#\/tm\/details\/view\/3696595_306958135\/0\/0\/1\/10\/0\/1\/0\/null_null\/KCFeIShhcHBsaWNhdGlvbk51bWJlcjooMzA2OTU4MTM1KSkhXiEgQU5EICFeIShpc0RlYWRSZWNvcmRJbmRpY2F0b3I6KGZhbHNlKSkhXiEpIEFORCB0bVJlY29yZFNlcTox?lang=en (2025)."},{"key":"3001_CR27","doi-asserted-by":"crossref","unstructured":"Auer, A. et al. XLSTM: extended long short-term memory. in Advances in Neural Information Processing Systems(eds. Globerson, A. et al.) Vol. 37 107547\u2013107603 (Neural Information Processing Systems Foundation, Inc. (NeurIPS), 2024).","DOI":"10.52202\/079017-3417"},{"key":"3001_CR28","doi-asserted-by":"publisher","first-page":"1860","DOI":"10.1093\/brain\/aws093","volume":"135","author":"RB Postuma","year":"2012","unstructured":"Postuma, R. B., Lang, A. E., Gagnon, J. F., Pelletier, A. & Montplaisir, J. Y. How does Parkinsonism start? Prodromal parkinsonism motor changes in idiopathic REM sleep behaviour disorder. Brain 135, 1860\u20131870 (2012).","journal-title":"Brain"},{"key":"3001_CR29","doi-asserted-by":"publisher","first-page":"2051","DOI":"10.1093\/brain\/awz111","volume":"142","author":"S-M Fereshtehnejad","year":"2019","unstructured":"Fereshtehnejad, S.-M. et al. Evolution of prodromal Parkinson\u2019s disease and dementia with Lewy bodies: a prospective study. Brain 142, 2051\u20132067 (2019).","journal-title":"Brain"},{"key":"3001_CR30","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1038\/s41531-022-00414-8","volume":"8","author":"WS Lim","year":"2022","unstructured":"Lim, W. S. et al. An integrated biometric voice and facial features for early detection of Parkinson\u2019s disease. NPJ Parkinsons Dis. 8, 145 (2022).","journal-title":"NPJ Parkinsons Dis."},{"key":"3001_CR31","doi-asserted-by":"publisher","first-page":"106825","DOI":"10.1016\/j.sleep.2025.106825","volume":"136","author":"C Lam","year":"2025","unstructured":"Lam, C. et al. Identifying multimodal digital features of insomnia using an app in Hong Kong: an ecological momentary assessment study. Sleep. Med. 136, 106825 (2025).","journal-title":"Sleep. Med."},{"key":"3001_CR32","doi-asserted-by":"publisher","first-page":"e2364","DOI":"10.1212\/WNL.0000000000207947","volume":"101","author":"W-J Lee","year":"2023","unstructured":"Lee, W.-J. et al. REM sleep behavior disorder and its possible prodromes in general population: Prevalence, polysomnography findings, and associated factors. Neurology 101, e2364\u2013e2375 (2023).","journal-title":"Neurology"},{"key":"3001_CR33","doi-asserted-by":"publisher","first-page":"744","DOI":"10.1093\/brain\/awz030","volume":"142","author":"RB Postuma","year":"2019","unstructured":"Postuma, R. B. et al. Risk and predictors of dementia and Parkinsonism in idiopathic REM sleep behaviour disorder: a multicentre study. Brain 142, 744\u2013759 (2019).","journal-title":"Brain"},{"key":"3001_CR34","doi-asserted-by":"publisher","first-page":"1495","DOI":"10.1109\/TNSRE.2018.2851787","volume":"26","author":"J Rusz","year":"2018","unstructured":"Rusz, J. et al. Smartphone allows capture of speech abnormalities associated with high risk of developing Parkinson\u2019s disease. IEEE Trans. Neural Syst. Rehabil. Eng. 26, 1495\u20131507 (2018).","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"3001_CR35","doi-asserted-by":"publisher","first-page":"1752","DOI":"10.1002\/mds.29921","volume":"39","author":"V Illner","year":"2024","unstructured":"Illner, V. et al. Smartphone voice calls provide early biomarkers of Parkinsonism in rapid eye movement sleep behavior disorder. Mov. Disord. 39, 1752\u20131762 (2024).","journal-title":"Mov. Disord."},{"key":"3001_CR36","doi-asserted-by":"publisher","first-page":"624","DOI":"10.1002\/mds.28859","volume":"37","author":"A Janzen","year":"2022","unstructured":"Janzen, A. et al. Rapid eye movement sleep behavior disorder: abnormal cardiac image and progressive abnormal metabolic brain pattern. Mov. Disord. 37, 624\u2013629 (2022).","journal-title":"Mov. Disord."},{"key":"3001_CR37","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1038\/s41746-024-01217-2","volume":"7","author":"Y-M Sun","year":"2024","unstructured":"Sun, Y.-M., Wang, Z.-Y., Liang, Y.-Y., Hao, C.-W. & Shi, C.-H. Digital biomarkers for precision diagnosis and monitoring in Parkinson\u2019s disease. NPJ Digit. Med. 7, 218 (2024).","journal-title":"NPJ Digit. Med."},{"key":"3001_CR38","first-page":"116780","volume":"247","author":"T He","year":"2025","unstructured":"He, T., Chen, J. & Camacho, D. Empowering the measurement of early Parkinson\u2019s disease digital biomarkers by a features-attention fusion network and smartphone. Measurment 247, 116780 (2025).","journal-title":"Measurment"},{"key":"3001_CR39","doi-asserted-by":"publisher","first-page":"107274","DOI":"10.1016\/j.bspc.2024.107274","volume":"102","author":"S Shanthini","year":"2025","unstructured":"Shanthini, S. & Chandrasekar, A. Multimodal Parkinson\u2019s disease classification using voice signals and hand-drawn images based on deep learning enabled by chronological tangent search optimization. Biomed. Signal Process. Control 102, 107274 (2025).","journal-title":"Biomed. Signal Process. Control"},{"key":"3001_CR40","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-21407-6","volume":"15","author":"A Sar","year":"2025","unstructured":"Sar, A. et al. Multi-modal deep learning framework for early detection of Parkinson\u2019s disease using neurological and physiological data for high-fidelity diagnosis. Sci. Rep. 15, 34835 (2025).","journal-title":"Sci. Rep."},{"key":"3001_CR41","doi-asserted-by":"crossref","unstructured":"Appe, S. N., Murty, S. & Agarwal, S. Multi-modal deep learning for Parkinson\u2019s Disease detection using voice and gait biomarkers. Int. J. Environ. Sci. 11, 2835\u20132843 (2025).","DOI":"10.64252\/rq3faf47"},{"key":"3001_CR42","doi-asserted-by":"publisher","first-page":"100042","DOI":"10.1016\/j.patter.2020.100042","volume":"1","author":"H Zhang","year":"2020","unstructured":"Zhang, H., Deng, K., Li, H., Albin, R. L. & Guan, Y. Deep learning identifies digital biomarkers for self-reported Parkinson\u2019s disease. Patterns 1, 100042 (2020).","journal-title":"Patterns"},{"key":"3001_CR43","doi-asserted-by":"crossref","unstructured":"Frasca, M., La Torre, D., Pravettoni, G. & Cutica, I. Explainable and interpretable artificial intelligence in medicine: a systematic bibliometric review. Discov. Artif. Intell. 4, 15 (2024).","DOI":"10.1007\/s44163-024-00114-7"},{"key":"3001_CR44","doi-asserted-by":"publisher","first-page":"109370","DOI":"10.1016\/j.compeleceng.2024.109370","volume":"118","author":"Z Sadeghi","year":"2024","unstructured":"Sadeghi, Z. et al. A review of explainable artificial intelligence in healthcare. Comput. Electr. Eng. 118, 109370 (2024).","journal-title":"Comput. Electr. Eng."},{"key":"3001_CR45","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1038\/s41746-025-01958-8","volume":"8","author":"S Hur","year":"2025","unstructured":"Hur, S. et al. Comparison of SHAP and clinician friendly explanations reveals effects on clinical decision behaviour. NPJ Digit. Med. 8, 578 (2025).","journal-title":"NPJ Digit. Med."},{"key":"3001_CR46","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1016\/S1474-4422(24)00402-2","volume":"23","author":"G May\u00e0","year":"2024","unstructured":"May\u00e0, G. et al. Post-mortem neuropathology of idiopathic rapid eye movement sleep behaviour disorder: a case series. Lancet Neurol. 23, 1238\u20131251 (2024).","journal-title":"Lancet Neurol."},{"key":"3001_CR47","doi-asserted-by":"publisher","first-page":"105684","DOI":"10.1016\/j.neubiorev.2024.105684","volume":"162","author":"KS Bress","year":"2024","unstructured":"Bress, K. S. & Cascio, C. J. Sensorimotor regulation of facial expression - An untouched frontier. Neurosci. Biobehav. Rev. 162, 105684 (2024).","journal-title":"Neurosci. Biobehav. Rev."},{"key":"3001_CR48","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1212\/WNL.0000000000200235","volume":"99","author":"RB Postuma","year":"2022","unstructured":"Postuma, R. B. Neuroprotective trials in REM sleep behavior disorder: the way forward becomes clearer. Neurology 99, 19\u201325 (2022).","journal-title":"Neurology"},{"key":"3001_CR49","first-page":"tre","volume":"3","author":"RB Postuma","year":"2013","unstructured":"Postuma, R. B., Gagnon, J.-F. & Montplaisir, J. Y. REM sleep behavior disorder and prodromal neurodegeneration\u2013where are we headed? Tremor Other Hyperkinet Mov. 3, tre\u201303 (2013).","journal-title":"Tremor Other Hyperkinet Mov."},{"key":"3001_CR50","doi-asserted-by":"publisher","first-page":"104996","DOI":"10.1016\/j.nbd.2020.104996","volume":"143","author":"MT Hu","year":"2020","unstructured":"Hu, M. T. REM sleep behavior disorder (RBD). Neurobiol. Dis. 143, 104996 (2020).","journal-title":"Neurobiol. Dis."},{"key":"3001_CR51","doi-asserted-by":"publisher","first-page":"3613","DOI":"10.3390\/s24113613","volume":"24","author":"D Trabassi","year":"2024","unstructured":"Trabassi, D. et al. Optimizing rare disease gait classification through data balancing and generative AI: Insights from hereditary cerebellar ataxia. Sensors 24, 3613 (2024).","journal-title":"Sensors"},{"key":"3001_CR52","doi-asserted-by":"crossref","unstructured":"Zhang, W. et al. Update on neurodegenerative markers in isolated rapid eye movement sleep behavior disorder. Ageing Neurodegener. Dis. 5, N-A (2025).","DOI":"10.20517\/and.2025.20"},{"key":"3001_CR53","first-page":"1","volume":"6","author":"Q Sun","year":"2025","unstructured":"Sun, Q., Akman, A. & Schuller, B. W. Explainable artificial intelligence for medical applications: a review. ACM Trans. Comput. Healthc. 6, 1\u201331 (2025).","journal-title":"ACM Trans. Comput. Healthc."},{"key":"3001_CR54","doi-asserted-by":"publisher","first-page":"6649","DOI":"10.3390\/s25216649","volume":"25","author":"N Hettikankanamage","year":"2025","unstructured":"Hettikankanamage, N. et al. EXplainable Artificial Intelligence (XAI): a systematic review for unveiling the black box models and their relevance to biomedical imaging and sensing. Sensors 25, 6649 (2025).","journal-title":"Sensors"},{"key":"3001_CR55","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2024.106480","volume":"95","author":"C Lv","year":"2024","unstructured":"Lv, C. et al. Leveraging multimodal deep learning framework and a comprehensive audio-visual dataset to advance Parkinson\u2019s detection. Biomed. Signal Process. Control 95, 106480 (2024).","journal-title":"Biomed. Signal Process. Control"},{"key":"3001_CR56","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1038\/s41531-025-01012-0","volume":"11","author":"S Tang","year":"2025","unstructured":"Tang, S. et al. Brain-clinical biotyping in patients with idiopathic REM sleep behavior disorder. NPJ Parkinsons Dis. 11, 156 (2025).","journal-title":"NPJ Parkinsons Dis."},{"key":"3001_CR57","doi-asserted-by":"publisher","first-page":"51","DOI":"10.3390\/jpm11010051","volume":"11","author":"F Assogna","year":"2021","unstructured":"Assogna, F. et al. Cognitive and neuropsychiatric profiles in idiopathic rapid eye movement sleep behavior disorder and Parkinson\u2019s disease. J. Pers. Med. 11, 51 (2021).","journal-title":"J. Pers. Med."},{"key":"3001_CR58","doi-asserted-by":"crossref","unstructured":"Moon, S. H., Hong, J. K., Lee, M., Kim, H. H. & Yoon, I.-Y. Impact of depression on cognitive function and phenoconversion to neurodegenerative diseases in patients with isolated REM sleep behavior disorder. Acta Neurol. Scand. 2024, 9980063 (2024).","DOI":"10.1155\/ane\/9980063"},{"key":"3001_CR59","doi-asserted-by":"crossref","unstructured":"Dalal, N. & Triggs, B. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905) vol. 1 886\u2013893 (IEEE, 2005).","DOI":"10.1109\/CVPR.2005.177"},{"key":"3001_CR60","doi-asserted-by":"crossref","unstructured":"Baltrusaitis, T., Robinson, P. & Morency, L.-P. OpenFace: an open source facial behavior analysis toolkit. In 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) 1\u201310 (IEEE, 2016).","DOI":"10.1109\/WACV.2016.7477553"},{"key":"3001_CR61","doi-asserted-by":"publisher","unstructured":"Eyben, F., W\u00f6llmer, M. & Schuller, B. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proc. 18th ACM international conference on Multimedia. https:\/\/doi.org\/10.1145\/1873951.1874246. (ACM, 2010).","DOI":"10.1145\/1873951.1874246"},{"key":"3001_CR62","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1002\/mds.21198","volume":"22","author":"CG Goetz","year":"2007","unstructured":"Goetz, C. G. et al. Movement Disorder Society-sponsored revision of the Unified Parkinson\u2019s Disease Rating Scale (MDS-UPDRS): process, format, and clinimetric testing plan. Mov. Disord. 22, 41\u201347 (2007).","journal-title":"Mov. Disord."},{"key":"3001_CR63","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1001\/archpsyc.1961.01710120031004","volume":"4","author":"AT Beck","year":"1961","unstructured":"Beck, A. T., Ward, C. H., Mendelson, M., Mock, J. & Erbaugh, J. An inventory for measuring depression. Arch. Gen. Psychiatry 4, 561\u2013571 (1961).","journal-title":"Arch. Gen. Psychiatry"},{"key":"3001_CR64","doi-asserted-by":"publisher","unstructured":"Wang, X., Fan, L. & Yu, H. PAI-CUHK\/RBDMDD: First Release for Paper Submission. https:\/\/doi.org\/10.5281\/ZENODO.17929750 (Zenodo, 2025).","DOI":"10.5281\/ZENODO.17929750"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-03001-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-03001-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-03001-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,11,17]],"date-time":"2026-11-17T02:03:10Z","timestamp":1794880990000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-026-03001-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,14]]},"references-count":64,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["3001"],"URL":"https:\/\/doi.org\/10.1038\/s41746-026-03001-w","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,14]]},"assertion":[{"value":"16 December 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 July 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 July 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Y.K.W. received a consultation fee from Eisai Co., Ltd., an honorarium from Eisai Hong Kong for a lecture, travel support from Lundbeck HK limited for the overseas conference, and an honorarium from Aculys Pharma, Inc for a lecture and J.W.Y.C. received a personal fee from Eisai Co., Ltd and travel support from Lundbeck HK limited for overseas conference, which is outside this work. The other authors do not have a competing interest. An US provisional patent application (No. 64\/069,357) with invention titled \u201cSystems and Methods for Identifying Digital Biomarkers and Predicting Comorbid REM Sleep Behavior Disorder in Major Depressive Disorder Using Multimodal Artificial Intelligence\u201d with L.F., Y.K.W., and B.H. as inventors and the Chinese University of Hong Kong (CUHK) as the patent owner was received. This US provisional patent covers the AI system and algorithm development aspects of this manuscript.","order":1,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Ethics"}}],"article-number":"894"}}