{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T18:26:46Z","timestamp":1789756006338,"version":"4.0.1"},"reference-count":24,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"52","license":[{"start":{"date-parts":[[2024,12,27]],"date-time":"2024-12-27T00:00:00Z","timestamp":1735257600000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,12,27]]},"abstract":"<jats:p>Alzheimer disease (AD) is the most commonly occurring neurodegenerative disease. However, current diagnostics for AD primarily rely on invasive tests, which limit the application of diagnostic procedures in early screening. Speech, as a noninvasive biomarker, is closely associated with AD but has not been fully leveraged as a diagnostic tool. This study develops a novel early AD diagnosis method that uses primitive speech and explores its potential application in community screening. Moreover, the study proposes an innovative multimodal method for speech feature fusion that combines acoustic and semantic information to differentiate patients with AD from normal controls. This method uses the ImageBind audio encoder to extract acoustic features and the Embeddings from Language Model to extract semantic features, thereby effectively integrating the features by mid-level fusion. The training set comprises 166 speech recordings, which comprise 87 samples from individuals with AD and 79 samples from healthy control subjects. The ratio of training set to test set is 7:3. Evaluation of the Alzheimer dementia recognition through spontaneous speech only dataset showed that the proposed model achieved a classification accuracy of 0.903 and a recall rate of 1, and it considerably outperformed existing baseline models, thereby confirming the effectiveness of the proposed approach to AD diagnosis. This study applies the multimodal fusion of speech features to an early AD diagnostic procedure and achieves excellent performance. The findings of this study not only provide a new approach to noninvasive AD screening but also open new pathways to the early diagnosis of other neurodegenerative diseases.<\/jats:p>","DOI":"10.1097\/md.0000000000041016","type":"journal-article","created":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T21:03:45Z","timestamp":1735592625000},"page":"e41016","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":7,"title":["Classification and diagnosis model for Alzheimer\u2019s disease based on multimodal data fusion"],"prefix":"10.1097","volume":"103","author":[{"given":"Yaqin","family":"Fu","sequence":"first","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yujie","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linshuai","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengfei","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lu","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9370-800X","authenticated-orcid":false,"given":"Tao","family":"Jiang","sequence":"additional","affiliation":[{"name":"School of Intelligent Medicine, Chengdu University of Traditional Chinese Medicine, Chengdu, China."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"276","reference":[{"key":"R1-20260911","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1016\/S0140-6736(20)32205-4","article-title":"Alzheimer\u2019s disease.","volume":"397","author":"Scheltens","year":"2021","journal-title":"Lancet"},{"key":"R2-20260911","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1111\/neup.12626","article-title":"Aging and Alzheimer\u2019s disease pathology.","volume":"40","author":"Sengoku","year":"2020","journal-title":"Neuropathology"},{"key":"R3-20260911","doi-asserted-by":"crossref","first-page":"104455","DOI":"10.1016\/j.ebiom.2023.104455","article-title":"Altered global signal topography in Alzheimer\u2019s disease.","volume":"89","author":"Chen","year":"2023","journal-title":"Ebiomedicine"},{"key":"R4-20260911","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.3390\/ijms24021059","article-title":"Alzheimer\u2019s disease \u2013 Biochemical and psychological background for diagnosis and treatment.","volume":"24","author":"Beata","year":"2023","journal-title":"Int J Mol Sci"},{"key":"R6-20260911","doi-asserted-by":"crossref","first-page":"15333175221106901","DOI":"10.1177\/15333175221106901","article-title":"The cognitive and language profile of dementia with Lewy bodies.","volume":"37","author":"Macoir","year":"2022","journal-title":"Am J Alzheimers Dis Other Demen"},{"key":"R7-20260911","doi-asserted-by":"crossref","first-page":"221640","DOI":"10.1109\/ACCESS.2020.3043201","article-title":"A novel approach for classification of speech emotions based on deep and acoustic features.","volume":"8","author":"Er","year":"2020","journal-title":"IEEE Access"},{"key":"R8-20260911","doi-asserted-by":"crossref","first-page":"27","DOI":"10.3390\/bioengineering9010027","article-title":"Speech-and language-based classification of Alzheimer\u2019s disease: a systematic review.","volume":"9","author":"Vigo","year":"2022","journal-title":"Bioengineering"},{"key":"R9-20260911","doi-asserted-by":"crossref","first-page":"16029","DOI":"10.1007\/s10489-022-04255-z","article-title":"Multimodal fusion for Alzheimer\u2019s disease recognition.","volume":"53","author":"Ying","year":"2023","journal-title":"Appl Intell"},{"key":"R11-20260911","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10072-005-0467-9","article-title":"Detecting subtle spontaneous language decline in early Alzheimer\u2019s disease with a picture description task.","volume":"26","author":"Forbes-McKay","year":"2005","journal-title":"Neurol Sci"},{"key":"R12-20260911","doi-asserted-by":"crossref","first-page":"437","DOI":"10.3389\/fnagi.2017.00437","article-title":"Declines in connected language are associated with very early mild cognitive impairment: results from the Wisconsin registry for Alzheimer\u2019s prevention.","volume":"9","author":"Mueller","year":"2018","journal-title":"Front Aging Neurosci"},{"key":"R16-20260911","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/j.trci.2017.01.006","article-title":"Predicting mild cognitive impairment from spontaneous spoken utterances.","volume":"3","author":"Asgari","year":"2017","journal-title":"Alzheimers Dement (N Y)"},{"key":"R17-20260911","doi-asserted-by":"crossref","first-page":"407","DOI":"10.3233\/JAD-150520","article-title":"Linguistic features identify Alzheimer\u2019s disease in narrative speech.","volume":"49","author":"Fraser","year":"2015","journal-title":"J Alzheimers Dis"},{"key":"R19-20260911","first-page":"4171","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding.","volume":"1","author":"Devlin","year":"2018"},{"key":"R20-20260911","article-title":"Roberta: A robustly optimized bert pretraining approach.","author":"Liu","year":"2019"},{"key":"R21-20260911","doi-asserted-by":"crossref","first-page":"830943","DOI":"10.3389\/fnagi.2022.830943","article-title":"Multimodal deep learning models for detecting dementia from speech and transcripts.","volume":"14","author":"Ilias","year":"2022","journal-title":"Front Aging Neurosci"},{"key":"R22-20260911","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.3233\/JAD-210064","article-title":"PET imaging of tau pathology and amyloid-\u03b2, and MRI for Alzheimer\u2019s disease feature fusion and multimodal classification.","volume":"84","author":"Shojaie","year":"2021","journal-title":"J Alzheimers Dis"},{"key":"R23-20260911","doi-asserted-by":"crossref","first-page":"114135","DOI":"10.1016\/j.psychres.2021.114135","article-title":"Detecting formal thought disorder by deep contextualized word representations.","volume":"304","author":"Sarzynska-Wawer","year":"2021","journal-title":"Psychiatry Res"},{"key":"R26-20260911","first-page":"267","article-title":"Using speech emotion recognition as a longitudinal biomarker for Alzheimer\u2019s disease.","volume":"17","author":"Gong","year":"2023","journal-title":"Int J Biomed Biol Eng"},{"key":"R28-20260911","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1075\/ps.17011.cum","article-title":"Describing the cookie theft picture: sources of breakdown in Alzheimer\u2019s dementia.","volume":"10","author":"Cummings","year":"2019","journal-title":"Pragmat Soc"},{"key":"R29-20260911","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1186\/s13040-023-00322-4","article-title":"The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification.","volume":"16","author":"Chicco","year":"2023","journal-title":"BioData Min"},{"key":"R30-20260911","first-page":"3830","article-title":"Automatic detection of Alzheimer\u2019s disease using spontaneous speech only.","volume":"2021","author":"Chen","year":"2021","journal-title":"Interspeech"},{"key":"R36-20260911","doi-asserted-by":"crossref","first-page":"110834","DOI":"10.1016\/j.knosys.2023.110834","article-title":"Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech.","volume":"277","author":"Ilias","year":"2023","journal-title":"Knowl-Based Syst"},{"key":"R37-20260911","first-page":"3820","article-title":"Alzheimer\u2019s dementia recognition using acoustic, lexical, disfluency and speech pause features robust to noisy inputs.","volume":"2106.15684","author":"Rohanian","year":"2021","journal-title":"ArXiv"},{"key":"R38-20260911","first-page":"1162","article-title":"Detecting depression in Alzheimer\u2019s disease and MCI by speech analysis.","volume":"99","author":"Abdallah-Qasaimeh","year":"2021","journal-title":"J Theor Appl Inf Technol"}],"container-title":["Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.ovid.com\/10.1097\/MD.0000000000041016","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T16:05:57Z","timestamp":1789142757000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ovid.com\/10.1097\/MD.0000000000041016"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,27]]},"references-count":24,"journal-issue":{"issue":"52","published-print":{"date-parts":[[2024]]}},"URL":"https:\/\/doi.org\/10.1097\/md.0000000000041016","relation":{},"ISSN":["1536-5964","0025-7974"],"issn-type":[{"value":"1536-5964","type":"electronic"},{"value":"0025-7974","type":"print"}],"subject":[],"published":{"date-parts":[[2024,12,27]]}}}