{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T21:18:18Z","timestamp":1793395098352,"version":"4.5.0"},"reference-count":35,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T00:00:00Z","timestamp":1727136000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004731","name":"Natural Science Foundation of Zhejiang Province","doi-asserted-by":"publisher","award":["LQ21F020025"],"award-info":[{"award-number":["LQ21F020025"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010248","name":"Zhejiang Province Public Welfare Technology Application Research Project","doi-asserted-by":"publisher","award":["LGF21F020024"],"award-info":[{"award-number":["LGF21F020024"]}],"id":[{"id":"10.13039\/501100010248","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Natural Science Foundation of Zhejiang Province","doi-asserted-by":"publisher","award":["LQ23F030002"],"award-info":[{"award-number":["LQ23F030002"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"<jats:p>Alzheimer\u2019s Disease is a neurodegenerative disorder, and one of its common and prominent early symptoms is language impairment. Therefore, early diagnosis of Alzheimer\u2019s Disease through speech and text information is of significant importance. However, the multimodal data is often complex and inconsistent, which leads to inadequate feature extraction. To address the problem, We propose a model for early diagnosis of Alzheimer\u2019s Disease based on multimodal attention(EDAMM). Specifically, we first evaluate and select three optimal feature extraction methods, Wav2Vec2.0, TF-IDF and Word2Vec, to extract acoustic and linguistic features. Next, by leveraging self-attention mechanism and cross-modal attention mechanisms, we generate fused features to enhance and capture the inter-modal correlation information. Finally, we concatenate the multimodal features into a composite feature vector and employ a Neural Network(NN) classifier to diagnose Alzheimer\u2019s Disease. To evaluate EDAMM, we perform experiments on two public datasets, i.e., NCMMSC2021 and ADReSSo. The results show that EDAMM improves the performance of Alzheimer\u2019s Disease diagnosis over state-of-the-art baseline approaches on both datasets.<\/jats:p>","DOI":"10.1371\/journal.pone.0310966","type":"journal-article","created":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T17:23:57Z","timestamp":1727198637000},"page":"e0310966","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":18,"title":["Early diagnosis of Alzheimer\u2019s Disease based on multi-attention mechanism"],"prefix":"10.1371","volume":"19","author":[{"given":"Xinli","family":"Yang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kefen","family":"Hong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4492-3643","authenticated-orcid":true,"given":"Denghui","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ke","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2024,9,24]]},"reference":[{"issue":"24","key":"pone.0310966.ref001","doi-asserted-by":"crossref","first-page":"5789","DOI":"10.3390\/molecules25245789","article-title":"Comprehensive review on alzheimer\u2019s disease: causes and treatment","volume":"25","author":"Z Breijyeh","year":"2020","journal-title":"Molecule"},{"issue":"3","key":"pone.0310966.ref002","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1097\/WAD.0000000000000373","article-title":"Conversion of mild cognitive impairment to alzheimer\u2019s disease in monolingual and bilingual patients","volume":"34","author":"M Berkes","year":"2020","journal-title":"Alzheimer disease and associated disorders"},{"issue":"1","key":"pone.0310966.ref003","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13024-019-0333-5","article-title":"The neuropathological diagnosis of alzheimer\u2019s disease","volume":"14","author":"MA DeTure","year":"2019","journal-title":"Molecular neurodegeneration"},{"key":"pone.0310966.ref004","doi-asserted-by":"crossref","unstructured":"Li J, Song B, Qian C. A multiple stage discriminative event based model for Alzheimer\u2019s disease progression timeline estimation. In 2020 IEEE\/WIC\/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), IEEE. 2020;339\u2013344.","DOI":"10.1109\/WIIAT50758.2020.00048"},{"issue":"2","key":"pone.0310966.ref005","doi-asserted-by":"crossref","first-page":"56","DOI":"10.31083\/j.jin2102056","article-title":"Diagnosis of alzheimer\u2019s disease by feature weighted-lstm: a preliminary study of temporal features in brain resting-state fmri","volume":"21","author":"J Li","year":"2022","journal-title":"Journal of Integrative Neuroscience"},{"key":"pone.0310966.ref006","doi-asserted-by":"crossref","first-page":"104537","DOI":"10.1016\/j.compbiomed.2021.104537","article-title":"Deep sequence modelling for alzheimer\u2019s disease detection using MRI","volume":"134","author":"A Ebrahimi","year":"2021","journal-title":"Computers in Biology and Medicine"},{"issue":"4","key":"pone.0310966.ref007","doi-asserted-by":"crossref","first-page":"1043","DOI":"10.1007\/s13246-022-01165-9","article-title":"Multi-modality mri for alzheimer\u2019s disease detection using deep learning","volume":"45","author":"L Houria","year":"2022","journal-title":"Physical and Engineering Sciences in Medicine"},{"issue":"2","key":"pone.0310966.ref008","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1006\/brln.2000.2357","article-title":"Phonological and articulatory impairment in alzheimer\u2019s disease: a case series","volume":"75","author":"K Croot","year":"2000","journal-title":"Brain and language"},{"issue":"3","key":"pone.0310966.ref009","doi-asserted-by":"crossref","first-page":"198","DOI":"10.3109\/02699206.2010.521612","article-title":"Syntactic and lexical context of pauses and hesitations in the discourse of alzheimer patients and healthy elderly subjects","volume":"25","author":"F Gayraud","year":"2011","journal-title":"Clinical linguistics & phonetics"},{"key":"pone.0310966.ref010","unstructured":"Qin Y, Liu W, Peng Z, Ng SI, Li J, Hu H, et al. Exploiting pre-trained asr models for alzheimer\u2019s disease recognition through spontaneous speech. arXiv preprint arXiv:2110.01493. 2021."},{"key":"pone.0310966.ref011","doi-asserted-by":"crossref","first-page":"102023","DOI":"10.1016\/j.simpat.2019.102023","article-title":"A new machine learning method for identifying alzheimer\u2019s disease","volume":"99","author":"L Liu","year":"2020","journal-title":"Simulation Modelling Practice and Theory"},{"key":"pone.0310966.ref012","doi-asserted-by":"crossref","first-page":"101113","DOI":"10.1016\/j.csl.2020.101113","article-title":"Linguistic features and automatic classifiers for identifying mild cognitive impairment and dementia","volume":"65","author":"L Calz\u00e0","year":"2021","journal-title":"Computer Speech & Language"},{"issue":"1","key":"pone.0310966.ref013","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1186\/s12916-022-02584-x","article-title":"Speech silence character as a diagnostic biomarker of early cognitive decline and its functional mechanism: a multicenter cross-sectional cohort study","volume":"20","author":"HL Wang","year":"2022","journal-title":"BMC medicine"},{"issue":"1","key":"pone.0310966.ref014","doi-asserted-by":"crossref","first-page":"28","DOI":"10.3390\/brainsci13010028","article-title":"Artificial intelligence-enabled end-to-end detection and assessment of alzheimer\u2019s disease using voice","volume":"13","author":"F Agbavor","year":"2022","journal-title":"Brain sciences"},{"key":"pone.0310966.ref015","doi-asserted-by":"crossref","first-page":"102761","DOI":"10.1016\/j.ijhcs.2021.102761","article-title":"Exploiting linguistic information from Nepali transcripts for early detection of alzheimer\u2019s disease using natural language processing and machine learning techniques","volume":"160","author":"S Adhikari","year":"2022","journal-title":"International Journal of Human-Computer Studies"},{"issue":"8","key":"pone.0310966.ref016","doi-asserted-by":"crossref","first-page":"4153","DOI":"10.1109\/JBHI.2022.3172479","article-title":"Explainable identification of dementia from transcripts using transformer networks","volume":"26","author":"L Ilias","year":"2022","journal-title":"IEEE Journal of Biomedical and Health Informatics"},{"issue":"6","key":"pone.0310966.ref017","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.31577\/cai_2022_6_1589","article-title":"Computational intelligent models for alzheimer\u2019s prediction using audio transcript data","volume":"41","author":"YF Khan","year":"2022","journal-title":"Computing and Informatics"},{"key":"pone.0310966.ref018","doi-asserted-by":"crossref","first-page":"32750","DOI":"10.1109\/ACCESS.2022.3161749","article-title":"Stacked deep dense neural network model to predict alzheimer\u2019s dementia using audio transcript data","volume":"10","author":"YF Khan","year":"2022","journal-title":"IEEE Access"},{"key":"pone.0310966.ref019","first-page":"10944","article-title":"What makes multi-modal learning better than single (provably)","volume":"34","author":"Y Huang","year":"2021","journal-title":"Advances in Neural Information Processing Systems"},{"issue":"12","key":"pone.0310966.ref020","doi-asserted-by":"crossref","first-page":"16029","DOI":"10.1007\/s10489-022-04255-z","article-title":"Multimodal fusion for alzheimer\u2019s disease recognition","volume":"53","author":"Y Ying","year":"2023","journal-title":"Applied Intelligence"},{"key":"pone.0310966.ref021","unstructured":"Mittal A, Sahoo S, Datar A, Kadiwala J, Shalu H, Mathew J. Multi-modal detection of alzheimer\u2019s disease from speech and text. arXiv preprint arXiv:2012.00096. 2020."},{"key":"pone.0310966.ref022","doi-asserted-by":"crossref","first-page":"4961","DOI":"10.21437\/Interspeech.2020-2698","volume-title":"Proceedings of Interspeech 2020","author":"Y Pan","year":"2020"},{"key":"pone.0310966.ref023","doi-asserted-by":"crossref","unstructured":"Li J, Song K, Li J, Zheng B, Li D, Wu X, et al. Leveraging pretrained representations with task-related keywords for alzheimer\u2019s disease detection. In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2023;1\u20135.","DOI":"10.1109\/ICASSP49357.2023.10096205"},{"key":"pone.0310966.ref024","doi-asserted-by":"crossref","first-page":"642647","DOI":"10.3389\/fnagi.2021.642647","article-title":"Temporal integration of text transcripts and acoustic features for alzheimer\u2019s diagnosis based on spontaneous speech","volume":"13","author":"M Martinc","year":"2021","journal-title":"Frontiers in Aging Neuroscience"},{"key":"pone.0310966.ref025","first-page":"12449","article-title":"Wav2vec 2.0: a framework for self-supervised learning of speech representations","volume":"33","author":"A Baevski","year":"2020","journal-title":"Advances in neural information processing systems"},{"key":"pone.0310966.ref026","unstructured":"Mikolov T. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. 2013."},{"key":"pone.0310966.ref027","article-title":"Attention is all you need","volume":"30","author":"A Vaswani","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"pone.0310966.ref028","doi-asserted-by":"crossref","unstructured":"Tsai YHH, Bai S, Liang PP, Kolter JZ, Morency LP, Salakhutdinov R. Multimodal transformer for unaligned multimodal language sequences. In: Proceedings of the conference. Association for computational linguistics. Meeting. NIH Public Access. 2019;2019:6558.","DOI":"10.18653\/v1\/P19-1656"},{"key":"pone.0310966.ref029","doi-asserted-by":"crossref","unstructured":"Luz S, Haider F, de la Fuente S, Fromm D, MacWhinney B. Detecting cognitive decline using speech only: the adresso challenge. arXiv preprint arXiv:2104.09356. 2021. Dataset available at: https:\/\/doi.org\/10.1038\/s42003-021-01817-8.","DOI":"10.1101\/2021.03.24.21254263"},{"key":"pone.0310966.ref030","unstructured":"NCMMSC 2021 Speech Corpus. National Conference on Man-Machine Speech Communication. 2021. Dataset available at: http:\/\/www.ncmmsc2021.org\/."},{"key":"pone.0310966.ref031","first-page":"3830","volume-title":"Interspeech","author":"J Chen","year":"2021"},{"key":"pone.0310966.ref032","first-page":"3790","volume-title":"Interspeech","author":"Y Zhu","year":"2021"},{"key":"pone.0310966.ref033","doi-asserted-by":"crossref","unstructured":"Qiao Y, Yin X, Wiechmann D, Kerz E. Alzheimer\u2019s disease detection from spontaneous speech through combining linguistic complexity and (dis) fluency features with pretrained language models. arXiv preprint arXiv:2106.08689. 2021.","DOI":"10.21437\/Interspeech.2021-1415"},{"key":"pone.0310966.ref034","first-page":"3810","article-title":"Using the outputs of different automatic speech recognition paradigms for acoustic-and bert-based Aalzheimer\u2019s dementia detection through Sspontaneous speech","author":"Y Pan","year":"2021","journal-title":"Interspeech"},{"key":"pone.0310966.ref035","doi-asserted-by":"crossref","unstructured":"Wang N, Cao Y, Hao S, Shao Z, Subbalakshmi K. Modular multi-modal attention network for alzheimer\u2019s disease detection using patient audio and language data. In: Interspeech. 2021;3835\u20133839.","DOI":"10.21437\/Interspeech.2021-2024"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0310966","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,24]],"date-time":"2024-09-24T17:24:11Z","timestamp":1727198651000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0310966"}},"subtitle":[],"editor":[{"given":"Suliman Mohamed","family":"Fati","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2024,9,24]]},"references-count":35,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,9,24]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0310966","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,24]]}}}