{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T17:10:35Z","timestamp":1786554635393,"version":"build-2736575974"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"7","funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2024YFB3311602"],"award-info":[{"award-number":["2024YFB3311602"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["72188101, 62272144, 62020106007, 62501224"],"award-info":[{"award-number":["72188101, 62272144, 62020106007, 62501224"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Major Project of Anhui Province","award":["202203a05020011"],"award-info":[{"award-number":["202203a05020011"]}]},{"DOI":"10.13039\/501100003995","name":"Anhui Provincial Natural Science Foundation","doi-asserted-by":"crossref","award":["2408085J040"],"award-info":[{"award-number":["2408085J040"]}],"id":[{"id":"10.13039\/501100003995","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["JZ2023YQTD0072, JZ2024HGTG0309, JZ2024AHST0337, JZ2025HGQA0139, JZ2025HGTA0160"],"award-info":[{"award-number":["JZ2023YQTD0072, JZ2024HGTG0309, JZ2024AHST0337, JZ2025HGQA0139, JZ2025HGTA0160"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2026,7,31]]},"abstract":"<jats:p>\n                    Video-based multimodal depression estimation is a challenging task that relies on comprehensive video understanding and analysis to predict depression severity. This process involves capturing discriminative information from multimodal features, including auditory, textual, and visual signals in the video data. Existing foundation models primarily focus on fusing multimodal features but often overlook two essential aspects: (1)\n                    <jats:italic toggle=\"yes\">the alignment of intra- and inter-modality<\/jats:italic>\n                    and (2)\n                    <jats:italic toggle=\"yes\">preserving modality-specific semantic information while eliminating redundancy<\/jats:italic>\n                    . Hence, these issues inevitably result in semantic misalignment between modalities, leading the model to learn redundant or even conflicting representations. To tackle these challenges, we propose a Contrastive Modality Alignment and Fusion (CMAF) framework. Specifically, each modality is divided into multiple segments, and a dynamic normalization strategy is applied to align the temporal context within each modality. Simultaneously, contrastive learning is utilized to achieve semantic alignment across different modalities. Together, these components form the Contextual Modality Alignment (CMA) module. Furthermore, a Cross-Attention Contrastive Fusion (CCF) module is presented, where cross-attention is used to achieve complementarity and enhancement between modalities, while contrastive learning further promotes modal consistency and preserves modality-specific information without redundancy. Therefore, the fused representation can integrate the features of each modality without losing the specific information of the unimodal. Extensive experiments on public datasets, including CMDC, DAIC-WOZ, and E-DAIC, demonstrate that CMAF outperforms advanced depression estimation methods and highlights the essence of effective alignment and fusion.\n                  <\/jats:p>","DOI":"10.1145\/3778172","type":"journal-article","created":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T09:19:54Z","timestamp":1764235194000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Multimodal Depression Estimation via Contrastive Modality Alignment and Fusion"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-8399-8322","authenticated-orcid":false,"given":"Xinke","family":"Wang","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence, Anhui University, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-0669-268X","authenticated-orcid":false,"given":"Xin","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4462-315X","authenticated-orcid":false,"given":"Jingyuan","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1911-4790","authenticated-orcid":false,"given":"Weijie","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2594-254X","authenticated-orcid":false,"given":"Dan","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3094-7735","authenticated-orcid":false,"given":"Meng","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China and Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,24]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3462763"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133952"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2021.3072579"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2019.8756584"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0924-9338(11)73416-2"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3347320.3357695"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","unstructured":"Xinyu Feng Yuming Lin Lihua He You Li Liang Chang and Ya Zhou. 2024. Knowledge-guided dynamic modality attention fusion framework for multimodal sentiment analysis. arXiv:2410.04491. Retrieved from https:\/\/arxiv.org\/abs\/2410.04491","DOI":"10.18653\/v1\/2024.findings-emnlp.865"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.63317\/3o7bccg9xequ"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2024.3358415"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2022.3202316"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2023.3273614"},{"key":"e_1_3_1_13_2","unstructured":"Edward J. Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2021. LoRA: Low-rank adaptation of large language models. arXiv:2106.09685. Retrieved from https:\/\/arxiv.org\/abs\/2106.09685"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2023.3263056"},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","unstructured":"Meiling Li Yuting Wei Yangfu Zhu Siqi Wei and Bin Wu. 2024. Enhancing multimodal depression detection with intra- and inter-sample contrastive learning. 684 C (2024) 1\u201315.","DOI":"10.1016\/j.ins.2024.121282"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3513263"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2024.3370149"},{"key":"e_1_3_1_18_2","unstructured":"Ping Luo Jiamin Ren Zhanglin Peng Ruimao Zhang and Jingyu Li. 2018. Differentiable learning-to-normalize via switchable normalization. arXiv:1806.10779. Retrieved from https:\/\/arxiv.org\/abs\/1806.10779"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3347320.3357694"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.biopsych.2012.03.015"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2025.3533480"},{"key":"e_1_3_1_22_2","unstructured":"Aaron van den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv:1807.03748. Retrieved from https:\/\/arxiv.org\/abs\/1807.03748"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1038\/nrdp.2016.65"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681227"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2023.3296318"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3347320.3357688"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133953"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1080\/02699930441000030"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.3390\/s21144764"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548025"},{"key":"e_1_3_1_31_2","unstructured":"Chayan Tank Sarthak Pol Vinayak Katoch Shaina Mehta Avinash Anand and Rajiv Ratn Shah. 2024. Depression detection and analysis using large language models on textual and audio-visual modalities. arXiv:2407.06125. Retrieved from https:\/\/arxiv.org\/abs\/2407.06125"},{"key":"e_1_3_1_32_2","first-page":"1","volume-title":"IEEE International Conference on Consumer Electronics","author":"Teng Shiyu","year":"2024","unstructured":"Shiyu Teng, Shurong Chai, Jiaqing Liu, Tomoko Tateyama, Lanfen Lin, and Yen-Wei Chen. 2024. Multi-modal and multi-task depression detection with sentiment assistance. In IEEE International Conference on Consumer Electronics, 1\u20135."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/EMBC53108.2024.10782904"},{"key":"e_1_3_1_34_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et al. 2023. Llama: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3179478"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00252"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2025.3533543"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2998532"},{"key":"e_1_3_1_39_2","first-page":"623","volume-title":"European Conference on Computer Vision","author":"Wei Ping-Cheng","year":"2022","unstructured":"Ping-Cheng Wei, Kunyu Peng, Alina Roitberg, Kailun Yang, Jiaming Zhang, and Rainer Stiefelhagen. 2022. Multi-modal depression estimation based on Sub-attentional fusion. In European Conference on Computer Vision. Springer, 623\u2013639."},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2024.3411616"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133944.3133948"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3347320.3357696"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-72353-7_5"},{"key":"e_1_3_1_44_2","unstructured":"Wei Zhang Juan Chen En Zhu Wenhong Cheng YunPeng Li and Yanbo J. Wang. 2025. MLlm-DR: Towards explainable depression recognition with multimodal large language models. arXiv:2507.05591. Retrieved from https:\/\/arxiv.org\/abs\/2507.05591"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s43657-023-00152-8"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3723005"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM55620.2022.9995184"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3665247"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2022.3181210"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3778172","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T14:40:42Z","timestamp":1782312042000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3778172"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,24]]},"references-count":48,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,7,31]]}},"alternative-id":["10.1145\/3778172"],"URL":"https:\/\/doi.org\/10.1145\/3778172","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,24]]},"assertion":[{"value":"2025-04-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-10","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}