{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T21:00:10Z","timestamp":1773954010301,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","funder":[{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LQ24F020033"],"award-info":[{"award-number":["LQ24F020033"]}]},{"DOI":"10.13039\/100022955","name":"Fundamental Research Funds for the Provincial Universities of Zhejiang","doi-asserted-by":"crossref","award":["GK239909299001-018"],"award-info":[{"award-number":["GK239909299001-018"]}],"id":[{"id":"10.13039\/100022955","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["2471169"],"award-info":[{"award-number":["2471169"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program of China","award":["2023YFE0114900"],"award-info":[{"award-number":["2023YFE0114900"]}]},{"name":"Zhejiang Key Research and Development Program","award":["2025C04001"],"award-info":[{"award-number":["2025C04001"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>Multimodal sarcasm analysis is one of the most challenging research branch of the sentiment analysis area, due to the presence of cross-modality incongruity. However, existing works mainly attend to the coarse-grained incongruity analysis, and totally ignore the sentiment semantic coupling issue. This indeed limits the discriminate capability and robustness of the sarcasm analysis model. In order to address the above issue, we propose a novel Fine-grained Semantic Disentanglement Network (FSDN). Specifically, the intra-modality semantic disentanglement is performed to investigate the more intrinsic semantic cues of the same modality. Additionally, the inter-modality semantic disentanglement is leveraged to simultaneously facilitate the common and intrinsic semantic cues across modalities. Furthermore, the dual-spatial semantic interaction block is presented to explore the long-range cross-spatial semantic context between the obtained verbal and non-verbal semantic space with the global view. The above semantic disentanglement processes with both local and global views significantly unleash much more robustness even for the sarcasm case consisting of multiple semantic message. Various experiments indicate that the FSDN can receive state-of-the-art or competitive performance.<\/jats:p>","DOI":"10.1145\/3722558","type":"journal-article","created":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T17:44:11Z","timestamp":1742319851000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Fine-grained Semantic Disentanglement Network for Multimodal Sarcasm Analysis"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6120-833X","authenticated-orcid":false,"given":"Jiajia","family":"Tang","sequence":"first","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3762-9486","authenticated-orcid":false,"given":"Binbin","family":"Ni","sequence":"additional","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5823-1376","authenticated-orcid":false,"given":"Feiwei","family":"Zhou","sequence":"additional","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9364-9228","authenticated-orcid":false,"given":"Dongjun","family":"Liu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1834-4429","authenticated-orcid":false,"given":"Yu","family":"Ding","sequence":"additional","affiliation":[{"name":"Happy Elements, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1208-972X","authenticated-orcid":false,"given":"Yong","family":"Peng","sequence":"additional","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8364-7226","authenticated-orcid":false,"given":"Andrzej","family":"Cichocki","sequence":"additional","affiliation":[{"name":"Systems Research Institute of Polish Academy of Science, Warsaw, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4442-3182","authenticated-orcid":false,"given":"Qibin","family":"Zhao","sequence":"additional","affiliation":[{"name":"RIKEN Center for Advanced Intelligence Project (AIP), Tokyo, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0113-6968","authenticated-orcid":false,"given":"Wanzeng","family":"Kong","sequence":"additional","affiliation":[{"name":"Key Laboratory of Brain Machine Collaborative Intelligence of Zhejiang Province, Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,7]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.20"},{"key":"e_1_3_1_3_2","first-page":"574","volume-title":"Proceedings of the 9th International AAAI Conference on Web and Social Media","author":"Bamman David","year":"2021","unstructured":"David Bamman and Noah Smith. 2021. Contextualized sarcasm detection on Twitter. In Proceedings of the 9th International AAAI Conference on Web and Social Media, 574\u2013577."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1239"},{"key":"e_1_3_1_5_2","doi-asserted-by":"crossref","first-page":"4619","DOI":"10.18653\/v1\/P19-1455","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Castro Santiago","year":"2019","unstructured":"Santiago Castro, Devamanyu Hazarika, Ver\u00f3nica P\u00e9rez-Rosas, Roger Zimmermann, Rada Mihalcea, and Soujanya Poria. 2019. Towards multimodal sarcasm detection (an _obviously_ perfect paper). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 4619\u20134629."},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2019.2956957"},{"key":"e_1_3_1_7_2","first-page":"4351","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Singh Chauhan Dushyant","year":"2020","unstructured":"Dushyant Singh Chauhan, S. R. Dhanush, Asif Ekbal, and Pushpak Bhattacharyya. 2020. Sentiment and emotion help sarcasm? A multi-task learning framework for multi-modal sarcasm, sentiment and emotion analysis. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 4351\u20134360."},{"key":"e_1_3_1_8_2","first-page":"10563","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence","author":"Desai Poorav","year":"2022","unstructured":"Poorav Desai, Tanmoy Chakraborty, and Md Shad Akhtar. 2022. Nice perfume. How long did you marinate in it? Multimodal Sarcasm Explanation. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, 10563\u201310571."},{"key":"e_1_3_1_9_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, 4171\u20134186."},{"key":"e_1_3_1_10_2","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929. Retrieved from https:\/\/arxiv.org\/abs\/2010.11929"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3252270"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.723"},{"key":"e_1_3_1_13_2","first-page":"12972","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence","author":"Hasan Md Kamrul","year":"2021","unstructured":"Md Kamrul Hasan, Sangwu Lee, Wasifur Rahman, Amir Zadeh, Rada Mihalcea, Louis-Philippe Morency, and Ehsan Hoque. 2021. Humor knowledge enriched transformer for understanding multimodal humor. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, 12972\u201312980."},{"key":"e_1_3_1_14_2","doi-asserted-by":"crossref","unstructured":"Md Kamrul Hasan Wasifur Rahman Amir Zadeh Jianyuan Zhong Md Iftekhar Tanveer Louis-Philippe Morency Mohammed (Ehsan)Hoque. 2019. UR-FUNNY: A multimodal language dataset for understanding humor. arXiv:1904.06618. Retrieved from https:\/\/arxiv.org\/abs\/1904.06618","DOI":"10.18653\/v1\/D19-1211"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413678"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3388861"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.109977"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01925"},{"key":"e_1_3_1_20_2","first-page":"7136","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Li Ziming","year":"2022","unstructured":"Ziming Li, Yan Zhou, Weibo Zhang, Yaxin Liu, Chuanpeng Yang, Zheng Lian, and Songlin Hu. 2022. AMOA: Global acoustic feature enhanced modal-order-aware network for multimodal sentiment analysis. In Proceedings of the 29th International Conference on Computational Linguistics, 7136\u20137146."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474085.3475190"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2021.107643"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.333"},{"key":"e_1_3_1_24_2","first-page":"871","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (EMNLP \u201921","author":"Liu Yaochen","year":"2021","unstructured":"Yaochen Liu, Yazhou Zhang, Qiuchi Li, Benyou Wang, and Dawei Song. 2021. What does your smile mean? Jointly detecting multi-modal sarcasm and sentiment using quantum probability. In Proceedings of the Findings of the Association for Computational Linguistics (EMNLP \u201921), 871\u2013880."},{"key":"e_1_3_1_25_2","doi-asserted-by":"crossref","unstructured":"Zhun Liu Ying Shen Varun Bharadhwaj Lakshminarasimhan Paul Pu Liang Amir Zadeh and Louis-Philippe Morency. 2018. Efficient low-rank multimodal fusion with modality-specific factors. arXiv:1806.00064. Retrieved from https:\/\/arxiv.org\/abs\/1806.00064","DOI":"10.18653\/v1\/P18-1209"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-024-02000-2"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671720"},{"key":"e_1_3_1_28_2","first-page":"1383","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics: (EMNLP \u201920)","author":"Pan Hongliang","year":"2020","unstructured":"Hongliang Pan, Zheng Lin, Peng Fu, Yatao Qi, and Weiping Wang. 2020. Modeling intra and inter-modality incongruity for multi-modal sarcasm detection. In Proceedings of the Findings of the Association for Computational Linguistics: (EMNLP \u201920), 1383\u20131392."},{"key":"e_1_3_1_29_2","first-page":"1601","volume-title":"Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING \u201916)","author":"Poria Soujanya","year":"2016","unstructured":"Soujanya Poria, Erik Cambria, Devamanyu Hazarika, and Prateek Vij. 2016. A deeper look into sarcastic tweets using deep convolutional neural networks. In Proceedings of the 26th International Conference on Computational Linguistics: Technical Papers (COLING \u201916), 1601\u20131612."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00062"},{"key":"e_1_3_1_31_2","first-page":"2359","volume-title":"Proceedings of the 2020 Conference on Association for Computational Linguistics, Meeting","author":"Rahman Wasifur","year":"2020","unstructured":"Wasifur Rahman, Md Kamrul Hasan, Sangwu Lee, Amir Zadeh, Chengfeng Mao, Louis-Philippe Morency, and Ehsan Hoque. 2020. Integrating multimodal information in large pretrained transformers. In Proceedings of the 2020 Conference on Association for Computational Linguistics, Meeting. NIH Public Access, 2359."},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3326335"},{"key":"e_1_3_1_33_2","first-page":"8992","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence","author":"Sun Zhongkai","year":"2020","unstructured":"Zhongkai Sun, Prathusha Sarma, William Sethares, and Yingyu Liang. 2020. Learning relationships between text, audio, and video via deep canonical correlation for multimodal language analysis. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, 8992\u20138999."},{"key":"e_1_3_1_34_2","first-page":"3458","volume-title":"Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI \u201922)","author":"Tang Jiajia","year":"2022","unstructured":"Jiajia Tang, Kang Li, Ming Hou, Xuanyu Jin, Wanzeng Kong, Yu Ding, and Qibin Zhao. 2022. MMT: Multi-way multi-modal transformer for multimodal learning. In Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI \u201922), 3458\u20133465."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIM.2024.3480232"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1093"},{"key":"e_1_3_1_37_2","first-page":"6558","volume-title":"Proceedings of the 2019 Conference on Association for Computational Linguistics, Meeting","author":"Tsai Yao-Hung Hubert","year":"2019","unstructured":"Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J. Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 2019 Conference on Association for Computational Linguistics, Meeting. NIH Public Access, 6558."},{"key":"e_1_3_1_38_2","unstructured":"Yao-Hung Hubert Tsai Paul Pu Liang Amir Zadeh Louis-Philippe Morency and Ruslan Salakhutdinov. 2018. Learning factorized multimodal representations. arXiv:1806.06176. Retrieved from https:\/\/arxiv.org\/abs\/1806.06176"},{"key":"e_1_3_1_39_2","first-page":"2017","article-title":"Attention is all you need. Advances in neural information processing systems","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems. Advances in Neural Information Processing Systems 30 (2017), 2017.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_40_2","unstructured":"Liang Wang Nan Yang Xiaolong Huang Linjun Yang Rangan Majumder and Furu Wei. 2024. Multilingual e5 text embeddings: A technical report. arXiv:2402.05672. Retrieved from https:\/\/arxiv.org\/abs\/2402.05672"},{"key":"e_1_3_1_41_2","doi-asserted-by":"crossref","first-page":"2115","DOI":"10.1145\/3308558.3313735","volume-title":"Proceedings of the World Wide Web Conference (WWW \u201919)","author":"Xiong Tao","year":"2019","unstructured":"Tao Xiong, Peiran Zhang, Hongbo Zhu, and Yihui Yang. 2019. Sarcasm detection with self-matching networks and low-rank bilinear pooling. In Proceedings of the World Wide Web Conference (WWW \u201919), 2115\u20132124."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.349"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413690"},{"key":"e_1_3_1_44_2","first-page":"10790","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence","author":"Yu Wenmeng","year":"2021","unstructured":"Wenmeng Yu, Hua Xu, Ziqi Yuan, and Jiele Wu. 2021. Learning modality-specific representations with self-supervised multi-task learning for multimodal sentiment analysis. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, 10790\u201310797."},{"key":"e_1_3_1_45_2","doi-asserted-by":"crossref","unstructured":"Amir Zadeh Minghai Chen Soujanya Poria Erik Cambria and Louis-Philippe Morency. 2017. Tensor fusion network for multimodal sentiment analysis. arXiv:1707.07250. Retrieved from https:\/\/arxiv.org\/abs\/1707.07250","DOI":"10.18653\/v1\/D17-1115"},{"key":"e_1_3_1_46_2","first-page":"5634","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","volume":"32","author":"Zadeh Amir","year":"2018","unstructured":"Amir Zadeh, Paul Pu Liang, Navonil Mazumder, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018. Memory fusion network for multi-view sequential learning. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, Vol. 32, No. 1, 5634\u20135641."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2016.94"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/TFUZZ.2021.3072492"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383184"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3680566"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3722558","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T14:48:36Z","timestamp":1751381316000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3722558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3722558"],"URL":"https:\/\/doi.org\/10.1145\/3722558","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"assertion":[{"value":"2024-06-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-28","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}