{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T14:10:17Z","timestamp":1774534217444,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":48,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,18]],"date-time":"2021-10-18T00:00:00Z","timestamp":1634515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"MOE Tier 1","award":["T1SRIS19149"],"award-info":[{"award-number":["T1SRIS19149"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,18]]},"DOI":"10.1145\/3462244.3479919","type":"proceedings-article","created":{"date-parts":[[2021,10,15]],"date-time":"2021-10-15T15:01:58Z","timestamp":1634310118000},"page":"6-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":191,"title":["Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis"],"prefix":"10.1145","author":[{"given":"Wei","family":"Han","sequence":"first","affiliation":[{"name":"Information Systems Technology and Design, Singapore University of Technology and Design, Singapore"}]},{"given":"Hui","family":"Chen","sequence":"additional","affiliation":[{"name":"Information Systems Technology and Design, Singapore University of Technology and Design, Singapore"}]},{"given":"Alexander","family":"Gelbukh","sequence":"additional","affiliation":[{"name":"Centro de Investigaci\u00f3n en Computaci\u00f3n, Instituto Polit\u00e9cnico Nacional, Mexico"}]},{"given":"Amir","family":"Zadeh","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University, USA"}]},{"given":"Louis-philippe","family":"Morency","sequence":"additional","affiliation":[{"name":"Language Technologies Institute, Carnegie Mellon University, USA"}]},{"given":"Soujanya","family":"Poria","sequence":"additional","affiliation":[{"name":"Singapore University of Technology and Design, Singapore"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1034"},{"key":"e_1_3_2_1_2_1","volume-title":"Multimodal fusion for multimedia analysis: a survey. Multimedia systems 16, 6","author":"Atrey K","year":"2010","unstructured":"Pradeep\u00a0 K Atrey , M\u00a0Anwar Hossain , Abdulmotaleb El\u00a0Saddik , and Mohan\u00a0 S Kankanhalli . 2010. Multimodal fusion for multimedia analysis: a survey. Multimedia systems 16, 6 ( 2010 ), 345\u2013379. Pradeep\u00a0K Atrey, M\u00a0Anwar Hossain, Abdulmotaleb El\u00a0Saddik, and Mohan\u00a0S Kankanhalli. 2010. Multimodal fusion for multimedia analysis: a survey. Multimedia systems 16, 6 (2010), 345\u2013379."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2016.7477553"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.18608\/jla.2016.32.11"},{"key":"e_1_3_2_1_5_1","unstructured":"Feiyang Chen Ziqian Luo Yanyan Xu and Dengfeng Ke. 2019. Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis. arXiv preprint arXiv:1904.08138(2019). http:\/\/ceur-ws.org\/Vol-2614\/AffCon20_session1_complementary.pdf  Feiyang Chen Ziqian Luo Yanyan Xu and Dengfeng Ke. 2019. Complementary Fusion of Multi-Features and Multi-Modalities in Sentiment Analysis. arXiv preprint arXiv:1904.08138(2019). http:\/\/ceur-ws.org\/Vol-2614\/AffCon20_session1_complementary.pdf"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3136801"},{"key":"e_1_3_2_1_7_1","volume-title":"NIPS 2014 Workshop on Deep Learning","author":"Chung Junyoung","year":"2014","unstructured":"Junyoung Chung , Caglar Gulcehre , Kyunghyun Cho , and Yoshua Bengio . 2014 . Empirical evaluation of gated recurrent neural networks on sequence modeling . In NIPS 2014 Workshop on Deep Learning , December 2014. https:\/\/arxiv.org\/pdf\/1412.3555.pdf Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. In NIPS 2014 Workshop on Deep Learning, December 2014. https:\/\/arxiv.org\/pdf\/1412.3555.pdf"},{"key":"e_1_3_2_1_8_1","volume-title":"COVAREP\u2014A collaborative voice analysis repository for speech technologies. In 2014 ieee international conference on acoustics, speech and signal processing (icassp)","author":"Degottex Gilles","unstructured":"Gilles Degottex , John Kane , Thomas Drugman , Tuomo Raitio , and Stefan Scherer . 2014. COVAREP\u2014A collaborative voice analysis repository for speech technologies. In 2014 ieee international conference on acoustics, speech and signal processing (icassp) . IEEE , 960\u2013964. https:\/\/ieeexplore.ieee.org\/stamp\/stamp.jsp?tp=&arnumber=6853739 Gilles Degottex, John Kane, Thomas Drugman, Tuomo Raitio, and Stefan Scherer. 2014. COVAREP\u2014A collaborative voice analysis repository for speech technologies. In 2014 ieee international conference on acoustics, speech and signal processing (icassp). IEEE, 960\u2013964. https:\/\/ieeexplore.ieee.org\/stamp\/stamp.jsp?tp=&arnumber=6853739"},{"key":"e_1_3_2_1_9_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). 4171\u20134186. https:\/\/www.aclweb.org\/anthology\/N19-1423.pdf Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171\u20134186. https:\/\/www.aclweb.org\/anthology\/N19-1423.pdf"},{"key":"e_1_3_2_1_10_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929(2020). https:\/\/arxiv.org\/pdf\/2010.11929.pdf  Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929(2020). https:\/\/arxiv.org\/pdf\/2010.11929.pdf"},{"key":"e_1_3_2_1_11_1","volume-title":"What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS)","author":"Ekman Rosenberg","unstructured":"Rosenberg Ekman . 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS) . Oxford University Press , USA. Rosenberg Ekman. 1997. What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Oxford University Press, USA."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1329"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143891"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1207"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1211"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413678"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Po-Yao Huang Junjie Hu Xiaojun Chang and Alexander Hauptmann. 2020. Unsupervised multimodal neural machine translation with pseudo visual pivoting. arXiv preprint arXiv:2005.03119(2020).  Po-Yao Huang Junjie Hu Xiaojun Chang and Alexander Hauptmann. 2020. Unsupervised multimodal neural machine translation with pseudo visual pivoting. arXiv preprint arXiv:2005.03119(2020).","DOI":"10.18653\/v1\/2020.acl-main.731"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1209"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1185"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i02.5492"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1078"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070481.2070509"},{"key":"e_1_3_2_1_23_1","volume-title":"International Conference on Machine Learning (ICML). 689\u2013696","author":"Ngiam Jiquan","year":"2011","unstructured":"Jiquan Ngiam , Aditya Khosla , Mingyu Kim , Juhan Nam , Honglak Lee , and Andrew\u00a0 Y Ng . 2011 . Multimodal deep learning . In International Conference on Machine Learning (ICML). 689\u2013696 . https:\/\/icml.cc\/2011\/papers\/399_icmlpaper.pdf Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew\u00a0Y Ng. 2011. Multimodal deep learning. In International Conference on Machine Learning (ICML). 689\u2013696. https:\/\/icml.cc\/2011\/papers\/399_icmlpaper.pdf"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33016892"},{"key":"e_1_3_2_1_26_1","unstructured":"Ngoc-Quan Pham Jan Niehues Thanh-Le Ha and Alex Waibel. 2019. Improving Zero-shot Translation with Language-Independent Constraints. arXiv preprint arXiv:1906.08584(2019).  Ngoc-Quan Pham Jan Niehues Thanh-Le Ha and Alex Waibel. 2019. Improving Zero-shot Translation with Language-Independent Constraints. arXiv preprint arXiv:1906.08584(2019)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.214"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1180"},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics. 3715\u20133727","author":"Shi Haoyue","year":"2018","unstructured":"Haoyue Shi , Jiayuan Mao , Tete Xiao , Yuning Jiang , and Jian Sun . 2018 . Learning Visually-Grounded Semantics from Contrastive Adversarial Samples . In Proceedings of the 27th International Conference on Computational Linguistics. 3715\u20133727 . https:\/\/www.aclweb.org\/anthology\/C18-1315.pdf Haoyue Shi, Jiayuan Mao, Tete Xiao, Yuning Jiang, and Jian Sun. 2018. Learning Visually-Grounded Semantics from Contrastive Adversarial Samples. In Proceedings of the 27th International Conference on Computational Linguistics. 3715\u20133727. https:\/\/www.aclweb.org\/anthology\/C18-1315.pdf"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W16-2346"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2697059"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6431"},{"key":"e_1_3_2_1_33_1","unstructured":"Zhongkai Sun Prathusha\u00a0K Sarma William Sethares and Erik\u00a0P Bucy. 2019. Multi-modal sentiment analysis using deep canonical correlation analysis. arXiv preprint arXiv:1907.08696(2019).  Zhongkai Sun Prathusha\u00a0K Sarma William Sethares and Erik\u00a0P Bucy. 2019. Multi-modal sentiment analysis using deep canonical correlation analysis. arXiv preprint arXiv:1907.08696(2019)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1656"},{"key":"e_1_3_2_1_35_1","volume-title":"Learning Factorized Multimodal Representations. In International Conference on Representation Learning. https:\/\/openreview.net\/pdf?id=rygqqsA9KX","author":"Tsai Hung\u00a0Hubert","year":"2019","unstructured":"Yao- Hung\u00a0Hubert Tsai , Paul\u00a0Pu Liang , Amir Zadeh , Louis-Philippe Morency , and Ruslan Salakhutdinov . 2019 . Learning Factorized Multimodal Representations. In International Conference on Representation Learning. https:\/\/openreview.net\/pdf?id=rygqqsA9KX Yao-Hung\u00a0Hubert Tsai, Paul\u00a0Pu Liang, Amir Zadeh, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. Learning Factorized Multimodal Representations. In International Conference on Representation Learning. https:\/\/openreview.net\/pdf?id=rygqqsA9KX"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.143"},{"key":"e_1_3_2_1_37_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998\u20136008. https:\/\/papers.nips.cc\/paper\/7181-attention-is-all-you-need.pdf  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan\u00a0N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998\u20136008. https:\/\/papers.nips.cc\/paper\/7181-attention-is-all-you-need.pdf"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICME.2017.8019301"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2013.34"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.400"},{"key":"e_1_3_2_1_41_1","volume-title":"Improving multimodal named entity recognition via entity span detection with unified multimodal transformer","author":"Yu Jianfei","unstructured":"Jianfei Yu , Jing Jiang , Li Yang , and Rui Xia . 2020. Improving multimodal named entity recognition via entity span detection with unified multimodal transformer . Association for Computational Linguistics . Jianfei Yu, Jing Jiang, Li Yang, and Rui Xia. 2020. Improving multimodal named entity recognition via entity span detection with unified multimodal transformer. Association for Computational Linguistics."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1121\/1.2935783"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1115"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12024"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2016.94"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1208"},{"key":"e_1_3_2_1_47_1","volume-title":"Proceedings of the AAAI conference on artificial intelligence, Vol.\u00a032","author":"Zhang Qi","year":"2018","unstructured":"Qi Zhang , Jinlan Fu , Xiaoyu Liu , and Xuanjing Huang . 2018 . Adaptive co-attention network for named entity recognition in tweets . In Proceedings of the AAAI conference on artificial intelligence, Vol.\u00a032 . Issue 1. https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI18\/paper\/view\/16432\/16127 Qi Zhang, Jinlan Fu, Xiaoyu Liu, and Xuanjing Huang. 2018. Adaptive co-attention network for named entity recognition in tweets. In Proceedings of the AAAI conference on artificial intelligence, Vol.\u00a032. Issue 1. https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI18\/paper\/view\/16432\/16127"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.354"}],"event":{"name":"ICMI '21: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION","location":"Montr\u00e9al QC Canada","acronym":"ICMI '21","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2021 International Conference on Multimodal Interaction"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3462244.3479919","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3462244.3479919","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:54Z","timestamp":1750193334000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3462244.3479919"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,18]]},"references-count":48,"alternative-id":["10.1145\/3462244.3479919","10.1145\/3462244"],"URL":"https:\/\/doi.org\/10.1145\/3462244.3479919","relation":{},"subject":[],"published":{"date-parts":[[2021,10,18]]},"assertion":[{"value":"2021-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}