{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T02:01:19Z","timestamp":1776132079498,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":59,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,10,29]],"date-time":"2023-10-29T00:00:00Z","timestamp":1698537600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,10,29]]},"DOI":"10.1145\/3607865.3613184","type":"proceedings-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T18:12:36Z","timestamp":1697566356000},"page":"21-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Learning Aligned Audiovisual Representations for Multimodal Sentiment Analysis"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-0161-4838","authenticated-orcid":false,"given":"Chaoyue","family":"Ding","sequence":"first","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8109-2943","authenticated-orcid":false,"given":"Daoming","family":"Zong","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-4490-2157","authenticated-orcid":false,"given":"Baoxiang","family":"Li","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5856-9969","authenticated-orcid":false,"given":"Ken","family":"Zheng","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-8519-4630","authenticated-orcid":false,"given":"Dinghao","family":"Zhou","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-2492-3528","authenticated-orcid":false,"given":"Jiakui","family":"Li","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0746-9249","authenticated-orcid":false,"given":"Qunyan","family":"Zhou","sequence":"additional","affiliation":[{"name":"SenseTime Group Limited, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2023,10,29]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"24206","article-title":"Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text","volume":"34","author":"Akbari Hassan","year":"2021","unstructured":"Hassan Akbari , Liangzhe Yuan , Rui Qian , Wei-Hong Chuang , Shih-Fu Chang , Yin Cui , and Boqing Gong . 2021 . Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text . NeurIPS , Vol. 34 (2021), 24206 -- 24221 . Hassan Akbari, Liangzhe Yuan, Rui Qian, Wei-Hong Chuang, Shih-Fu Chang, Yin Cui, and Boqing Gong. 2021. Vatt: Transformers for multimodal self-supervised learning from raw video, audio and text. NeurIPS , Vol. 34 (2021), 24206--24221.","journal-title":"NeurIPS"},{"key":"e_1_3_2_1_2_1","first-page":"25","article-title":"Self-supervised multimodal versatile networks","volume":"33","author":"Alayrac Jean-Baptiste","year":"2020","unstructured":"Jean-Baptiste Alayrac , Adria Recasens , Rosalia Schneider , Relja Arandjelovi ć, Jason Ramapuram , Jeffrey De Fauw , Lucas Smaira , Sander Dieleman , and Andrew Zisserman . 2020 . Self-supervised multimodal versatile networks . NeurIPS , Vol. 33 (2020), 25 -- 37 . Jean-Baptiste Alayrac, Adria Recasens, Rosalia Schneider, Relja Arandjelović, Jason Ramapuram, Jeffrey De Fauw, Lucas Smaira, Sander Dieleman, and Andrew Zisserman. 2020. Self-supervised multimodal versatile networks. NeurIPS , Vol. 33 (2020), 25--37.","journal-title":"NeurIPS"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Shahin Amiriparian Maurice Gerczuk Sandra Ottl Nicholas Cummins Michael Freitag Sergey Pugachevskiy and Bj\u00f6rn Schuller. 2017. Snore Sound Classification Using Image-based Deep Spectrum Features. In Interspeech. 3512--3516.  Shahin Amiriparian Maurice Gerczuk Sandra Ottl Nicholas Cummins Michael Freitag Sergey Pugachevskiy and Bj\u00f6rn Schuller. 2017. Snore Sound Classification Using Image-based Deep Spectrum Features. In Interspeech. 3512--3516.","DOI":"10.21437\/Interspeech.2017-434"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Relja Arandjelovic and Andrew Zisserman. 2017. Look listen and learn. In ICCV. 609--617.  Relja Arandjelovic and Andrew Zisserman. 2017. Look listen and learn. In ICCV. 609--617.","DOI":"10.1109\/ICCV.2017.73"},{"key":"e_1_3_2_1_5_1","first-page":"12449","article-title":"wav2vec 2.0: A framework for self-supervised learning of speech representations","volume":"33","author":"Baevski Alexei","year":"2020","unstructured":"Alexei Baevski , Yuhao Zhou , Abdelrahman Mohamed , and Michael Auli . 2020 . wav2vec 2.0: A framework for self-supervised learning of speech representations . NeurIPS , Vol. 33 (2020), 12449 -- 12460 . Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. wav2vec 2.0: A framework for self-supervised learning of speech representations. NeurIPS , Vol. 33 (2020), 12449--12460.","journal-title":"NeurIPS"},{"key":"e_1_3_2_1_6_1","first-page":"423","article-title":"Multimodal machine learning: A survey and taxonomy","volume":"41","author":"Tadas Baltruvs","year":"2018","unstructured":"Tadas Baltruvs aitis, Chaitanya Ahuja , and Louis-Philippe Morency . 2018 . Multimodal machine learning: A survey and taxonomy . IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 41 (2018), 423 -- 443 . Tadas Baltruvs aitis, Chaitanya Ahuja, and Louis-Philippe Morency. 2018. Multimodal machine learning: A survey and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence , Vol. 41 (2018), 423--443.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Mathilde Caron Hugo Touvron Ishan Misra Herv\u00e9 J\u00e9gou Julien Mairal Piotr Bojanowski and Armand Joulin. 2021. Emerging properties in self-supervised vision transformers. In ICCV. 9650--9660.  Mathilde Caron Hugo Touvron Ishan Misra Herv\u00e9 J\u00e9gou Julien Mairal Piotr Bojanowski and Armand Joulin. 2021. Emerging properties in self-supervised vision transformers. In ICCV. 9650--9660.","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Purnima Chandrasekar Santosh Chapaneri and Deepak Jayaswal. 2014. Automatic speech emotion recognition: A survey. In CSCITA. 341--346.  Purnima Chandrasekar Santosh Chapaneri and Deepak Jayaswal. 2014. Automatic speech emotion recognition: A survey. In CSCITA. 341--346.","DOI":"10.1109\/CSCITA.2014.6839284"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.3390\/asi5040080"},{"key":"e_1_3_2_1_10_1","unstructured":"Ting Chen Simon Kornblith Mohammad Norouzi and Geoffrey Hinton. 2020b. A simple framework for contrastive learning of visual representations. In ICML. 1597--1607.  Ting Chen Simon Kornblith Mohammad Norouzi and Geoffrey Hinton. 2020b. A simple framework for contrastive learning of visual representations. In ICML. 1597--1607."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.664274"},{"key":"e_1_3_2_1_12_1","volume-title":"Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297","author":"Chen Xinlei","year":"2020","unstructured":"Xinlei Chen , Haoqi Fan , Ross Girshick , and Kaiming He. 2020a. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 ( 2020 ). Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. 2020a. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Lukas Christ Shahin Amiriparian Alice Baird Alexander Kathan Niklas M\u00fcller Steffen Klug Chris Gagne Panagiotis Tzirakis Eva-Maria Me\u00dfner Andreas K\u00f6nig etal 2023. The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions Cross-Cultural Humour and Personalisation. arXiv preprint arXiv:2305.03369 (2023).  Lukas Christ Shahin Amiriparian Alice Baird Alexander Kathan Niklas M\u00fcller Steffen Klug Chris Gagne Panagiotis Tzirakis Eva-Maria Me\u00dfner Andreas K\u00f6nig et al. 2023. The MuSe 2023 Multimodal Sentiment Analysis Challenge: Mimicked Emotions Cross-Cultural Humour and Personalisation. arXiv preprint arXiv:2305.03369 (2023).","DOI":"10.1145\/3606039.3613114"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551876.3554817"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.58"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Chaoyue Ding Jiakui Li Martin Zong and Baoxiang Li. 2023. Speed-Robust Keyword Spotting Via Soft Self-Attention on Multi-Scale Features. In SLT. 1104--1111.  Chaoyue Ding Jiakui Li Martin Zong and Baoxiang Li. 2023. Speed-Robust Keyword Spotting Via Soft Self-Attention on Multi-Scale Features. In SLT. 1104--1111.","DOI":"10.1109\/SLT54892.2023.10023254"},{"key":"e_1_3_2_1_17_1","volume-title":"LETR: A lightweight and efficient transformer for keyword spotting. In ICASSP. 7987--7991.","author":"Ding Kevin","year":"2022","unstructured":"Kevin Ding , Martin Zong , Jiakui Li , and Baoxiang Li . 2022 . LETR: A lightweight and efficient transformer for keyword spotting. In ICASSP. 7987--7991. Kevin Ding, Martin Zong, Jiakui Li, and Baoxiang Li. 2022. LETR: A lightweight and efficient transformer for keyword spotting. In ICASSP. 7987--7991."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Carl Doersch Abhinav Gupta and Alexei A Efros. 2015. Unsupervised visual representation learning by context prediction. In ICCV. 1422--1430.  Carl Doersch Abhinav Gupta and Alexei A Efros. 2015. Unsupervised visual representation learning by context prediction. In ICCV. 1422--1430.","DOI":"10.1109\/ICCV.2015.167"},{"key":"e_1_3_2_1_19_1","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly etal 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).  Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)."},{"key":"e_1_3_2_1_20_1","volume-title":"Facial action coding system. Environmental Psychology & Nonverbal Behavior","author":"Ekman Paul","year":"1978","unstructured":"Paul Ekman and Wallace V Friesen . 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior ( 1978 ). Paul Ekman and Wallace V Friesen. 1978. Facial action coding system. Environmental Psychology & Nonverbal Behavior (1978)."},{"key":"e_1_3_2_1_21_1","volume-title":"Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separation. arXiv preprint arXiv:1804.03619","author":"Ephrat Ariel","year":"2018","unstructured":"Ariel Ephrat , Inbar Mosseri , Oran Lang , Tali Dekel , Kevin Wilson , Avinatan Hassidim , William T Freeman , and Michael Rubinstein . 2018. Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separation. arXiv preprint arXiv:1804.03619 ( 2018 ). Ariel Ephrat, Inbar Mosseri, Oran Lang, Tali Dekel, Kevin Wilson, Avinatan Hassidim, William T Freeman, and Michael Rubinstein. 2018. Looking to listen at the cocktail party: A speaker-independent audio-visual model for speech separation. arXiv preprint arXiv:1804.03619 (2018)."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2015.2457417"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Florian Eyben Martin W\u00f6llmer and Bj\u00f6rn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In ACM Multimedia. 1459--1462.  Florian Eyben Martin W\u00f6llmer and Bj\u00f6rn Schuller. 2010. Opensmile: the munich versatile and fast open-source audio feature extractor. In ACM Multimedia. 1459--1462.","DOI":"10.1145\/1873951.1874246"},{"key":"e_1_3_2_1_24_1","volume-title":"Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821","author":"Gao Tianyu","year":"2021","unstructured":"Tianyu Gao , Xingcheng Yao , and Danqi Chen . 2021 . Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021). Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. Simcse: Simple contrastive learning of sentence embeddings. arXiv preprint arXiv:2104.08821 (2021)."},{"key":"e_1_3_2_1_25_1","volume-title":"Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al.","author":"Goodfellow Ian J","year":"2013","unstructured":"Ian J Goodfellow , Dumitru Erhan , Pierre Luc Carrier , Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al. 2013 . Challenges in representation learning: A report on three machine learning contests. In Neural Information Processing . 117--124. Ian J Goodfellow, Dumitru Erhan, Pierre Luc Carrier, Aaron Courville, Mehdi Mirza, Ben Hamner, Will Cukierski, Yichuan Tang, David Thaler, Dong-Hyun Lee, et al. 2013. Challenges in representation learning: A report on three machine learning contests. In Neural Information Processing. 117--124."},{"key":"e_1_3_2_1_26_1","unstructured":"Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Doll\u00e1r and Ross Girshick. 2022a. Masked autoencoders are scalable vision learners. In CVPR. 16000--16009.  Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Doll\u00e1r and Ross Girshick. 2022a. Masked autoencoders are scalable vision learners. In CVPR. 16000--16009."},{"key":"e_1_3_2_1_27_1","unstructured":"Kaiming He Haoqi Fan Yuxin Wu Saining Xie and Ross Girshick. 2020a. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729--9738.  Kaiming He Haoqi Fan Yuxin Wu Saining Xie and Ross Girshick. 2020a. Momentum contrast for unsupervised visual representation learning. In CVPR. 9729--9738."},{"key":"e_1_3_2_1_28_1","unstructured":"Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778.  Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. 770--778."},{"key":"e_1_3_2_1_29_1","volume-title":"Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654","author":"He Pengcheng","year":"2020","unstructured":"Pengcheng He , Xiaodong Liu , Jianfeng Gao , and Weizhu Chen . 2020 b. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 (2020). Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020b. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 (2020)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551876.3554811"},{"key":"e_1_3_2_1_31_1","volume-title":"Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al.","author":"Hershey Shawn","year":"2017","unstructured":"Shawn Hershey , Sourish Chaudhuri , Daniel PW Ellis , Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. 2017 . CNN architectures for large-scale audio classification. In ICASSP. 131--135. Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et al. 2017. CNN architectures for large-scale audio classification. In ICASSP. 131--135."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TASLP.2021.3122291"},{"key":"e_1_3_2_1_33_1","unstructured":"Yu Huang Junyang Lin Chang Zhou Hongxia Yang and Longbo Huang. 2022. Modality competition: What makes joint training of multi-modal network fail in deep learning?(provably). In ICML. 9226--9259.  Yu Huang Junyang Lin Chang Zhou Hongxia Yang and Longbo Huang. 2022. Modality competition: What makes joint training of multi-modal network fail in deep learning?(provably). In ICML. 9226--9259."},{"key":"e_1_3_2_1_34_1","volume-title":"Epic-fusion: Audio-visual temporal binding for egocentric action recognition. In ICCV. 5492--5501.","author":"Kazakos Evangelos","year":"2019","unstructured":"Evangelos Kazakos , Arsha Nagrani , Andrew Zisserman , and Dima Damen . 2019 . Epic-fusion: Audio-visual temporal binding for egocentric action recognition. In ICCV. 5492--5501. Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, and Dima Damen. 2019. Epic-fusion: Audio-visual temporal binding for egocentric action recognition. In ICCV. 5492--5501."},{"key":"e_1_3_2_1_35_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186.","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT. 4171--4186."},{"key":"e_1_3_2_1_36_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_3_2_1_37_1","volume-title":"Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942","author":"Lan Zhenzhong","year":"2019","unstructured":"Zhenzhong Lan , Mingda Chen , Sebastian Goodman , Kevin Gimpel , Piyush Sharma , and Radu Soricut . 2019 . Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019). Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)."},{"key":"e_1_3_2_1_38_1","first-page":"9694","article-title":"Align before fuse: Vision and language representation learning with momentum distillation","volume":"34","author":"Li Junnan","year":"2021","unstructured":"Junnan Li , Ramprasaath Selvaraju , Akhilesh Gotmare , Shafiq Joty , Caiming Xiong , and Steven Chu Hong Hoi . 2021 . Align before fuse: Vision and language representation learning with momentum distillation . NeurIPS , Vol. 34 (2021), 9694 -- 9705 . Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. NeurIPS , Vol. 34 (2021), 9694--9705.","journal-title":"NeurIPS"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551876.3554809"},{"key":"e_1_3_2_1_40_1","unstructured":"Shan Li Weihong Deng and JunPing Du. 2017. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In CVPR. 2852--2861.  Shan Li Weihong Deng and JunPing Du. 2017. Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. In CVPR. 2852--2861."},{"key":"e_1_3_2_1_41_1","volume-title":"MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning. arXiv preprint arXiv:2304","author":"Lian Zheng","year":"2023","unstructured":"Zheng Lian , Haiyang Sun , Licai Sun , Jinming Zhao , Ye Liu , Bin Liu , Jiangyan Yi , Meng Wang , Erik Cambria , Guoying Zhao , 2023 . MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning. arXiv preprint arXiv:2304 .08981 (2023). Zheng Lian, Haiyang Sun, Licai Sun, Jinming Zhao, Ye Liu, Bin Liu, Jiangyan Yi, Meng Wang, Erik Cambria, Guoying Zhao, et al. 2023. MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning. arXiv preprint arXiv:2304.08981 (2023)."},{"key":"e_1_3_2_1_42_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019 . Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_3_2_1_43_1","unstructured":"Jiquan Ngiam Aditya Khosla Mingyu Kim Juhan Nam Honglak Lee and Andrew Y Ng. 2011. Multimodal deep learning. In ICML. 689--696.  Jiquan Ngiam Aditya Khosla Mingyu Kim Juhan Nam Honglak Lee and Andrew Y Ng. 2011. Multimodal deep learning. In ICML. 689--696."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Mehdi Noroozi and Paolo Favaro. 2016. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV. 69--84.  Mehdi Noroozi and Paolo Favaro. 2016. Unsupervised learning of visual representations by solving jigsaw puzzles. In ECCV. 69--84.","DOI":"10.1007\/978-3-319-46466-4_5"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Andrew Owens and Alexei A Efros. 2018. Audio-visual scene analysis with self-supervised multisensory features. In ECCV. 631--648.  Andrew Owens and Alexei A Efros. 2018. Audio-visual scene analysis with self-supervised multisensory features. In ECCV. 631--648.","DOI":"10.1007\/978-3-030-01231-1_39"},{"key":"e_1_3_2_1_46_1","volume-title":"Glove: Global vectors for word representation. In EMNLP. 1532--1543.","author":"Pennington Jeffrey","year":"2014","unstructured":"Jeffrey Pennington , Richard Socher , and Christopher D Manning . 2014 . Glove: Global vectors for word representation. In EMNLP. 1532--1543. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In EMNLP. 1532--1543."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11431-020-1647-3"},{"key":"e_1_3_2_1_48_1","volume-title":"wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862","author":"Schneider Steffen","year":"2019","unstructured":"Steffen Schneider , Alexei Baevski , Ronan Collobert , and Michael Auli . 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862 ( 2019 ). Steffen Schneider, Alexei Baevski, Ronan Collobert, and Michael Auli. 2019. wav2vec: Unsupervised pre-training for speech recognition. arXiv preprint arXiv:1904.05862 (2019)."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Sefik Ilkin Serengil and Alper Ozpinar. 2020. LightFace: A Hybrid Deep Face Recognition Framework. In ASYU. 23--27.  Sefik Ilkin Serengil and Alper Ozpinar. 2020. LightFace: A Hybrid Deep Face Recognition Framework. In ASYU. 23--27.","DOI":"10.1109\/ASYU50717.2020.9259802"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3475957.3484450"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3423327.3423673"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551876.3554806"},{"key":"e_1_3_2_1_54_1","volume-title":"NeurIPS","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , \u0141ukasz Kaiser , and Illia Polosukhin . 2017 . Attention is all you need . NeurIPS , Vol. 30 (2017). Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. NeurIPS , Vol. 30 (2017)."},{"key":"e_1_3_2_1_55_1","volume-title":"Kristen Grauman, Jitendra Malik, and Christoph Feichtenhofer.","author":"Xiao Fanyi","year":"2020","unstructured":"Fanyi Xiao , Yong Jae Lee , Kristen Grauman, Jitendra Malik, and Christoph Feichtenhofer. 2020 . Audiovisual slowfast networks for video recognition. arXiv preprint arXiv:2001.08740 (2020). Fanyi Xiao, Yong Jae Lee, Kristen Grauman, Jitendra Malik, and Christoph Feichtenhofer. 2020. Audiovisual slowfast networks for video recognition. arXiv preprint arXiv:2001.08740 (2020)."},{"key":"e_1_3_2_1_56_1","volume-title":"NeurIPS","volume":"32","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le . 2019 . Xlnet: Generalized autoregressive pretraining for language understanding . NeurIPS , Vol. 32 (2019). Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. NeurIPS , Vol. 32 (2019)."},{"key":"e_1_3_2_1_57_1","volume-title":"Wenetspeech: A 10000 hours multi-domain mandarin corpus for speech recognition. In ICASSP. 6182--6186.","author":"Zhang Binbin","year":"2022","unstructured":"Binbin Zhang , Hang Lv , Pengcheng Guo , Qijie Shao , Chao Yang , Lei Xie , Xin Xu , Hui Bu , Xiaoyu Chen , Chenchen Zeng , 2022 . Wenetspeech: A 10000 hours multi-domain mandarin corpus for speech recognition. In ICASSP. 6182--6186. Binbin Zhang, Hang Lv, Pengcheng Guo, Qijie Shao, Chao Yang, Lei Xie, Xin Xu, Hui Bu, Xiaoyu Chen, Chenchen Zeng, et al. 2022. Wenetspeech: A 10000 hours multi-domain mandarin corpus for speech recognition. In ICASSP. 6182--6186."},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/LSP.2016.2603342"},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3093397"}],"event":{"name":"MM '23: The 31st ACM International Conference on Multimedia","location":"Ottawa ON Canada","acronym":"MM '23","sponsor":["SIGMM ACM Special Interest Group on Multimedia"]},"container-title":["Proceedings of the 1st International Workshop on Multimodal and Responsible Affective Computing"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607865.3613184","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3607865.3613184","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:06Z","timestamp":1750178226000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3607865.3613184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,29]]},"references-count":59,"alternative-id":["10.1145\/3607865.3613184","10.1145\/3607865"],"URL":"https:\/\/doi.org\/10.1145\/3607865.3613184","relation":{},"subject":[],"published":{"date-parts":[[2023,10,29]]},"assertion":[{"value":"2023-10-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}