{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T14:26:52Z","timestamp":1783520812457,"version":"3.55.0"},"reference-count":68,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T00:00:00Z","timestamp":1775001600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"},{"start":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T00:00:00Z","timestamp":1775001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.springernature.com\/gp\/researchers\/text-and-data-mining"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach. Intell. Res."],"published-print":{"date-parts":[[2026,4]]},"DOI":"10.1007\/s11633-025-1630-x","type":"journal-article","created":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T10:35:58Z","timestamp":1775644558000},"page":"484-500","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["From Uni- to Multi-modal: Progressive Multi-source Domain Adaptation with Gradient Disentangling for Audio-visual Deception Detection"],"prefix":"10.1007","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4530-4529","authenticated-orcid":false,"given":"Ronghao","family":"Lin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-2301-7897","authenticated-orcid":false,"given":"Aolin","family":"Xiong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2204-8668","authenticated-orcid":false,"given":"Qiaolin","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9763-375X","authenticated-orcid":false,"given":"Sijie","family":"Mai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8396-5243","authenticated-orcid":false,"given":"Li","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4884-323X","authenticated-orcid":false,"given":"Haifeng","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,8]]},"reference":[{"key":"1630_CR1","unstructured":"X. Guo, Z. Yu, N. M. Selvaraj, B. Shen, A. W. K. Kong, A. C. Kot. Benchmarking cross-domain audio-visual deception detection, [Online], Available: https:\/\/arxiv.org\/html\/2405.06995v1, 2024."},{"key":"1630_CR2","doi-asserted-by":"publisher","first-page":"7794","DOI":"10.1109\/CVPR.2019.00799","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"M Ding","year":"2019","unstructured":"M. Ding, A. Zhao, Z. Lu, T. Xiang, J. R. Wen. Face-focused cross-stream network for deception detection in videos. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, USA, pp. 7794\u20137803, 2019. DOI: https:\/\/doi.org\/10.1109\/CVPR.2019.00799."},{"key":"1630_CR3","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1609\/icwsm.v15i1.18065","volume-title":"Proceedings of the 15th International AAAI Conference on Web and Social Media","author":"S Kumar","year":"2021","unstructured":"S. Kumar, C. Bai, V. S. Subrahmanian, J. Leskovec. Deception detection in group video conversations using dynamic interaction networks. In Proceedings of the 15th International AAAI Conference on Web and Social Media, pp. 339\u2013350, 2021. DOI: https:\/\/doi.org\/10.1609\/icwsm.v15i1.18065."},{"key":"1630_CR4","unstructured":"J. T. Yang, G. M. Liu, S. C. H. Huang. Constructing robust emotional state-based feature with a novel voting scheme for multi-modal deception detection in videos, [Online], Available: https:\/\/arxiv.org\/abs\/2104.08373, 2021."},{"issue":"3","key":"1630_CR5","doi-asserted-by":"publisher","first-page":"971","DOI":"10.1109\/TCDS.2021.3086011","volume":"14","author":"M Karnati","year":"2022","unstructured":"M. Karnati, A. Seal, A. Yazidi, O. Krejcar. LieNet: A deep convolution neural network framework for detecting deception. IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 3, pp. 971\u2013984, 2022. DOI: https:\/\/doi.org\/10.1109\/TCDS.2021.3086011.","journal-title":"IEEE Transactions on Cognitive and Developmental Systems"},{"key":"1630_CR6","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1109\/TPSISA50397.2020.00023","volume-title":"Proceedings of the 2nd IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications","author":"V Belavadi","year":"2020","unstructured":"V. Belavadi, Y. Zhou, J. Z. Bakdash, M. Kantarcioglu, D. C. Krawczyk, L. Nguyen, J. Rakic, B. Thuriasingham. Multimodal deception detection: Accuracy, applicability and generalizability. In Proceedings of the 2nd IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications, Atlanta, USA, pp. 99\u2013106, 2020. DOI: https:\/\/doi.org\/10.1109\/TPSISA50397.2020.00023."},{"issue":"1","key":"1630_CR7","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1109\/TAFFC.2020.3015684","volume":"13","author":"M U Sen","year":"2022","unstructured":"M. U. Sen, V. P\u00e9rez-Rosas, B. Yanikoglu, M. Abouelenien, M. Burzo, R. Mihalcea. Multimodal deception detection using real-life trial data. IEEE Transactions on Affective Computing, vol. 13, no. 1, pp. 306\u2013319, 2022. DOI: https:\/\/doi.org\/10.1109\/TAFFC.2020.3015684.","journal-title":"IEEE Transactions on Affective Computing"},{"key":"1630_CR8","doi-asserted-by":"publisher","first-page":"12957","DOI":"10.1145\/3746027.3758242","volume-title":"Proceedings of the 33rd ACM International Conference on Multimedia","author":"C Cai","year":"2025","unstructured":"C. Cai, S. Liang, X. Liu, K. Zhu, Z. Wen, J. Tao, H. Xie, J. Cui, Y. Ma, Z. Cheng, H. Xu, R. Fu, B. Liu, Y. Li. MDPE: A multimodal deception dataset with personality and emotional characteristics. In Proceedings of the 33rd ACM International Conference on Multimedia, Dublin, Ireland, pp. 12957\u201312964, 2025. DOI: https:\/\/doi.org\/10.1145\/3746027.3758242."},{"key":"1630_CR9","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1109\/CVPRW.2019.00016","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"V Gupta","year":"2019","unstructured":"V. Gupta, M. Agarwal, M. Arora, T. Chakraborty, R. Singh, M. Vatsa. Bag-of-lies: A multimodal dataset for deception detection. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, USA, pp. 83\u201390, 2019. DOI: https:\/\/doi.org\/10.1109\/CVPRW.2019.00016."},{"issue":"1","key":"1630_CR10","doi-asserted-by":"publisher","first-page":"429","DOI":"10.3758\/s13428-018-1061-4","volume":"51","author":"E P Lloyd","year":"2019","unstructured":"E. P. Lloyd, J. C. Deska, K. Hugenberg, A. R. McConnell, B. T. Humphrey, J. W. Kunstman. Miami university deception detection database. Behavior Research Methods, vol. 51, no. 1, pp. 429\u2013439, 2019. DOI: https:\/\/doi.org\/10.3758\/s13428-018-1061-4.","journal-title":"Behavior Research Methods"},{"key":"1630_CR11","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1145\/2818346.2820758","volume-title":"Proceedings of ACM on International Conference on Multimodal Interaction","author":"V P\u00e9rez-Rosas","year":"2015","unstructured":"V. P\u00e9rez-Rosas, M. Abouelenien, R. Mihalcea, M. Burzo. Deception detection using real-life trial data. In Proceedings of ACM on International Conference on Multimodal Interaction, Seattle, USA, pp. 59\u201366, 2015. DOI: https:\/\/doi.org\/10.1145\/2818346.2820758."},{"key":"1630_CR12","doi-asserted-by":"publisher","first-page":"1768","DOI":"10.18653\/v1\/N19-1175","volume-title":"Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"F Soldner","year":"2019","unstructured":"F. Soldner, V. P\u00e9rez-Rosas, R. Mihalcea. Box of lies: Multimodal deception detection in dialogues. In Proceedings of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, USA, pp. 1768\u20131777, 2019. DOI: https:\/\/doi.org\/10.18653\/v1\/N19-1175."},{"key":"1630_CR13","unstructured":"S. Zhao, B. Li, C. Reed, P. Xu, K. Keutzer. Multi-source domain adaptation in the deep learning era: A systematic survey, [Online], Available: https:\/\/arxiv.org\/abs\/2002.12169, 2020."},{"issue":"8","key":"1630_CR14","doi-asserted-by":"publisher","first-page":"5743","DOI":"10.1109\/TPAMI.2024.3370978","volume":"46","author":"J Li","year":"2024","unstructured":"J. Li, Z. Yu, Z. Du, L. Zhu, H. T. Shen. A comprehensive survey on source-free domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 8, pp. 5743\u20135762, 2024. DOI: https:\/\/doi.org\/10.1109\/TPAMI.2024.3370978.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1630_CR15","unstructured":"H. Dong, M. Liu, K. Zhou, E. Chatzi, J. Kannala, C. Stachniss, O. Fink. Advances in multimodal adaptation and generalization: From traditional approaches to foundation models, [Online], Available: https:\/\/arxiv.org\/abs\/2501.18592, 2025."},{"key":"1630_CR16","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1145\/3728425.3759924","volume-title":"Proceedings of the 1st International Workshop & Challenge on Subtle Visual Computing","author":"R Lin","year":"2025","unstructured":"R. Lin, S. Mai, Y. Zeng, Q. He, A. Xiong, H. Hu. Multisource multimodal progressive domain adaptation for audio-visual deception detection. In Proceedings of the 1st International Workshop & Challenge on Subtle Visual Computing, Dublin, Ireland, pp. 52\u201358, 2025. DOI: https:\/\/doi.org\/10.1145\/3728425.3759924."},{"key":"1630_CR17","doi-asserted-by":"publisher","first-page":"1120","DOI":"10.18653\/v1\/D15-1133","volume-title":"Proceedings of Conference on Empirical Methods in Natural Language Processing","author":"V P\u00e9rez-Rosas","year":"2015","unstructured":"V. P\u00e9rez-Rosas, R. Mihalcea. Experiments in open domain deception detection. In Proceedings of Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1120\u20131125, 2015. DOI: https:\/\/doi.org\/10.18653\/v1\/D15-1133."},{"key":"1630_CR18","doi-asserted-by":"publisher","first-page":"1278","DOI":"10.1109\/BigData.2018.8621909","volume-title":"Proceedings of IEEE International Conference on Big Data","author":"H Karimi","year":"2018","unstructured":"H. Karimi, J. Tang, Y. Li. Toward end-to-end deception detection in videos. In Proceedings of IEEE International Conference on Big Data, Seattle, USA, pp. 1278\u20131283, 2018. DOI: https:\/\/doi.org\/10.1109\/BigData.2018.8621909."},{"key":"1630_CR19","doi-asserted-by":"publisher","first-page":"2336","DOI":"10.18653\/v1\/D15-1281","volume-title":"Proceedings of Conference on Empirical Methods in Natural Language Processing","author":"V P\u00e9rez-Rosas","year":"2015","unstructured":"V. P\u00e9rez-Rosas, M. Abouelenien, R. Mihalcea, Y. Xiao, C. Linton, M. Burzo. Verbal and nonverbal clues for real-life deception detection. In Proceedings of Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 2336\u20132346, 2015. DOI: https:\/\/doi.org\/10.18653\/v1\/D15-1281."},{"key":"1630_CR20","doi-asserted-by":"publisher","first-page":"28098","DOI":"10.1109\/ACCESS.2025.3533545","volume":"13","author":"Y D Rahayu","year":"2025","unstructured":"Y. D. Rahayu, C. Fatichah, A. Yuniarti, Y. P. Rahayu. Advancements and challenges in video-based deception detection: A systematic literature review of datasets, modalities, and methods. IEEE Access, vol. 13, pp. 28098\u201328122, 2025. DOI: https:\/\/doi.org\/10.1109\/ACCESS.2025.3533545.","journal-title":"IEEE Access"},{"issue":"4","key":"1630_CR21","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1109\/TBIOM.2022.3218956","volume":"4","author":"N Vance","year":"2022","unstructured":"N. Vance, J. Speth, S. Khan, A. Czajka, K. W. Bowyer, D. Wright, P. Flynn. Deception detection and remote physiological monitoring: A dataset and baseline experimental results. IEEE Transactions on Biometrics, Behavior, and Identity Science, vol. 4, no. 4, pp. 522\u2013532, 2022. DOI: https:\/\/doi.org\/10.1109\/TBIOM.2022.3218956.","journal-title":"IEEE Transactions on Biometrics, Behavior, and Identity Science"},{"key":"1630_CR22","doi-asserted-by":"publisher","DOI":"10.1109\/SSCI.2017.8285382","volume-title":"Proceedings of IEEE Symposium Series on Computational Intelligence","author":"M Gogate","year":"2017","unstructured":"M. Gogate, A. Adeel, A. Hussain. Deep learning driven multimodal fusion for automated deception detection. In Proceedings of IEEE Symposium Series on Computational Intelligence, Honolulu, USA, 2017. DOI: https:\/\/doi.org\/10.1109\/SSCI.2017.8285382."},{"key":"1630_CR23","doi-asserted-by":"publisher","first-page":"1336","DOI":"10.1609\/aaai.v39i2.32123","volume-title":"Proceedings of the 39th AAAI Conference on Artificial Intelligence","author":"Z Ji","year":"2025","unstructured":"Z. Ji, X. Tian, Y. Liu. AFFAKT: A hierarchical optimal transport based method for affective facial knowledge transfer in video deception detection. In Proceedings of the 39th AAAI Conference on Artificial Intelligence, Philadelphia, USA, pp. 1336\u20131344, 2025. DOI: https:\/\/doi.org\/10.1609\/aaai.v39i2.32123."},{"key":"1630_CR24","doi-asserted-by":"publisher","first-page":"22078","DOI":"10.1109\/ICCV51070.2023.02023","volume-title":"Proceedings of IEEE\/CVF International Conference on Computer Vision","author":"X Guo","year":"2023","unstructured":"X. Guo, N. M. Selvaraj, Z. Yu, A. W. K. Kong, B. Shen, A. Kot. Audio-visual deception detection: DOLOS dataset and parameter-efficient crossmodal learning. In Proceedings of IEEE\/CVF International Conference on Computer Vision, Paris, France, pp. 22078\u201322088, 2023. DOI: https:\/\/doi.org\/10.1109\/ICCV51070.2023.02023."},{"key":"1630_CR25","doi-asserted-by":"publisher","first-page":"6502","DOI":"10.1609\/aaai.v34i04.6123","volume-title":"Proceedings of the 34th AAAI Conference on Artificial Intelligence","author":"M Xu","year":"2020","unstructured":"M. Xu, J. Zhang, B. Ni, T. Li, C. Wang, Q. Tian, W. Zhang. Adversarial domain adaptation with domain mixup. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, USA, pp. 6502\u20136509, 2020. DOI: https:\/\/doi.org\/10.1609\/aaai.v34i04.6123."},{"key":"1630_CR26","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11596","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"D Li","year":"2018","unstructured":"D. Li, Y. Yang, Y. Z. Song, T. Hospedales. Learning to generalize: Meta-learning for domain generalization. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, USA, Article number 427, 2018. DOI: https:\/\/doi.org\/10.1609\/aaai.v32i1.11596."},{"key":"1630_CR27","first-page":"8265","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems","author":"D Arpit","year":"2022","unstructured":"D. Arpit, H. Wang, Y. Zhou, C. Xiong. Ensemble of averages: Improving model selection and boosting performance in domain generalization. In Proceedings of the 36th International Conference on Neural Information Processing Systems, New Orleans, USA, pp. 8265\u20138277, 2022."},{"key":"1630_CR28","doi-asserted-by":"publisher","first-page":"3150","DOI":"10.1609\/aaai.v29i1.9542","volume-title":"Proceedings of the 29th AAAI Conference on Artificial Intelligence","author":"K Zhang","year":"2015","unstructured":"K. Zhang, M. Gong, B. Schoelkopf. Multi-source domain adaptation: A causal view. In Proceedings of the 29th AAAI Conference on Artificial Intelligence, Austin, USA, pp. 3150\u20133157, 2015. DOI: https:\/\/doi.org\/10.1609\/aaai.v29i1.9542."},{"key":"1630_CR29","doi-asserted-by":"publisher","first-page":"8354","DOI":"10.24963\/ijcai.2024\/923","volume-title":"Proceedings of the 33rd International Joint Conference on Artificial Intelligence","author":"S Zhao","year":"2024","unstructured":"S. Zhao, H. Chen, H. Huang, P. Xu, G. Ding. More is better: Deep domain adaptation with multiple sources. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence, Jeju, Republic of Korea, pp. 8354\u20138362, 2024. DOI: https:\/\/doi.org\/10.24963\/ijcai.2024\/923."},{"key":"1630_CR30","doi-asserted-by":"publisher","first-page":"6816","DOI":"10.1109\/CVPR42600.2020.00685","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Y Shi","year":"2020","unstructured":"Y. Shi, X. Yu, K. Sohn, M. Chandraker, A. K. Jain. Towards universal representation learning for deep face recognition. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 6816\u20136825, 2020. DOI: https:\/\/doi.org\/10.1109\/CVPR42600.2020.00685."},{"key":"1630_CR31","volume-title":"Proceedings of the 6th International Conference on Learning Representations","author":"S Shankar","year":"2018","unstructured":"S. Shankar, V. Piratla, S. Chakrabarti, S. Chaudhuri, P. Jyothi, S. Sarawagi. Generalizing across domains via cross-gradient training. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018."},{"key":"1630_CR32","doi-asserted-by":"publisher","first-page":"7439","DOI":"10.1109\/ICASSP40776.2020.9053074","volume-title":"Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing","author":"A Mathur","year":"2020","unstructured":"A. Mathur, F. Kawsar, N. Berthouze, N. D. Lane. Libri-Adapt: A new speech dataset for unsupervised domain adaptation. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, Spain, pp. 7439\u20137443, 2020. DOI: https:\/\/doi.org\/10.1109\/ICASSP40776.2020.9053074."},{"key":"1630_CR33","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1609\/aaai.v35i2.16192","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence","author":"Q Chen","year":"2021","unstructured":"Q. Chen, Y. Liu, S. Albanie. Mind-the-gap! Unsupervised domain adaptation for text-video retrieval. In Proceedings of the 35th AAAI Conference on Artificial Intelligence, pp. 1072\u20131080, 2021. DOI: https:\/\/doi.org\/10.1609\/aaai.v35i2.16192."},{"key":"1630_CR34","doi-asserted-by":"publisher","first-page":"23595","DOI":"10.1109\/CVPR52733.2024.02227","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"D Cheng","year":"2024","unstructured":"D. Cheng, Z. Xu, X. Jiang, N. Wang, D. Li, X. Gao. Disentangled prompt representation for domain generalization. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 23595\u201323604, 2024. DOI: https:\/\/doi.org\/10.1109\/CVPR52733.2024.02227."},{"key":"1630_CR35","doi-asserted-by":"publisher","unstructured":"S. Zhao, J. Jiang, W. Tang, J. Zhu, H. Chen, P. Xu, B. W. Schuller, J. Tao, H. Yao, G. Ding. Multi-source multi-modal domain adaptation. Information Fusion, vol. 117, Article number 102862, 2025. DOI: https:\/\/doi.org\/10.1016\/j.inffus.2024.102862.","DOI":"10.1016\/j.inffus.2024.102862"},{"key":"1630_CR36","doi-asserted-by":"publisher","first-page":"12692","DOI":"10.1109\/CVPR42600.2020.01271","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"W Wang","year":"2020","unstructured":"W. Wang, D. Tran, M. Feiszli. What makes training multi-modal classification networks hard? In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 12692\u201312702, 2020. DOI: https:\/\/doi.org\/10.1109\/CVPR42600.2020.01271."},{"key":"1630_CR37","first-page":"9226","volume-title":"Proceedings of the 39th International Conference on Machine Learning","author":"Y Huang","year":"2022","unstructured":"Y. Huang, J. Lin, C. Zhou, H. Yang, L. Huang. Modality competition: What makes joint training of multimodal network fail in deep learning? (Provably). In Proceedings of the 39th International Conference on Machine Learning, Baltimore, USA, pp. 9226\u20139259, 2022."},{"key":"1630_CR38","doi-asserted-by":"publisher","first-page":"8228","DOI":"10.1109\/CVPR52688.2022.00806","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"X Peng","year":"2022","unstructured":"X. Peng, Y. Wei, A. Deng, D. Wang, D. Hu. Balanced multimodal learning via on-the-fly gradient modulation. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 8228\u20138237, 2022. DOI: https:\/\/doi.org\/10.1109\/CVPR52688.2022.00806."},{"key":"1630_CR39","first-page":"24043","volume-title":"Proceedings of the 39th International Conference on Machine Learning","author":"N Wu","year":"2022","unstructured":"N. Wu, S. Jastrzebski, K. Cho, K. J. Geras. Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks. In Proceedings of the 39th International Conference on Machine Learning, Baltimore, USA, pp. 24043\u201324055, 2022."},{"key":"1630_CR40","doi-asserted-by":"publisher","first-page":"4364","DOI":"10.18653\/v1\/2025.naacllong.222","volume-title":"Proceedings of Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"J Kwon","year":"2025","unstructured":"J. Kwon, M. Kim, E. Lee, J. Choi, Y. Kim. See-saw modality balance: See gradient, and sew impaired vision-language balance to mitigate dominant modality bias. In Proceedings of Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, Albuquerque, USA, pp. 4364\u20134378, 2025. DOI: https:\/\/doi.org\/10.18653\/v1\/2025.naacllong.222."},{"key":"1630_CR41","doi-asserted-by":"publisher","first-page":"22157","DOI":"10.1109\/ICCV51070.2023.02030","volume-title":"Proceedings of IEEE\/CVF International Conference on Computer Vision","author":"H Li","year":"2023","unstructured":"H. Li, X. Li, P. Hu, Y. Lei, C. Li, Y. Zhou. Boosting multi-modal model performance with adaptive gradient modulation. In Proceedings of IEEE\/CVF International Conference on Computer Vision, Paris, France, pp. 22157\u201322167, 2023. DOI: https:\/\/doi.org\/10.1109\/ICCV51070.2023.02030."},{"issue":"1","key":"1630_CR42","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1109\/TPAMI.2024.3468315","volume":"47","author":"Y Wei","year":"2025","unstructured":"Y. Wei, D. Hu, H. Du, J. R. Wen. On-the-fly modulation for balanced multimodal learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 1, pp. 469\u2013485, 2025. DOI: https:\/\/doi.org\/10.1109\/TPAMI.2024.3468315.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1630_CR43","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP49357.2023.10096655","volume-title":"Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing","author":"R Xu","year":"2023","unstructured":"R. Xu, R. Feng, S. X. Zhang, D. Hu. MMCosine: Multimodal cosine loss towards balanced audio-visual finegrained learning. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece, 2023. DOI: https:\/\/doi.org\/10.1109\/ICASSP49357.2023.10096655."},{"key":"1630_CR44","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Z Yang","year":"2024","unstructured":"Z. Yang, Y. Wei, C. Liang, D. Hu. Quantifying and enhancing multi-modal robustness with modality preference. In Proceedings of the 12th International Conference on Learning Representations, Vienna, Austria, 2024."},{"key":"1630_CR45","doi-asserted-by":"publisher","first-page":"27328","DOI":"10.1109\/CVPR52733.2024.02581","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Y Wei","year":"2024","unstructured":"Y. Wei, R. Feng, Z. Wang, D. Hu. Enhancing multimodal cooperation via sample-level modality valuation. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, pp. 27328\u201327337, 2024. DOI: https:\/\/doi.org\/10.1109\/CVPR52733.2024.02581."},{"key":"1630_CR46","first-page":"8632","volume-title":"Proceedings of the 40th International Conference on Machine Learning","author":"C Du","year":"2023","unstructured":"C. Du, J. Teng, T. Li, Y. Liu, T. Yuan, Y. Wang, Y. Yuan, H. Zhao. On Uni-modal feature learning in supervised multi-modal learning. In Proceedings of the 40th International Conference on Machine Learning, Honolulu, USA, pp. 8632\u20138656, 2023."},{"key":"1630_CR47","doi-asserted-by":"publisher","first-page":"20029","DOI":"10.1109\/CVPR52729.2023.01918","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Y Fan","year":"2023","unstructured":"Y. Fan, W. Xu, H. Wang, J. Wang, S. Guo. PMR: Prototypical modal rebalance for multimodal learning. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, pp. 20029\u201320038, 2023. DOI: https:\/\/doi.org\/10.1109\/CVPR52729.2023.01918."},{"key":"1630_CR48","volume-title":"Proceedings of the 41st International Conference on Machine Learning","author":"Y Wei","year":"2024","unstructured":"Y. Wei, D. Hu. MMPareto: Boosting multimodal learning with innocent unimodal assistance. In Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, Article number 2154, 2024."},{"key":"1630_CR49","first-page":"22879","volume-title":"Proceedings of IEEE\/CVF International Conference on Computer Vision","author":"S Wei","year":"2025","unstructured":"S. Wei, C. Luo, Y. Luo. Boosting multimodal learning via disentangled gradient learning. In Proceedings of IEEE\/CVF International Conference on Computer Vision, Honolulu, USA, pp. 22879\u201322888, 2025."},{"key":"1630_CR50","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1109\/CVPR.2016.90","volume-title":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition","author":"K He","year":"2016","unstructured":"K. He, X. Zhang, S. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 770\u2013778, 2016. DOI: https:\/\/doi.org\/10.1109\/CVPR.2016.90."},{"issue":"1","key":"1630_CR51","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1038\/s42256-020-00280-0","volume":"3","author":"A Toisoul","year":"2021","unstructured":"A. Toisoul, J. Kossaifi, A. Bulat, G. Tzimiropoulos, M. Pantic. Estimation of continuous valence and arousal levels from faces in naturalistic conditions. Nature Machine Intelligence, vol. 3, no. 1, pp. 42\u201350, 2021. DOI: https:\/\/doi.org\/10.1038\/s42256-020-00280-0.","journal-title":"Nature Machine Intelligence"},{"key":"1630_CR52","series-title":"Technical Report","volume-title":"OpenFace: A General-Purpose Face Recognition Library with Mobile Applications","author":"B Amos","year":"2016","unstructured":"B. Amos, B. Ludwiczuk, M. Satyanarayanan. OpenFace: A General-Purpose Face Recognition Library with Mobile Applications, Technical Report CMU-CS-16-118, School of Computer Science, Carnegie Mellon University, USA, 2016."},{"key":"1630_CR53","doi-asserted-by":"publisher","first-page":"6558","DOI":"10.18653\/v1\/P19-1656","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Y H H Tsai","year":"2019","unstructured":"Y. H. H. Tsai, S. Bai, P. P. Liang, J. Z. Kolter, L. P. Morency, R. Salakhutdinov. Multimodal transformer for unaligned multimodal language sequences. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 6558\u20136569, 2019. DOI: https:\/\/doi.org\/10.18653\/v1\/P19-1656."},{"key":"1630_CR54","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1007\/978-3-319-49409-8_35","volume-title":"Proceedings of European Conference on Computer Vision","author":"B Sun","year":"2016","unstructured":"B. Sun, K. Saenko. Deep CORAL: Correlation alignment for deep domain adaptation. In Proceedings of European Conference on Computer Vision, Amsterdam, The Netherlands, pp. 443\u2013450, 2016. DOI: https:\/\/doi.org\/10.1007\/978-3-319-49409-8_35."},{"issue":"11","key":"1630_CR55","doi-asserted-by":"publisher","first-page":"3918","DOI":"10.1109\/TPAMI.2020.2991050","volume":"43","author":"J Li","year":"2021","unstructured":"J. Li, E. Chen, Z. Ding, L. Zhu, K. Lu, H. T. Shen. Maximum density divergence for domain adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 11, pp. 3918\u20133930, 2021. DOI: https:\/\/doi.org\/10.1109\/TPAMI.2020.2991050.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"1630_CR56","first-page":"2672","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems","author":"I J Goodfellow","year":"2014","unstructured":"I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 2672\u20132680, 2014."},{"key":"1630_CR57","volume-title":"Proceedings of the 10th International Conference on Learning Representations","author":"D Acuna","year":"2022","unstructured":"D. Acuna, M. T. Law, G. Zhang, S. Fidler. Domain adversarial training: A game perspective. In Proceedings of the 10th International Conference on Learning Representations, 2022."},{"key":"1630_CR58","first-page":"1647","volume-title":"Proceedings of the 32nd International Conference on Neural Information Processing Systems","author":"M Long","year":"2018","unstructured":"M. Long, Z. Cao, J. Wang, M. I. Jordan. Conditional adversarial domain adaptation. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 1647\u20131657, 2018."},{"key":"1630_CR59","first-page":"529","volume-title":"Proceedings of the 18th International Conference on Neural Information Processing Systems","author":"Y Grandvalet","year":"2004","unstructured":"Y. Grandvalet, Y. Bengio. Semi-supervised learning by entropy minimization. In Proceedings of the 18th International Conference on Neural Information Processing Systems, Vancouver, Canada, pp. 529\u2013536, 2004."},{"key":"1630_CR60","first-page":"1180","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","author":"Y Ganin","year":"2015","unstructured":"Y. Ganin, V. Lempitsky. Unsupervised domain adaptation by backpropagation. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, pp. 1180\u20131189, 2015."},{"issue":"4","key":"1630_CR61","doi-asserted-by":"publisher","first-page":"4396","DOI":"10.1109\/TPAMI.2022.3195549","volume":"45","author":"K Zhou","year":"2023","unstructured":"K. Zhou, Z. Liu, Y. Qiao, T. Xiang, C. C. Loy. Domain generalization: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 4, pp. 4396\u20134415, 2023. DOI: https:\/\/doi.org\/10.1109\/TPAMI.2022.3195549.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"8","key":"1630_CR62","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"S. Hochreiter, J. Schmidhuber. Long short-term memory. Neural Computation, vol. 9, no. 8, pp. 1735\u20131780, 1997. DOI: https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735.","journal-title":"Neural Computation"},{"key":"1630_CR63","volume-title":"Proceedings of the 10th International Conference on Learning Representations","author":"Y Shi","year":"2022","unstructured":"Y. Shi, J. Seely, P. H. S. Torr, S. Narayanaswamy, A. Y. Hannun, N. Usunier, G. Synnaeve. Gradient matching for domain generalization. In Proceedings of the 10th International Conference on Learning Representations, 2022."},{"key":"1630_CR64","volume-title":"Proceedings of the 7th International Conference on Learning Representations","author":"I Loshchilov","year":"2019","unstructured":"I. Loshchilov, F. Hutter. Decoupled weight decay regularization. In Proceedings of the 7th International Conference on Learning Representations, New Orleans, USA, 2019."},{"key":"1630_CR65","doi-asserted-by":"publisher","first-page":"4113","DOI":"10.1109\/CVPR52688.2022.00409","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Z Wang","year":"2022","unstructured":"Z. Wang, Z. Wang, Z. Yu, W. Deng, J. Li, T. Gao, Z. Wang. Domain generalization via shuffled style assembly for face anti-spoofing. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, pp. 4113\u20134123, 2022. DOI: https:\/\/doi.org\/10.1109\/CVPR52688.2022.00409."},{"issue":"1","key":"1630_CR66","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1109\/TAFFC.2023.3265063","volume":"15","author":"T Varanka","year":"2024","unstructured":"T. Varanka, Y. Li, W. Peng, G. Zhao. Data leakage and evaluation issues in micro-expression analysis. IEEE Transactions on Affective Computing, vol. 15, no. 1, pp. 186\u2013197, 2024. DOI: https:\/\/doi.org\/10.1109\/TAFFC.2023.3265063.","journal-title":"IEEE Transactions on Affective Computing"},{"key":"1630_CR67","doi-asserted-by":"publisher","first-page":"7132","DOI":"10.1109\/CVPR.2018.00745","volume-title":"Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"J Hu","year":"2018","unstructured":"J. Hu, L. Shen, G. Sun. Squeeze-and-excitation networks. In Proceedings of IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 7132\u20137141, 2018. DOI: https:\/\/doi.org\/10.1109\/CVPR.2018.00745."},{"key":"1630_CR68","first-page":"6000","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"A Vaswani","year":"2017","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, I. Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, pp. 6000\u20136010, 2017."}],"container-title":["Machine Intelligence Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11633-025-1630-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11633-025-1630-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11633-025-1630-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T12:02:25Z","timestamp":1775649745000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11633-025-1630-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4]]},"references-count":68,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["1630"],"URL":"https:\/\/doi.org\/10.1007\/s11633-025-1630-x","relation":{},"ISSN":["2731-538X","2731-5398"],"issn-type":[{"value":"2731-538X","type":"print"},{"value":"2731-5398","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4]]},"assertion":[{"value":"29 August 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 December 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 April 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declared that they have no conflicts of interest to this work.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations of conflict of interest"}}]}}