{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T07:17:04Z","timestamp":1769757424036,"version":"3.49.0"},"reference-count":85,"publisher":"Association for Computing Machinery (ACM)","issue":"11","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62101524"],"award-info":[{"award-number":["62101524"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,11,30]]},"abstract":"<jats:p>Non-annotated visual description (NaVD) aims to describe generic visuals without human-annotated pairwise data. The generic visuals refer to images and videos. Existing works mainly focus on one specific visual modality, i.e., image or video. In this article, we propose a new framework for this task, which can directly be applied to both image and video with the pipeline unchanged. Essentially, it is a unified framework that flexibly adapts to images and videos. Recently, contrastive visual-language pre-training models (CVLPs) have experienced rapid development, demonstrating powerful abilities to align vision and language. To continuously leverage advanced CVLPs, our framework is designed to work well with general CVLPs. It can easily use image-language CVLPs for image input and switch to video-language CVLPs for video input. Specifically, we propose a CVLP-based framework for NaVD, named CVLP-NaVD. It follows the paradigm of adversarial learning, containing a generator and a discriminator. The generator takes an image or a video as input and produces a corresponding language description, while the discriminator evaluates the generated sentence for its naturalness in human-like language. Apart from the naturalness, CVLPs play a crucial role in enhancing the alignment between visual and language signals during generation. Particularly, we explore three rewarding strategies to compute the alignment score, including directly calculating cosine similarity (i.e., VL-cross), projecting visual embeddings into the textual domain (i.e., VL-project), and their combination (i.e., VL-mix). The three strategies are fully examined in different scenarios. Finally, we conduct extensive experiments with various unpaired and unsupervised setups in both image and video captioning tasks. The experimental results demonstrate that our CVLP-NaVD outperforms the state-of-the-art methods significantly.<\/jats:p>","DOI":"10.1145\/3708348","type":"journal-article","created":{"date-parts":[[2024,12,20]],"date-time":"2024-12-20T06:10:16Z","timestamp":1734675016000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["CVLP-NaVD: Contrastive Visual-language Pre-training Models for Non-annotated Visual Description"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-3593-7169","authenticated-orcid":false,"given":"Haoran","family":"Li","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0695-1566","authenticated-orcid":false,"given":"Yanbin","family":"Hao","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6327-6811","authenticated-orcid":false,"given":"Jiarui","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Data Science, University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9213-2611","authenticated-orcid":false,"given":"Bin","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computing and Information Systems, Singapore Management University (SMU), Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4881-9344","authenticated-orcid":false,"given":"Shuo","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4246-5386","authenticated-orcid":false,"given":"Tong","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Data Science, University of Science and Technology of China, Hefei, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,11,7]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"382","volume-title":"Proceedings of 14th European Conference Computer Vision (ECCV 2016), Part V","author":"Anderson Peter","year":"2016","unstructured":"Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. Spice: Semantic propositional image caption evaluation. In Proceedings of 14th European Conference Computer Vision (ECCV 2016), Part V, 382\u2013398."},{"key":"e_1_3_1_3_2","first-page":"65","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee, and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, 65\u201372."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3060948"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3652583.3658080"},{"issue":"1","key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/TCSVT.2021.3063297","article-title":"Entity slot filling for visual captioning","volume":"32","author":"Bin Yi","year":"2021","unstructured":"Yi Bin, Yujuan Ding, Bo Peng, Liang Peng, Yang Crossruff, and Tat-Seng Chua. 2021. Entity slot filling for visual captioning. IEEE Transactions on Circuits Systems and Video Technology 32, 1 (2021), 52\u201362.","journal-title":"IEEE Transactions on Circuits Systems and Video Technology"},{"key":"e_1_3_1_7_2","first-page":"5110","volume-title":"ACM International Conference on Multimedia","author":"Bin Yi","year":"2021","unstructured":"Yi Bin, Xindi Shang, Bo Peng, Yujuan Ding, and Tat-Seng Chua. 2021. Multi-perspective video captioning. In Proceedings of the 29th ACM International Conference on Multimedia, 5110\u20135118."},{"key":"e_1_3_1_8_2","volume-title":"Proceedings of the 32nd ACM International Conference on Multimedia.","author":"Bin Yi","unstructured":"Yi Bin, Wenhao. Shi, Yujuan Ding, Zhiqiang Hu, Wang Zheng, Yang Xinyi, See-Kiong Ng, and Heng Tao Shen. 2024. GalleryGPT: Analyzing paintings with large multimodal models. In Proceedings of the 32nd ACM International Conference on Multimedia."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2831447"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.08.019"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.667"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00347"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.323"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.121"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3059295"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01038"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00425"},{"issue":"3","key":"e_1_3_1_18_2","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1561\/0600000105","article-title":"Vision-language pre-training: Basics, recent advances, and future trends","volume":"14","author":"Gan Zhe","year":"2022","unstructured":"Zhe Gan, Linjie Li, Chunyuan Li, Lijuan Wang, Zicheng Liu, and Jianfeng Gao. 2022. Vision-language pre-training: Basics, recent advances, and future trends. Foundations and Trends in Computer Graphics and Vision 14, 3\u20134 (2022), 163\u2013352.","journal-title":"Foundations and Trends in Computer Graphics and Vision"},{"key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"10654","DOI":"10.1609\/aaai.v36i10.21310","article-title":"UNISON: Unpaired cross-lingual image captioning","volume":"36","author":"Gao Jiahui","year":"2022","unstructured":"Jiahui Gao, Yi Zhou, L. H. Philip, Shafiq Joty, and Jiuxiang Gu. 2022. UNISON: Unpaired cross-lingual image captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, 36 (2022), 10654\u201310662.","journal-title":"In Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2017.2729019"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01457"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3422622"},{"key":"e_1_3_1_23_2","first-page":"503","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Gu Jiuxiang","year":"2018","unstructured":"Jiuxiang Gu, Shafiq Joty, Jianfei Cai, and Gang Wang. 2018. Unpaired image captioning by language pivoting. In Proceedings of the European Conference on Computer Vision (ECCV), 503\u2013519."},{"key":"e_1_3_1_24_2","first-page":"10323","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Gu Jiuxiang","year":"2019","unstructured":"Jiuxiang Gu, Shafiq Joty, Jianfei Cai, Handong Zhao, Xu Yang, and Gang Wang. 2019. Unpaired image captioning via scene graph alignments. In Proceedings of the IEEE International Conference on Computer Vision, 10323\u201310332."},{"key":"e_1_3_1_25_2","first-page":"2672","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Gu Sophia","year":"2023","unstructured":"Sophia Gu, Christopher Clark, and Aniruddha Kembhavi. 2023. I can\u2019t believe there\u2019s no images! Learning visual tasks using only language supervision. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2672\u20132683."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2024.3358415"},{"key":"e_1_3_1_27_2","doi-asserted-by":"crossref","unstructured":"Dan Guo Yang Wang Peipei Song and Meng Wang. 2020. Recurrent relational memory network for unsupervised image captioning. arXiv:2006.13611. Retrieved from https:\/\/arxiv.org\/abs\/2006.13611","DOI":"10.24963\/ijcai.2020\/128"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00380"},{"key":"e_1_3_1_29_2","doi-asserted-by":"crossref","unstructured":"Ukyo Honda Yoshitaka Ushiku Atsushi Hashimoto Taro Watanabe and Yuji Matsumoto. 2021. Removing word-level spurious alignment between images and pseudo-captions in unsupervised image captioning. arXiv:2104.13872. Retrieved from https:\/\/arxiv.org\/abs\/2104.13872","DOI":"10.18653\/v1\/2021.eacl-main.323"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00769"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.351"},{"key":"e_1_3_1_32_2","first-page":"495","volume-title":"Proceedings of International Conference on Artificial Neural Networks","author":"Ji Puzhao","year":"2022","unstructured":"Puzhao Ji, Meng Cao, and Yuexian Zou. 2022. Visual relation-aware unsupervised video captioning. In Proceedings of International Conference on Artificial Neural Networks. Springer, 495\u2013507."},{"key":"e_1_3_1_33_2","first-page":"4904","volume-title":"Proceedings of International Conference on Machine Learning","author":"Jia Chao","year":"2021","unstructured":"Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In Proceedings of International Conference on Machine Learning. PMLR, 4904\u20134916."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01216-8_35"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00751"},{"key":"e_1_3_1_36_2","volume-title":"Proceedings of 11th International Conference on Learning Representations","author":"Li Wei","year":"2022","unstructured":"Wei Li, Linchao Zhu, Longyin Wen, and Yi Yang. 2022. DeCap: Decoding CLIP latents for zero-shot captioning via text-only training. In Proceedings of 11th International Conference on Learning Representations."},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2019.2896516"},{"key":"e_1_3_1_38_2","first-page":"17612","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems","author":"Liang Victor Weixin","year":"2022","unstructured":"Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena YeVung, and James Y. Zou. 2022. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. In Proceedings of the 36th International Conference on Neural Information Processing Systems, 17612\u201317625."},{"key":"e_1_3_1_39_2","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Proceedings of the Workshop on Text Summarization Branches Out. Association for Computational Linguistics 74\u201381."},{"key":"e_1_3_1_40_2","first-page":"740","volume-title":"Proceedings of 13th European Conference on Computer Vision (ECCV \u201914), Part V","author":"Lin Tsung-Yi","year":"2014","unstructured":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll\u00e1r, and C. Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Proceedings of 13th European Conference on Computer Vision (ECCV \u201914), Part V. Springer, 740\u2013755."},{"key":"e_1_3_1_41_2","first-page":"439","volume-title":"Proceedings of 2019 IEEE International Conference on Data Mining (ICDM)","author":"Liu Fenglin","year":"2019","unstructured":"Fenglin Liu, Meng Gao, Tianhao Zhang, and Yuexian Zou. 2019. Exploring semantic relationships for image captioning without parallel data. In Proceedings of 2019 IEEE International Conference on Data Mining (ICDM). IEEE, 439\u2013448."},{"key":"e_1_3_1_42_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_1_43_2","unstructured":"Ilya Loshchilov and Frank Hutter. 2018. Decoupled weight decay regularization. arXiv:1711.05101. Retrieved from https:\/\/arxiv.org\/abs\/1711.05101"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503161.3548358"},{"key":"e_1_3_1_45_2","first-page":"219","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Meng Zihang","year":"2022","unstructured":"Zihang Meng, David Yang, Xuefei Cao, Ashish Shah, and Ser-Nam Lim. 2022. Object-centric unsupervised image captioning. In Proceedings of the European Conference on Computer Vision (ECCV), 219\u2013235."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00990"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00272"},{"key":"e_1_3_1_48_2","unstructured":"Ron Mokady Amir Hertz and Amit H. Bermano. 2021. Clipcap: Clip prefix for image captioning. arXiv:2111.09734. Retrieved from https:\/\/arxiv.org\/abs\/2111.09734"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","unstructured":"David Nukrai Ron Mokady and Amir Globerson. 2022. Text-only training for image captioning using noise-injected CLIP. In EMNLP Findings. 4055\u20134063.","DOI":"10.18653\/v1\/2022.findings-emnlp.299"},{"key":"e_1_3_1_50_2","unstructured":"Aaron van den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv:1807.03748. Retrieved from https:\/\/arxiv.org\/abs\/1807.03748"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.117"},{"key":"e_1_3_1_52_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_1_53_2","article-title":"Referring image segmentation with attention guided cross modal fusion for semantic oriented languages","volume":"16","author":"Hu Haimiao","year":"2022","unstructured":"Haimiao Hu, Quange Tan, Wenjin Zhang, Qianli Zhou, and Rong Wang. 2022. Referring image segmentation with attention guided cross modal fusion for semantic oriented languages. Frontiers of Computer Science 16, Article 166342 (2022). https:\/\/journal.hep.com.cn\/fcs\/EN\/abstract\/article_30623.shtml","journal-title":"Frontiers of Computer Science"},{"key":"e_1_3_1_54_2","first-page":"8748","volume-title":"Proceedings of International Conference on Machine Learning","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In Proceedings of International Conference on Machine Learning. PMLR, 8748\u20138763."},{"issue":"8","key":"e_1_3_1_55_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"issue":"140","key":"e_1_3_1_56_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_57_2","first-page":"7008","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Steven","year":"2017","unstructured":"Steven, J. Rennie, Etienne Marcheret, Youssef Mroueh, Jerret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7008\u20137024."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1238"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2022.3175012"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350996"},{"key":"e_1_3_1_61_2","unstructured":"Yixuan Su Tian Lan Yahui Liu Fangyu Liu Dani Yogatama Yan Wang Lingpeng Kong and Nigel Collier. 2022. Language models can see: Plugging visual controls in text generation. arXiv:2205.02655. Retrieved from https:\/\/arxiv.org\/abs\/2205.02655"},{"key":"e_1_3_1_62_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01366"},{"key":"e_1_3_1_64_2","unstructured":"Yoad Tewel Yoav Shalev Roy Nadler Idan Schwartz and Lior Wolf. 2022. Zero-shot video captioning with evolving pseudo-tokens. arXiv:2207.11100. Retrieved from https:\/\/arxiv.org\/abs\/2207.11100"},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01739"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123354"},{"key":"e_1_3_1_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"e_1_3_1_68_2","doi-asserted-by":"crossref","unstructured":"Junyang Wang Ming Yan and Yi Zhang. 2023. From association to generation: Text-only captioning by unsupervised cross-modal mapping. arXiv:2304.13273. Retrieved from https:\/\/arxiv.org\/abs\/2304.13273","DOI":"10.24963\/ijcai.2023\/481"},{"key":"e_1_3_1_69_2","unstructured":"Shengqiong Wu Hao Fei Wei Ji and Tat-Seng Chua. 2023. Cross2StrA: Unpaired cross-lingual image captioning with cross-lingual cross-modal structure-pivoted alignment. arXiv:2305.12260. Retrieved from https:\/\/arxiv.org\/abs\/2305.12260"},{"key":"e_1_3_1_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998364"},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","unstructured":"Zuxuan Wu Ting Yao Yanwei Fu and Yu-Gang Jiang. 2017. Deep learning for video classification and captioning. In Frontiers of Multimedia Research. Association for Computing Machinery and Morgan & Claypool 3\u201329. 10.1145\/3122865.3122867","DOI":"10.1145\/3122865.3122867"},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2023.3321501"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01267-0_19"},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.544"},{"key":"e_1_3_1_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.571"},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01857"},{"key":"e_1_3_1_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3611709"},{"issue":"11","key":"e_1_3_1_78_2","doi-asserted-by":"crossref","first-page":"5600","DOI":"10.1109\/TIP.2018.2855422","article-title":"Video captioning by adversarial LSTM","volume":"27","author":"Yujiro Yang","year":"2018","unstructured":"Yang Yujiro, Jie Zhou, Jiangbo Ai, Yi Bin, Alan Hanjalic, Heng Tao Shen, and Yanli Ji. 2018. Video captioning by adversarial LSTM. IEEE Transactions on Image Processing 27, 11 (2018), 5600\u20135611.","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3581783.3611891"},{"key":"e_1_3_1_80_2","unstructured":"Youngjae Yu Jiwan Chung Heeseung Yun Jack Hessel Jae Sung Park Ximing Lu Prithviraj Ammanabrolu Rowan Zellers Ronan Le Bras Gunhee Kim et al. 2022. Multimodal knowledge alignment with reinforcement learning. arXiv:2205.12630. Retrieved from https:\/\/arxiv.org\/abs\/2205.12630"},{"key":"e_1_3_1_81_2","unstructured":"Lu Yuan Dongdong Chen Yi-Ling Chen Noel Codella Xiyang Dai Jianfeng Gao Houdong Hu Xuedong Huang Boxin Li Chunyuan Li et al. 2021. Florence: A new foundation model for computer vision. arXiv:2111.11432. Retrieved from https:\/\/arxiv.org\/abs\/2111.11432"},{"key":"e_1_3_1_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00611"},{"key":"e_1_3_1_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543857"},{"key":"e_1_3_1_84_2","first-page":"7598","volume-title":"Proceedings of ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)","author":"Zhou Yucheng","year":"2021","unstructured":"Yucheng Zhou, Wei Tao, and Wenqiang Zhang. 2021. Triple sequence generative adversarial nets for unsupervised image captioning. In Proceedings of ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 7598\u20137602."},{"key":"e_1_3_1_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3214090"},{"key":"e_1_3_1_86_2","first-page":"379","article-title":"Prompt-based learning for unpaired image captioning","volume":"26","author":"Zhu Peipei","year":"2023","unstructured":"Peipei Zhu, Xiao Wang, Lin Zhu, Zhenglong Sun, Wei-Shi Zheng, Yaowei Wang, and Changwen Chen. 2023. Prompt-based learning for unpaired image captioning. IEEE Transactions on Multimedia 26 (2023), 379\u2013393.","journal-title":"IEEE Transactions on Multimedia"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3708348","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T15:10:03Z","timestamp":1762528203000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3708348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,7]]},"references-count":85,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2025,11,30]]}},"alternative-id":["10.1145\/3708348"],"URL":"https:\/\/doi.org\/10.1145\/3708348","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,7]]},"assertion":[{"value":"2024-05-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}