{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T16:50:06Z","timestamp":1780764606995,"version":"3.54.1"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>With the development of Vision-Language Pre-training Models (VLPMs) represented by CLIP and ALIGN, significant breakthroughs have been achieved for association-based visual tasks such as image classification and image-text retrieval by the zero-shot capability of CLIP without fine-tuning. However, CLIP is hard to apply to generation-based tasks. This is due to the lack of decoder architecture and pre-training tasks for generation. Although previous works have created generation capacity for CLIP through additional language models, a modality gap between the CLIP representations of different modalities and the inability of CLIP to model the offset of this gap, which results in the failure of the concept to transfer across modes. To solve the problem, we try to map images\/videos to the language modality and generate captions from the language modality. In this paper, we propose the K-nearest-neighbor Cross-modality Mapping (Knight), a zero-shot method from association to generation. With vision-free unsupervised training, Knight achieves state-of-the-art performance in zero-shot methods for image captioning and video captioning.<\/jats:p>","DOI":"10.24963\/ijcai.2023\/481","type":"proceedings-article","created":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:31:30Z","timestamp":1691742690000},"page":"4326-4334","source":"Crossref","is-referenced-by-count":11,"title":["From Association to Generation: Text-only Captioning by Unsupervised Cross-modal Mapping"],"prefix":"10.24963","author":[{"given":"Junyang","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer and Information Technology & Beijing Key Lab of Traffc Data Analysis and Mining, Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ming","family":"Yan","sequence":"additional","affiliation":[{"name":"DAMO Academy, Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology & Beijing Key Lab of Traffc Data Analysis and Mining, Beijing Jiaotong University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jitao","family":"Sang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology & Beijing Key Lab of Traffc Data Analysis and Mining, Beijing Jiaotong University"},{"name":"Peng Cheng Lab"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10584","event":{"name":"Thirty-Second International Joint Conference on Artificial Intelligence {IJCAI-23}","theme":"Artificial Intelligence","location":"Macau, SAR China","acronym":"IJCAI-2023","number":"32","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2023,8,19]]},"end":{"date-parts":[[2023,8,25]]}},"container-title":["Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2023,8,11]],"date-time":"2023-08-11T08:49:34Z","timestamp":1691743774000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2023\/481"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2023\/481","relation":{},"subject":[],"published":{"date-parts":[[2023,8]]}}}