{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:03:21Z","timestamp":1750309401904,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":38,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T00:00:00Z","timestamp":1730073600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"This work was supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2020-0-00004, Development of Previsional Intelligence based on Long-term Visual Memory Network).","award":["2020-0-00004"],"award-info":[{"award-number":["2020-0-00004"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,10,28]]},"DOI":"10.1145\/3689091.3690089","type":"proceedings-article","created":{"date-parts":[[2024,10,23]],"date-time":"2024-10-23T22:29:53Z","timestamp":1729722593000},"page":"26-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Bridging the Lexical Gap: Generative Text-to-Image Retrieval for Parts-of-Speech Imbalance in Vision-Language Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8673-2432","authenticated-orcid":false,"given":"Hyesu","family":"Hwang","sequence":"first","affiliation":[{"name":"University of Seoul, Seoul, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9545-8937","authenticated-orcid":false,"given":"Daeun","family":"Kim","sequence":"additional","affiliation":[{"name":"University of Seoul, Seoul, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7385-5494","authenticated-orcid":false,"given":"Jaehui","family":"Park","sequence":"additional","affiliation":[{"name":"University of Seoul, Seoul, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8818-4657","authenticated-orcid":false,"given":"Yongjin","family":"Kwon","sequence":"additional","affiliation":[{"name":"Electronics and Telecommunications Research Institute, Daejeon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,10,28]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Lin (Eds.)","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877--1901. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/ 1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 2023 Conf. on Human Information Interaction and Retrieval","author":"Deckers Niklas","year":"2022","unstructured":"Niklas Deckers, Maik Frobe, Johannes Kiesel, Gianluca Pandolfo, Christopher Schr\u00f6der, Benno Stein, and Martin Potthast. 2022. The Infinite Index: Information Retrieval on Generative Text-To-Image Models. Proceedings of the 2023 Conf. on Human Information Interaction and Retrieval (2022). https:\/\/api.semanticscholar.org\/CorpusID:254685940"},{"key":"e_1_3_2_1_3_1","volume-title":"2021 IEEE\/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2020","author":"Desai Karan","year":"2020","unstructured":"Karan Desai and Justin Johnson. 2020. VirTex: Learning Visual Representations from Textual Annotations. 2021 IEEE\/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2020), 11157--11168. https:\/\/api.semanticscholar.org\/ CorpusID:219573658"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00180"},{"key":"e_1_3_2_1_5_1","volume-title":"International Conf. on Learning Representations. https:\/\/openreview.net\/forum?id= YicbFdNTTy","author":"Dosovitskiy Alexey","year":"2021","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conf. on Learning Representations. https:\/\/openreview.net\/forum?id= YicbFdNTTy"},{"key":"e_1_3_2_1_6_1","unstructured":"Zi-Yi* Dou Aishwarya* Kamath Zhe* Gan Pengchuan Zhang Jianfeng Wang Linjie Li Zicheng Liu Ce Liu Yann LeCun Nanyun Peng Jianfeng Gao and Lijuan Wang. 2022. Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone. In NeurIPS."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber","author":"Gao Luyu","year":"1865","unstructured":"Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. Precise Zero-Shot Dense Retrieval without Relevance Labels. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (Eds.). Association for Computational Linguistics, Toronto, Canada, 1762--1777. https:\/\/doi.org\/10. 18653\/v1\/2023.acl-long.99"},{"key":"e_1_3_2_1_8_1","volume-title":"Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv abs\/2312.10997","author":"Gao Yunfan","year":"2023","unstructured":"Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. 2023. Retrieval-Augmented Generation for Large Language Models: A Survey. ArXiv abs\/2312.10997 (2023). https:\/\/api.semanticscholar.org\/CorpusID:266359151"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983835"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462838"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData59044.2023.10386812"},{"key":"e_1_3_2_1_12_1","volume-title":"Proceedings of the IEEE\/CVF Conf. on computer vision and pattern recognition. 17980--17989","author":"Hu Xiaowei","year":"2022","unstructured":"Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, and Lijuan Wang. 2022. Scaling up vision-language pre-training for image captioning. In Proceedings of the IEEE\/CVF Conf. on computer vision and pattern recognition. 17980--17989."},{"key":"e_1_3_2_1_13_1","volume-title":"NAPReg: Nouns As Proxies Regularization for Semantically Aware Cross-Modal Embeddings. In 2023 IEEE\/CVF Winter Conf. on Applications of Computer Vision (WACV). 1135--1144","author":"Jawade Bhavin","year":"2023","unstructured":"Bhavin Jawade, Deen Dayal Mohan, Naji Mohamed Ali, Srirangaraj Setlur, and Venu Govindaraju. 2023. NAPReg: Nouns As Proxies Regularization for Semantically Aware Cross-Modal Embeddings. In 2023 IEEE\/CVF Winter Conf. on Applications of Computer Vision (WACV). 1135--1144. https:\/\/doi.org\/10.1109\/ WACV56688.2023.00119"},{"key":"e_1_3_2_1_14_1","unstructured":"Albert Qiaochu Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de Las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier Lelio Renard Lavaud Marie-Anne Lachaux Pierre Stock Teven Le Scao Thibaut Lavril Thomas Wang Timoth\u00e9e Lacroix and William El Sayed. 2023. Mistral 7B. ArXiv abs\/2310.06825 (2023). https: \/\/api.semanticscholar.org\/CorpusID:263830494"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_17_1","volume-title":"International Conf. on Machine Learning. https:\/\/api.semanticscholar.org\/CorpusID:231839613","author":"Kim Wonjae","year":"2021","unstructured":"Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision. In International Conf. on Machine Learning. https:\/\/api.semanticscholar.org\/CorpusID:231839613"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Ranjay Krishna Yuke Zhu Oliver Groth Justin Johnson Kenji Hata Joshua Kravitz Stephanie Chen Yannis Kalantidis Li-Jia Li David A Shamma et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision 123 (2017) 32--73.","DOI":"10.1007\/s11263-016-0981-7"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.397"},{"key":"e_1_3_2_1_20_1","volume-title":"International Conf. on Machine Learning. PMLR, 12888--12900","author":"Li Junnan","year":"2022","unstructured":"Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conf. on Machine Learning. PMLR, 12888--12900."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings, Part V 13","author":"Lin Tsung-Yi","year":"2014","unstructured":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll\u00e1r, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conf., Zurich, Switzerland, September 6--12, 2014, Proceedings, Part V 13. Springer, 740--755."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.153"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2022\/773"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.322"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591992"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_27_1","unstructured":"Kelong Mao Zhicheng Dou Haonan Chen Fengran Mo and Hongjin Qian. 2023. Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search. In Conf. on Empirical Methods in Natural Language Processing. https:\/\/api.semanticscholar.org\/CorpusID:257495903"},{"key":"e_1_3_2_1_28_1","volume-title":"2019 IEEE\/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)","author":"Qi Mengshi","year":"2018","unstructured":"Mengshi Qi,Weijian Li, Zhengyuan Yang, YunhongWang, and Jiebo Luo. 2018. Attentive Relational Networks for Mapping Images to Scene Graphs. 2019 IEEE\/CVF Conf. on Computer Vision and Pattern Recognition (CVPR) (2018), 3952--3961. https:\/\/api.semanticscholar.org\/CorpusID:53735704"},{"key":"e_1_3_2_1_29_1","volume-title":"International conf. on machine learning. PMLR, 8748--8763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conf. on machine learning. PMLR, 8748--8763."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01519"},{"key":"e_1_3_2_1_31_1","volume-title":"Zephyr: Direct Distillation of LM Alignment. ArXiv abs\/2310.16944","author":"Tunstall Lewis","year":"2023","unstructured":"Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Cl\u00e9mentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2023. Zephyr: Direct Distillation of LM Alignment. ArXiv abs\/2310.16944 (2023). https:\/\/api.semanticscholar.org\/CorpusID:264490502"},{"key":"e_1_3_2_1_32_1","first-page":"2579","article-title":"Visualizing Data using t-SNE","volume":"9","author":"van der Maaten Laurens","year":"2008","unstructured":"Laurens van der Maaten and Geoffrey E. Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579--2605. https:\/\/api. semanticscholar.org\/CorpusID:5855042","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_33_1","unstructured":"Ashish Vaswani Noam M. Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention is All you Need. In Neural Information Processing Systems. https:\/\/api.semanticscholar.org\/ CorpusID:13756489"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.585"},{"key":"e_1_3_2_1_35_1","volume-title":"Oh (Eds.)","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 24824--24837. https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2022\/file\/ 9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1"},{"key":"e_1_3_2_1_37_1","volume-title":"Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Thirty-seventh Conf. on Neural Information Processing Systems Datasets and Benchmarks Track. https:\/\/openreview.net\/forum?id=uccHPGDlao","author":"Zheng Lianmin","year":"2023","unstructured":"Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Thirty-seventh Conf. on Neural Information Processing Systems Datasets and Benchmarks Track. https:\/\/openreview.net\/forum?id=uccHPGDlao"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.11"}],"event":{"name":"MM '24: The 32nd ACM International Conference on Multimedia","sponsor":["SIGMM ACM Special Interest Group on Multimedia"],"location":"Melbourne VIC Australia","acronym":"MM '24"},"container-title":["Proceedings of the 2nd International Workshop on Deep Multimodal Generation and Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3689091.3690089","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3689091.3690089","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:57:54Z","timestamp":1750294674000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3689091.3690089"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,28]]},"references-count":38,"alternative-id":["10.1145\/3689091.3690089","10.1145\/3689091"],"URL":"https:\/\/doi.org\/10.1145\/3689091.3690089","relation":{},"subject":[],"published":{"date-parts":[[2024,10,28]]},"assertion":[{"value":"2024-10-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}