{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T05:30:59Z","timestamp":1781587859926,"version":"3.54.5"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T00:00:00Z","timestamp":1646352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"JSPS KAKENHI","award":["JP17H01744"],"award-info":[{"award-number":["JP17H01744"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2022,8,31]]},"abstract":"<jats:p>Cross-modal image-retrieval methods retrieve desired images from a query text by learning relationships between texts and images. Such a retrieval approach is one of the most effective ways of achieving the easiness of query preparation. Recent cross-modal image-retrieval methods are convenient and accurate when users input a query text that can be used to uniquely identify the desired image. However, in reality, users frequently input ambiguous query texts, and these ambiguous queries make it difficult to obtain desired images. To overcome these difficulties, in this study, we propose a novel interactive cross-modal image-retrieval method based on question answering. The proposed method analyzes candidate images and asks users questions to obtain information that can narrow down retrieval candidates. By only answering questions generated by the proposed method, users can reach their desired images, even when using an ambiguous query text. Experimental results show the proposed method\u2019s effectiveness.<\/jats:p>","DOI":"10.1145\/3485042","type":"journal-article","created":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T10:26:32Z","timestamp":1646389592000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Interactive Re-ranking via Object Entropy-Guided Question Answering for Cross-Modal Image Retrieval"],"prefix":"10.1145","volume":"18","author":[{"given":"Rintaro","family":"Yanagi","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Technology, Hokkaido University, Kita-ku, Sapporo, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ren","family":"Togo","sequence":"additional","affiliation":[{"name":"Education and Research Center for Mathematical and Data Science, Hokkaido University, Kita-ku, Sapporo, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Takahiro","family":"Ogawa","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology Division of Media and Network Technologies, Hokkaido University, Kita-ku, Sapporo, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miki","family":"Haseyama","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology Division of Media and Network Technologies, Hokkaido University, Kita-ku, Sapporo, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,3,4]]},"reference":[{"key":"e_1_3_1_2_2","article-title":"ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ)","author":"Aliannejadi Mohammad","year":"2020","unstructured":"Mohammad Aliannejadi, Julia Kiseleva, Aleksandr Chuklin, Jeff Dalton, and Mikhail Burtsev. 2020. ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ). arXiv:2009.11352. Retrieved from https:\/\/arxiv.org\/abs\/2009.11352.","journal-title":"arXiv:2009.11352"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331265"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.09.039"},{"key":"e_1_3_1_5_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . 4171\u20134186."},{"key":"e_1_3_1_6_2","unstructured":"Fartash Faghri David J. Fleet Jamie Ryan Kiros Google Brain Toronto and Sanja Fidler. 2018. VSE ++: Improving visual-semantic embeddings with hard negatives. In Proceedings of the British Machine Vision Conference ."},{"key":"e_1_3_1_7_2","unstructured":"Bin Fu Yunqi Qiu Chengguang Tang Yang Li Haiyang Yu and Jian Sun. 2021. A survey on complex question answering over knowledge base: Recent advances and challenges. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence . 4483\u20134491."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/1282280.1282347"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00750"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2016.04.001"},{"key":"e_1_3_1_11_2","first-page":"678","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Guo Xiaoxiao","year":"2018","unstructured":"Xiaoxiao Guo, Hui Wu, Yu Cheng, Steven Rennie, Gerald Tesauro, and Rogerio Feris. 2018. Dialog-based interactive image retrieval. In Proceedings of the Advances in Neural Information Processing Systems. 678\u2013688."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4380-9_14"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00645"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00585"},{"key":"e_1_3_1_16_2","article-title":"Unifying visual-semantic embeddings with multimodal neural language models","author":"Kiros Ryan","year":"2014","unstructured":"Ryan Kiros, Ruslan Salakhutdinov, and Richard S. Zemel. 2014. Unifying visual-semantic embeddings with multimodal neural language models. arXiv:1411.2539. Retrieved from https:\/\/arxiv.org\/abs\/1411.2539.","journal-title":"arXiv:1411.2539"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-016-0981-7"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2008.05.004"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2015.04.037"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.442"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2016.2568099"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2017.09.008"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.06.096"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2536798"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2017.01.003"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-020-09292-9"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2699623"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401110"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3020165.3020183"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2020.2980944"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00208"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210002"},{"key":"e_1_3_1_36_2","first-page":"2651","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Tan Fuwen","year":"2019","unstructured":"Fuwen Tan, Paola Cascante-Bonilla, Xiaoxiao Guo, Hui Wu, Song Feng, and Vicente Ordonez. 2019. Drill-down: Interactive retrieval of complex scenes using natural language queries. In Proceedings of the Advances in Neural Information Processing Systems. 2651\u20132661."},{"key":"e_1_3_1_37_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Vendrov Ivan","year":"2016","unstructured":"Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. 2016. Order-embeddings of images and language. In Proceedings of the International Conference on Learning Representations. 1\u201312."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00660"},{"key":"e_1_3_1_39_2","article-title":"A comprehensive survey on cross-modal retrieval","author":"Wang Kaiye","year":"2016","unstructured":"Kaiye Wang, Qiyue Yin, Wei Wang, Shu Wu, and Liang Wang. 2016. A comprehensive survey on cross-modal retrieval. arXiv:1607.06215. Retrieved from https:\/\/arxiv.org\/abs\/1607.06215.","journal-title":"arXiv:1607.06215"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350875"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2992187"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.124"},{"issue":"1","key":"e_1_3_1_43_2","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/TKDE.2013.70","article-title":"EMR: A scalable graph-based ranking model for content-based image retrieval","volume":"27","author":"Xu Bin","year":"2013","unstructured":"Bin Xu, Jiajun Bu, Chun Chen, Can Wang, Deng Cai, and Xiaofei He. 2013. EMR: A scalable graph-based ranking model for content-based image retrieval. IEEE Transactions on Knowledge and Data Engineering 27, 1 (2013), 102\u2013114.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.330"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3444685.3446290"},{"key":"e_1_3_1_46_2","unstructured":"Liu Yang Hamed Zamani Yongfeng Zhang Jiafeng Guo and W. Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence ."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00223"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380126"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00611"},{"issue":"1","key":"e_1_3_1_50_2","article-title":"Image search\u2014from thousands to billions in 20 years","volume":"9","author":"Zhang Lei","year":"2013","unstructured":"Lei Zhang and Yong Rui. 2013. Image search\u2014from thousands to billions in 20 years. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 1 (2013), 1\u201320.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271776"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01246-5_42"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33019259"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383184"},{"key":"e_1_3_1_55_2","article-title":"Recent advance in content-based image retrieval: A literature survey","author":"Zhou Wengang","year":"2017","unstructured":"Wengang Zhou, Houqiang Li, and Qi Tian. 2017. Recent advance in content-based image retrieval: A literature survey. arXiv:1706.06064. Retrieved from https:\/\/arxiv.org\/abs\/1706.06064.","journal-title":"arXiv:1706.06064"},{"key":"e_1_3_1_56_2","article-title":"Retrieving and reading: A comprehensive survey on open-domain question answering","author":"Zhu Fengbin","year":"2021","unstructured":"Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-Seng Chua. 2021. Retrieving and reading: A comprehensive survey on open-domain question answering. arXiv:2101.00774. Retrieved from https:\/\/arxiv.org\/abs\/2101.00774.","journal-title":"arXiv:2101.00774"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3485042","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3485042","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:15Z","timestamp":1750191435000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3485042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,4]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,8,31]]}},"alternative-id":["10.1145\/3485042"],"URL":"https:\/\/doi.org\/10.1145\/3485042","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,4]]},"assertion":[{"value":"2021-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-03-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}