{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:11:51Z","timestamp":1784178711891,"version":"3.55.0"},"reference-count":99,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T00:00:00Z","timestamp":1680566400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,7,31]]},"abstract":"<jats:p>\n            The\n            <jats:bold>Visual Question Answering (VQA)<\/jats:bold>\n            task utilizes both visual image and language analysis to answer a textual question with respect to an image. It has been a popular research topic with an increasing number of real-world applications in the last decade. This paper introduces a novel hierarchical integration of vision and language\n            <jats:bold>AliceMind-MMU (ALIbaba\u2019s Collection of Encoder-decoders from Machine IntelligeNce lab of Damo academy - MultiMedia Understanding)<\/jats:bold>\n            , which leads to similar or even slightly better results than a human being does on VQA. A hierarchical framework is designed to tackle the practical problems of VQA in a cascade manner including: (1) diverse visual semantics learning for comprehensive image content understanding; (2) enhanced multi-modal pre-training with modality adaptive attention; and (3) a knowledge-guided model integration with three specialized expert modules for the complex VQA task. Treating different types of visual questions with corresponding expertise needed plays an important role in boosting the performance of our VQA architecture up to the human level. An extensive set of experiments and analysis are conducted to demonstrate the effectiveness of the new research work.\n          <\/jats:p>","DOI":"10.1145\/3572833","type":"journal-article","created":{"date-parts":[[2022,12,20]],"date-time":"2022-12-20T07:23:07Z","timestamp":1671520987000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Achieving Human Parity on Visual Question Answering"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4959-8878","authenticated-orcid":false,"given":"Ming","family":"Yan","sequence":"first","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9442-5912","authenticated-orcid":false,"given":"Haiyang","family":"Xu","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9077-3928","authenticated-orcid":false,"given":"Chenliang","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1759-3255","authenticated-orcid":false,"given":"Junfeng","family":"Tian","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2207-9146","authenticated-orcid":false,"given":"Bin","family":"Bi","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7028-9845","authenticated-orcid":false,"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7498-1903","authenticated-orcid":false,"given":"Xianzhe","family":"Xu","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3835-7975","authenticated-orcid":false,"given":"Ji","family":"Zhang","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8084-0904","authenticated-orcid":false,"given":"Songfang","family":"Huang","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3709-5053","authenticated-orcid":false,"given":"Fei","family":"Huang","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3263-234X","authenticated-orcid":false,"given":"Luo","family":"Si","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8797-4646","authenticated-orcid":false,"given":"Rong","family":"Jin","sequence":"additional","affiliation":[{"name":"Alibaba DAMO Academy, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,4,4]]},"reference":[{"issue":"1","key":"e_1_3_3_2_2","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1007\/s11263-016-0966-6","article-title":"VQA: Visual question answering","volume":"123","author":"Agrawal Aishwarya","year":"2017","unstructured":"Aishwarya Agrawal, Jiasen Lu, Stanislaw Antol, Margaret Mitchell, C. Lawrence Zitnick, Devi Parikh, and Dhruv Batra. 2017. VQA: Visual question answering. Int. J. Comput. Vision 123, 1 (May 2017), 4\u201331.","journal-title":"Int. J. Comput. Vision"},{"key":"e_1_3_3_3_2","first-page":"6077","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Anderson Peter","year":"2018","unstructured":"Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6077\u20136086."},{"key":"e_1_3_3_4_2","first-page":"1247","volume-title":"International Conference on Machine Learning","author":"Andrew Galen","year":"2013","unstructured":"Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. 2013. Deep canonical correlation analysis. In International Conference on Machine Learning. PMLR, 1247\u20131255."},{"key":"e_1_3_3_5_2","first-page":"2425","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Antol Stanislaw","year":"2015","unstructured":"Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. 2015. VQA: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision. 2425\u20132433."},{"key":"e_1_3_3_6_2","article-title":"Scene text visual question answering","volume":"1905","author":"Biten Ali Furkan","year":"2019","unstructured":"Ali Furkan Biten, Rub\u00e8n Tito, Andr\u00e9s Mafla, Llu\u00eds G\u00f3mez, Mar\u00e7al Rusi\u00f1ol, Ernest Valveny, C. V. Jawahar, and Dimosthenis Karatzas. 2019. Scene text visual question answering. CoRR abs\/1905.13648 (2019). arxiv:1905.13648http:\/\/arxiv.org\/abs\/1905.13648.","journal-title":"CoRR"},{"key":"e_1_3_3_7_2","first-page":"6154","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Cai Zhaowei","year":"2018","unstructured":"Zhaowei Cai and Nuno Vasconcelos. 2018. Cascade R-CNN: Delving into high quality object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6154\u20136162."},{"key":"e_1_3_3_8_2","doi-asserted-by":"crossref","first-page":"1445","DOI":"10.1145\/2939672.2939812","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Cao Yue","year":"2016","unstructured":"Yue Cao, Mingsheng Long, Jianmin Wang, Qiang Yang, and Philip S. Yu. 2016. Deep visual-semantic hashing for cross-modal retrieval. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1445\u20131454."},{"key":"e_1_3_3_9_2","first-page":"3558","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Changpinyo Soravit","year":"2021","unstructured":"Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Conceptual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 3558\u20133568."},{"key":"e_1_3_3_10_2","article-title":"Reading Wikipedia to answer open-domain questions","author":"Chen Danqi","year":"2017","unstructured":"Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. arXiv preprint arXiv:1704.00051 (2017).","journal-title":"arXiv preprint arXiv:1704.00051"},{"key":"e_1_3_3_11_2","first-page":"12299","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Chen Hanting","year":"2021","unstructured":"Hanting Chen, Yunhe Wang, Tianyu Guo, Chang Xu, Yiping Deng, Zhenhua Liu, Siwei Ma, Chunjing Xu, Chao Xu, and Wen Gao. 2021. Pre-trained image processing transformer. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 12299\u201312310."},{"key":"e_1_3_3_12_2","first-page":"104","volume-title":"European Conference on Computer Vision","author":"Chen Yen-Chun","year":"2020","unstructured":"Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. 2020. UNITER: Universal image-text representation learning. In European Conference on Computer Vision. Springer, 104\u2013120."},{"issue":"3","key":"e_1_3_3_13_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3389547","article-title":"Robust unsupervised cross-modal hashing for multimedia retrieval","volume":"38","author":"Cheng Miaomiao","year":"2020","unstructured":"Miaomiao Cheng, Liping Jing, and Michael K. Ng. 2020. Robust unsupervised cross-modal hashing for multimedia retrieval. ACM Transactions on Information Systems (TOIS) 38, 3 (2020), 1\u201325.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"e_1_3_3_14_2","doi-asserted-by":"crossref","unstructured":"Yuhao Cui Zhou Yu Chunqi Wang Zhongzhou Zhao Ji Zhang Meng Wang and Jun Yu. 2021. ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration. arxiv:2108.07073 [cs.CV]","DOI":"10.1145\/3474085.3475251"},{"key":"e_1_3_3_15_2","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng Jia","year":"2009","unstructured":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 248\u2013255."},{"key":"e_1_3_3_16_2","first-page":"11162","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Desai Karan","year":"2021","unstructured":"Karan Desai and Justin Johnson. 2021. VirTex: Learning visual representations from textual annotations. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 11162\u201311173."},{"key":"e_1_3_3_17_2","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).","journal-title":"arXiv preprint arXiv:1810.04805"},{"key":"e_1_3_3_18_2","volume-title":"International Conference on Learning Representations","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et\u00a0al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations."},{"key":"e_1_3_3_19_2","article-title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity","author":"Fedus William","year":"2021","unstructured":"William Fedus, Barret Zoph, and Noam Shazeer. 2021. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961 (2021).","journal-title":"arXiv preprint arXiv:2101.03961"},{"issue":"3","key":"e_1_3_3_20_2","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1609\/aimag.v31i3.2303","article-title":"Building Watson: An overview of the DeepQA project","volume":"31","author":"Ferrucci David","year":"2010","unstructured":"David Ferrucci, Eric Brown, Jennifer Chu-Carroll, James Fan, David Gondek, Aditya A. Kalyanpur, Adam Lally, J. William Murdock, Eric Nyberg, John Prager, et\u00a0al. 2010. Building Watson: An overview of the DeepQA project. AI Magazine 31, 3 (2010), 59\u201379.","journal-title":"AI Magazine"},{"key":"e_1_3_3_21_2","article-title":"Multimodal compact bilinear pooling for visual question answering and visual grounding","author":"Fukui Akira","year":"2016","unstructured":"Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. 2016. Multimodal compact bilinear pooling for visual question answering and visual grounding. arXiv preprint arXiv:1606.01847 (2016).","journal-title":"arXiv preprint arXiv:1606.01847"},{"key":"e_1_3_3_22_2","volume-title":"NeurIPS","author":"Gan Zhe","year":"2020","unstructured":"Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. 2020. Large-scale adversarial training for vision-and-language representation learning. In NeurIPS."},{"key":"e_1_3_3_23_2","article-title":"Bilinear graph networks for visual question answering","author":"Guo Dalu","year":"2019","unstructured":"Dalu Guo, Chang Xu, and Dacheng Tao. 2019. Bilinear graph networks for visual question answering. arXiv preprint arXiv:1907.09815 (2019).","journal-title":"arXiv preprint arXiv:1907.09815"},{"key":"e_1_3_3_24_2","first-page":"75","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Guo Yangyang","year":"2019","unstructured":"Yangyang Guo, Zhiyong Cheng, Liqiang Nie, Yibing Liu, Yinglong Wang, and Mohan Kankanhalli. 2019. Quantifying and alleviating the language prior problem in visual question answering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 75\u201384."},{"key":"e_1_3_3_25_2","article-title":"REALM: Retrieval-augmented language model pre-training","author":"Guu Kelvin","year":"2020","unstructured":"Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. arXiv preprint arXiv:2002.08909 (2020).","journal-title":"arXiv preprint arXiv:2002.08909"},{"key":"e_1_3_3_26_2","first-page":"770","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770\u2013778."},{"key":"e_1_3_3_27_2","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.1145\/3404835.3462838","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Hong Weixiang","year":"2021","unstructured":"Weixiang Hong, Kaixiang Ji, Jiajia Liu, Jian Wang, Jingdong Chen, and Wei Chu. 2021. GilBERT: Generative vision-language pre-training for image-text retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1379\u20131388."},{"key":"e_1_3_3_28_2","first-page":"7132","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Hu Jie","year":"2018","unstructured":"Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132\u20137141."},{"key":"e_1_3_3_29_2","first-page":"635","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Hu Peng","year":"2019","unstructured":"Peng Hu, Liangli Zhen, Dezhong Peng, and Pei Liu. 2019. Scalable deep multimodal learning for cross-modal retrieval. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 635\u2013644."},{"key":"e_1_3_3_30_2","article-title":"Pixel-BERT: Aligning image pixels with text by deep multi-modal transformers","author":"Huang Zhicheng","year":"2020","unstructured":"Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu. 2020. Pixel-BERT: Aligning image pixels with text by deep multi-modal transformers. arXiv preprint arXiv:2004.00849 (2020).","journal-title":"arXiv preprint arXiv:2004.00849"},{"key":"e_1_3_3_31_2","first-page":"6700","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Hudson Drew A.","year":"2019","unstructured":"Drew A. Hudson and Christopher D. Manning. 2019. GQA: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6700\u20136709."},{"issue":"1","key":"e_1_3_3_32_2","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","article-title":"Adaptive mixtures of local experts","volume":"3","author":"Jacobs Robert A.","year":"1991","unstructured":"Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. 1991. Adaptive mixtures of local experts. Neural Computation 3, 1 (1991), 79\u201387.","journal-title":"Neural Computation"},{"key":"e_1_3_3_33_2","article-title":"Scaling up visual and vision-language representation learning with noisy text supervision","author":"Jia Chao","year":"2021","unstructured":"Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yunhsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. arXiv preprint arXiv:2102.05918 (2021).","journal-title":"arXiv preprint arXiv:2102.05918"},{"key":"e_1_3_3_34_2","first-page":"10267","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Jiang Huaizu","year":"2020","unstructured":"Huaizu Jiang, Ishan Misra, Marcus Rohrbach, Erik Learned-Miller, and Xinlei Chen. 2020. In defense of grid features for visual question answering. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10267\u201310276."},{"key":"e_1_3_3_35_2","volume-title":"Empirical Methods in Natural Language Processing (EMNLP)","author":"Karpukhin Vladimir","year":"2020","unstructured":"Vladimir Karpukhin, Barlas O\u011fuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Empirical Methods in Natural Language Processing (EMNLP)."},{"key":"e_1_3_3_36_2","article-title":"ViLT: Vision-and-language transformer without convolution or region supervision","author":"Kim Wonjae","year":"2021","unstructured":"Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. ViLT: Vision-and-language transformer without convolution or region supervision. arXiv preprint arXiv:2102.03334 (2021).","journal-title":"arXiv preprint arXiv:2102.03334"},{"issue":"1","key":"e_1_3_3_37_2","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1007\/s11263-016-0981-7","article-title":"Visual genome: Connecting language and vision using crowdsourced dense image annotations","volume":"123","author":"Krishna Ranjay","year":"2017","unstructured":"Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, et\u00a0al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International Journal of Computer Vision 123, 1 (2017), 32\u201373.","journal-title":"International Journal of Computer Vision"},{"key":"e_1_3_3_38_2","first-page":"785","volume-title":"Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing","author":"Lai Guokun","year":"2017","unstructured":"Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. RACE: Large-scale reading comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 785\u2013794."},{"key":"e_1_3_3_39_2","doi-asserted-by":"crossref","unstructured":"D. Lewis G. Agam S. Argamon O. Frieder D. Grossman and J. Heard. 2006. Building a test collection for complex document information processing(SIGIR\u201906). Association for Computing Machinery New York NY USA 665\u2013666. DOI:https:\/\/doi.org\/10.1145\/1148170.1148307","DOI":"10.1145\/1148170.1148307"},{"key":"e_1_3_3_40_2","article-title":"StructuralLM: Structural pre-training for form understanding","volume":"2105","author":"Li Chenliang","year":"2021","unstructured":"Chenliang Li, Bin Bi, Ming Yan, Wei Wang, Songfang Huang, Fei Huang, and Luo Si. 2021. StructuralLM: Structural pre-training for form understanding. CoRR abs\/2105.11210 (2021). arxiv:2105.11210https:\/\/arxiv.org\/abs\/2105.11210.","journal-title":"CoRR"},{"key":"e_1_3_3_41_2","article-title":"SemVLP: Vision-language pre-training by aligning semantics at multiple levels","author":"Li Chenliang","year":"2021","unstructured":"Chenliang Li, Ming Yan, Haiyang Xu, Fuli Luo, Wei Wang, Bin Bi, and Songfang Huang. 2021. SemVLP: Vision-language pre-training by aligning semantics at multiple levels. arXiv preprint arXiv:2103.07829 (2021).","journal-title":"arXiv preprint arXiv:2103.07829"},{"key":"e_1_3_3_42_2","first-page":"11336","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"34","author":"Li Gen","year":"2020","unstructured":"Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. 2020. Unicoder-VL: A universal encoder for vision and language by cross-modal pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 11336\u201311344."},{"key":"e_1_3_3_43_2","article-title":"UNIMO: Towards unified-modal understanding and generation via cross-modal contrastive learning","author":"Li Wei","year":"2020","unstructured":"Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, and Haifeng Wang. 2020. UNIMO: Towards unified-modal understanding and generation via cross-modal contrastive learning. arXiv preprint arXiv:2012.15409 (2020).","journal-title":"arXiv preprint arXiv:2012.15409"},{"key":"e_1_3_3_44_2","first-page":"121","volume-title":"European Conference on Computer Vision","author":"Li Xiujun","year":"2020","unstructured":"Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et\u00a0al. 2020. OSCAR: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision. Springer, 121\u2013137."},{"key":"e_1_3_3_45_2","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Liang Zujie","year":"2021","unstructured":"Zujie Liang, Haifeng Hu, and Jiaying Zhu. 2021. LPF: A language-prior feedback objective function for de-biased visual question answering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval."},{"key":"e_1_3_3_46_2","article-title":"InterBERT: Vision-and-language interaction for multi-modal pretraining","author":"Lin Junyang","year":"2020","unstructured":"Junyang Lin, An Yang, Yichang Zhang, Jie Liu, Jingren Zhou, and Hongxia Yang. 2020. InterBERT: Vision-and-language interaction for multi-modal pretraining. arXiv preprint arXiv:2003.13198 (2020).","journal-title":"arXiv preprint arXiv:2003.13198"},{"key":"e_1_3_3_47_2","first-page":"740","volume-title":"European Conference on Computer Vision","author":"Lin Tsung-Yi","year":"2014","unstructured":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll\u00e1r, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common objects in context. In European Conference on Computer Vision. Springer, 740\u2013755."},{"key":"e_1_3_3_48_2","first-page":"1379","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Liu Song","year":"2020","unstructured":"Song Liu, Shengsheng Qian, Yang Guan, Jiawei Zhan, and Long Ying. 2020. Joint-modal distribution-based similarity hashing for large-scale unsupervised deep cross-modal retrieval. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1379\u20131388."},{"key":"e_1_3_3_49_2","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019).","journal-title":"arXiv preprint arXiv:1907.11692"},{"key":"e_1_3_3_50_2","article-title":"Swin Transformer: Hierarchical vision transformer using shifted windows","author":"Liu Ze","year":"2021","unstructured":"Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. 2021. Swin Transformer: Hierarchical vision transformer using shifted windows. arXiv preprint arXiv:2103.14030 (2021).","journal-title":"arXiv preprint arXiv:2103.14030"},{"key":"e_1_3_3_51_2","first-page":"13","volume-title":"Advances in Neural Information Processing Systems","author":"Lu Jiasen","year":"2019","unstructured":"Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. ViLBERT: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. In Advances in Neural Information Processing Systems. 13\u201323."},{"key":"e_1_3_3_52_2","volume-title":"The IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Lu Jiasen","year":"2020","unstructured":"Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 2020. 12-in-1: Multi-task vision and language representation learning. In The IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_3_3_53_2","first-page":"289","article-title":"Hierarchical question-image co-attention for visual question answering","volume":"29","author":"Lu Jiasen","year":"2016","unstructured":"Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. 2016. Hierarchical question-image co-attention for visual question answering. Advances in Neural Information Processing Systems 29 (2016), 289\u2013297.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_54_2","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1145\/3397271.3401094","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"MacAvaney Sean","year":"2020","unstructured":"Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, and Ophir Frieder. 2020. Training curricula for open domain answer re-ranking. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 529\u2013538."},{"key":"e_1_3_3_55_2","first-page":"281","volume-title":"Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability","volume":"1","author":"MacQueen James","year":"1967","unstructured":"James MacQueen et\u00a0al. 1967. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1. Oakland, CA, USA, 281\u2013297."},{"key":"e_1_3_3_56_2","article-title":"MoVie: Revisiting modulated convolutions for visual counting and beyond","author":"Nguyen Duy-Kien","year":"2020","unstructured":"Duy-Kien Nguyen, Vedanuj Goswami, and Xinlei Chen. 2020. MoVie: Revisiting modulated convolutions for visual counting and beyond. arXiv preprint arXiv:2004.11883 (2020).","journal-title":"arXiv preprint arXiv:2004.11883"},{"key":"e_1_3_3_57_2","volume-title":"CoCo@ NIPS","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In CoCo@ NIPS."},{"key":"e_1_3_3_58_2","first-page":"464","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV)","author":"Pan Xingang","year":"2018","unstructured":"Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang. 2018. Two at once: Enhancing learning and generalization capacities via IBN-Net. In Proceedings of the European Conference on Computer Vision (ECCV). 464\u2013479."},{"key":"e_1_3_3_59_2","first-page":"539","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Qu Chen","year":"2020","unstructured":"Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W. Bruce Croft, and Mohit Iyyer. 2020. Open-retrieval conversational question answering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 539\u2013548."},{"key":"e_1_3_3_60_2","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Qu Chen","year":"2021","unstructured":"Chen Qu, Hamed Zamani, Liu Yang, W. Bruce Croft, and Erik Learned-Miller. 2021. Passage retrieval for outside-knowledge visual question answering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval."},{"key":"e_1_3_3_61_2","first-page":"1104","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Qu Leigang","year":"2021","unstructured":"Leigang Qu, Meng Liu, Jianlong Wu, Zan Gao, and Liqiang Nie. 2021. Dynamic modality interaction modeling for image-text retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1104\u20131113."},{"key":"e_1_3_3_62_2","article-title":"Learning transferable visual models from natural language supervision","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et\u00a0al. 2021. Learning transferable visual models from natural language supervision. arXiv preprint arXiv:2103.00020 (2021).","journal-title":"arXiv preprint arXiv:2103.00020"},{"key":"e_1_3_3_63_2","article-title":"SQuAD: 100,000+ questions for machine comprehension of text","author":"Rajpurkar Pranav","year":"2016","unstructured":"Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016).","journal-title":"arXiv preprint arXiv:1606.05250"},{"key":"e_1_3_3_64_2","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1145\/1873951.1873987","volume-title":"Proceedings of the 18th ACM International Conference on Multimedia","author":"Rasiwasia Nikhil","year":"2010","unstructured":"Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert R. G. Lanckriet, Roger Levy, and Nuno Vasconcelos. 2010. A new approach to cross-modal multimedia retrieval. In Proceedings of the 18th ACM International Conference on Multimedia. 251\u2013260."},{"key":"e_1_3_3_65_2","first-page":"91","article-title":"Faster R-CNN: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28 (2015), 91\u201399.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_66_2","first-page":"806","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","author":"Razavian Ali Sharif","year":"2014","unstructured":"Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN features off-the-shelf: An astounding baseline for recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 806\u2013813."},{"key":"e_1_3_3_67_2","doi-asserted-by":"crossref","first-page":"2160","DOI":"10.1109\/CVPR.2012.6247923","volume-title":"2012 IEEE Conference on Computer Vision and Pattern Recognition","author":"Sharma Abhishek","year":"2012","unstructured":"Abhishek Sharma, Abhishek Kumar, Hal Daume, and David W. Jacobs. 2012. Generalized multiview analysis: A discriminative latent space. In 2012 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2160\u20132167."},{"key":"e_1_3_3_68_2","doi-asserted-by":"crossref","first-page":"2556","DOI":"10.18653\/v1\/P18-1238","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Sharma Piyush","year":"2018","unstructured":"Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2556\u20132565."},{"key":"e_1_3_3_69_2","article-title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer","author":"Shazeer Noam","year":"2017","unstructured":"Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538 (2017).","journal-title":"arXiv preprint arXiv:1701.06538"},{"key":"e_1_3_3_70_2","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver David","year":"2016","unstructured":"David Silver, Aja Huang, Christopher J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484\u2013503.","journal-title":"Nature"},{"issue":"1","key":"e_1_3_3_71_2","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1145\/363707.363732","article-title":"Answering English questions by computer: A survey","volume":"8","author":"Simmons Robert F.","year":"1965","unstructured":"Robert F. Simmons. 1965. Answering English questions by computer: A survey. Commun. ACM 8, 1 (1965), 53\u201370.","journal-title":"Commun. ACM"},{"key":"e_1_3_3_72_2","article-title":"Towards VQA models that can read","volume":"1904","author":"Singh Amanpreet","year":"2019","unstructured":"Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards VQA models that can read. CoRR abs\/1904.08920 (2019). arxiv:1904.08920http:\/\/arxiv.org\/abs\/1904.08920.","journal-title":"CoRR"},{"key":"e_1_3_3_73_2","article-title":"VL-BERT: Pre-training of generic visual-linguistic representations","author":"Su Weijie","year":"2019","unstructured":"Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. 2019. VL-BERT: Pre-training of generic visual-linguistic representations. arXiv preprint arXiv:1908.08530 (2019).","journal-title":"arXiv preprint arXiv:1908.08530"},{"key":"e_1_3_3_74_2","article-title":"A corpus for reasoning about natural language grounded in photographs","author":"Suhr Alane","year":"2018","unstructured":"Alane Suhr, Stephanie Zhou, Ally Zhang, Iris Zhang, Huajun Bai, and Yoav Artzi. 2018. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491 (2018).","journal-title":"arXiv preprint arXiv:1811.00491"},{"key":"e_1_3_3_75_2","article-title":"LXMERT: Learning cross-modality encoder representations from transformers","author":"Tan Hao","year":"2019","unstructured":"Hao Tan and Mohit Bansal. 2019. LXMERT: Learning cross-modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 (2019).","journal-title":"arXiv preprint arXiv:1908.07490"},{"key":"e_1_3_3_76_2","first-page":"10347","volume-title":"International Conference on Machine Learning","author":"Touvron Hugo","year":"2021","unstructured":"Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Herv\u00e9 J\u00e9gou. 2021. Training data-efficient image transformers & distillation through attention. In International Conference on Machine Learning. PMLR, 10347\u201310357."},{"key":"e_1_3_3_77_2","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1145\/345508.345577","volume-title":"Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Voorhees Ellen M.","year":"2000","unstructured":"Ellen M. Voorhees and Dawn M. Tice. 2000. Building a question answering test collection. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 200\u2013207."},{"key":"e_1_3_3_78_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.18653\/v1\/W18-5446","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Wang Alex","year":"2018","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. Association for Computational Linguistics, Brussels, Belgium, 353\u2013355."},{"key":"e_1_3_3_79_2","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1145\/3331184.3331190","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang Bingning","year":"2019","unstructured":"Bingning Wang, Ting Yao, Qi Zhang, Jingfang Xu, Zhixing Tian, Kang Liu, and Jun Zhao. 2019. Document gated reader for open-domain question answering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 85\u201394."},{"key":"e_1_3_3_80_2","article-title":"A comprehensive survey on cross-modal retrieval","author":"Wang Kaiye","year":"2016","unstructured":"Kaiye Wang, Qiyue Yin, Wei Wang, Shu Wu, and Liang Wang. 2016. A comprehensive survey on cross-modal retrieval. arXiv preprint arXiv:1607.06215 (2016).","journal-title":"arXiv preprint arXiv:1607.06215"},{"key":"e_1_3_3_81_2","volume-title":"Thirty-Second AAAI Conference on Artificial Intelligence","author":"Wang Shuohang","year":"2018","unstructured":"Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, and Jing Jiang. 2018. R 3: Reinforced ranker-reader for open-domain question answering. In Thirty-Second AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_3_82_2","doi-asserted-by":"crossref","first-page":"746","DOI":"10.18653\/v1\/P18-2118","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)","author":"Wang Shuohang","year":"2018","unstructured":"Shuohang Wang, Mo Yu, Jing Jiang, and Shiyu Chang. 2018. A co-matching model for multi-choice reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 746\u2013751."},{"key":"e_1_3_3_83_2","article-title":"StructBERT: Incorporating language structures into pre-training for deep language understanding","author":"Wang Wei","year":"2019","unstructured":"Wei Wang, Bin Bi, Ming Yan, Chen Wu, Zuyi Bao, Jiangnan Xia, Liwei Peng, and Luo Si. 2019. StructBERT: Incorporating language structures into pre-training for deep language understanding. arXiv preprint arXiv:1908.04577 (2019).","journal-title":"arXiv preprint arXiv:1908.04577"},{"key":"e_1_3_3_84_2","doi-asserted-by":"crossref","first-page":"189","DOI":"10.18653\/v1\/P17-1018","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Wang Wenhui","year":"2017","unstructured":"Wenhui Wang, Nan Yang, Furu Wei, Baobao Chang, and Ming Zhou. 2017. Gated self-matching networks for reading comprehension and question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 189\u2013198."},{"key":"e_1_3_3_85_2","doi-asserted-by":"crossref","first-page":"2900","DOI":"10.1145\/3442381.3449798","volume-title":"Proceedings of the Web Conference 2021","author":"Wang Yongxin","year":"2021","unstructured":"Yongxin Wang, Zhen-Duo Chen, Xin Luo, and Xin-Shun Xu. 2021. High-dimensional sparse cross-modal hashing with fine-grained similarity embedding. In Proceedings of the Web Conference 2021. 2900\u20132909."},{"key":"e_1_3_3_86_2","article-title":"SimVLM: Simple visual language model pretraining with weak supervision","volume":"2108","author":"Wang Zirui","year":"2021","unstructured":"Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, and Yuan Cao. 2021. SimVLM: Simple visual language model pretraining with weak supervision. CoRR abs\/2108.10904 (2021).","journal-title":"CoRR"},{"key":"e_1_3_3_87_2","first-page":"1492","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Xie Saining","year":"2017","unstructured":"Saining Xie, Ross Girshick, Piotr Doll\u00e1r, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1492\u20131500."},{"key":"e_1_3_3_88_2","article-title":"E2E-VLP: End-to-end vision-language pre-training enhanced by visual learning","author":"Xu Haiyang","year":"2021","unstructured":"Haiyang Xu, Ming Yan, Chenliang Li, Bin Bi, Songfang Huang, Wenming Xiao, and Fei Huang. 2021. E2E-VLP: End-to-end vision-language pre-training enhanced by visual learning. arXiv preprint arXiv:2106.01804 (2021).","journal-title":"arXiv preprint arXiv:2106.01804"},{"key":"e_1_3_3_89_2","unstructured":"Ming Yan Haiyang Xu Chenliang Li Bin Bi Junfeng Tian Min Gui and Wei Wang. 2021. Grid-VLP: Revisiting Grid Features for Vision-Language Pre-training. arxiv:2108.09479 [cs.MM]"},{"key":"e_1_3_3_90_2","first-page":"21","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Yang Zichao","year":"2016","unstructured":"Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. 2016. Stacked attention networks for image question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 21\u201329."},{"key":"e_1_3_3_91_2","doi-asserted-by":"crossref","first-page":"2369","DOI":"10.18653\/v1\/D18-1259","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Yang Zhilin","year":"2018","unstructured":"Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2369\u20132380."},{"key":"e_1_3_3_92_2","first-page":"3208","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Yu Fei","year":"2021","unstructured":"Fei Yu, Jiji Tang, Weichong Yin, Yu Sun, Hao Tian, Hua Wu, and Haifeng Wang. 2021. ERNIE-ViL: Knowledge enhanced vision-language representations through scene graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 3208\u20133216."},{"key":"e_1_3_3_93_2","first-page":"1146","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Yu Tan","year":"2021","unstructured":"Tan Yu, Yi Yang, Yi Li, Lin Liu, Hongliang Fei, and Ping Li. 2021. Heterogeneous attention network for effective and efficient cross-modal retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1146\u20131156."},{"key":"e_1_3_3_94_2","first-page":"6281","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Yu Zhou","year":"2019","unstructured":"Zhou Yu, Jun Yu, Yuhao Cui, Dacheng Tao, and Qi Tian. 2019. Deep modular co-attention networks for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 6281\u20136290."},{"key":"e_1_3_3_95_2","first-page":"1821","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Yu Zhou","year":"2017","unstructured":"Zhou Yu, Jun Yu, Jianping Fan, and Dacheng Tao. 2017. Multi-modal factorized bilinear pooling with co-attention learning for visual question answering. In Proceedings of the IEEE International Conference on Computer Vision. 1821\u20131830."},{"key":"e_1_3_3_96_2","first-page":"5579","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zhang Pengchuan","year":"2021","unstructured":"Pengchuan Zhang, Xiujun Li, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, and Jianfeng Gao. 2021. VinVL: Revisiting visual representations in vision-language models. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 5579\u20135588."},{"key":"e_1_3_3_97_2","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhang Xinyu","year":"2021","unstructured":"Xinyu Zhang, Ke Zhan, Enrui Hu, Chengzhen Fu, Lan Luo, Hao Jiang, Yantao Jia, Fan Yu, Zhicheng Dou, et\u00a0al. 2021. Answer complex questions: Path ranker is all you need. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval."},{"key":"e_1_3_3_98_2","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1145\/3404835.3462853","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhang Yuyu","year":"2021","unstructured":"Yuyu Zhang, Ping Nie, Arun Ramamurthy, and Le Song. 2021. Answering any-hop open-domain questions with iterative document reranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 481\u2013490."},{"key":"e_1_3_3_99_2","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1145\/3331184.3331231","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zheng Yukun","year":"2019","unstructured":"Yukun Zheng, Jiaxin Mao, Yiqun Liu, Zixin Ye, Min Zhang, and Shaoping Ma. 2019. Human behavior inspired machine reading comprehension. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 425\u2013434."},{"key":"e_1_3_3_100_2","first-page":"4995","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Zhu Yuke","year":"2016","unstructured":"Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. 2016. Visual7w: Grounded question answering in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4995\u20135004."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3572833","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3572833","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:08Z","timestamp":1750182668000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3572833"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,4]]},"references-count":99,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2023,7,31]]}},"alternative-id":["10.1145\/3572833"],"URL":"https:\/\/doi.org\/10.1145\/3572833","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,4]]},"assertion":[{"value":"2022-01-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}