{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,31]],"date-time":"2026-08-31T08:35:41Z","timestamp":1788165341402,"version":"build-2803163510"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,6]],"date-time":"2022-07-06T00:00:00Z","timestamp":1657065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Beijing Academy of Artificial Intelligence(BAAI)"},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61872370"],"award-info":[{"award-number":["61872370"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"China Unicom Innovation Ecological Cooperation Plan"},{"name":"Beijing Outstanding Young Scientist Program","award":["BJJWZYJH012019100020098"],"award-info":[{"award-number":["BJJWZYJH012019100020098"]}]},{"name":"Intelligent Social Governance Platform?Major Innovation & Planning Interdisciplinary Platform for the ``Double-First Class' Initiative, Renmin University of China"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,6]]},"DOI":"10.1145\/3477495.3532086","type":"proceedings-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T11:12:08Z","timestamp":1657192328000},"page":"1502-1512","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Webformer"],"prefix":"10.1145","author":[{"given":"Yu","family":"Guo","sequence":"first","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengyi","family":"Ma","sequence":"additional","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiaxin","family":"Mao","sequence":"additional","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongjin","family":"Qian","sequence":"additional","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Distributed and Parallel Software Lab, Huawei, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Jiang","sequence":"additional","affiliation":[{"name":"Distributed and Parallel Software Lab, Huawei, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhao","family":"Cao","sequence":"additional","affiliation":[{"name":"Distributed and Parallel Software Lab, Huawei, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhicheng","family":"Dou","sequence":"additional","affiliation":[{"name":"Gaoling School of Artificial Intelligence, Renmin University of China & Beijing Key Laboratory of Big Data Management and Analysis Methods, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"HTLM: Hyper-Text Pre-Training and Prompting of Language Models. CoRR","author":"Aghajanyan Armen","year":"2021","unstructured":"Armen Aghajanyan , Dmytro Okhonko , Mike Lewis , Mandar Joshi , Hu Xu , Gargi Ghosh , and Luke Zettlemoyer . 2021 . HTLM: Hyper-Text Pre-Training and Prompting of Language Models. CoRR , Vol. abs\/ 2107 .06955 (2021). [arXiv]2107.06955 https:\/\/arxiv.org\/abs\/2107.06955 Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, and Luke Zettlemoyer. 2021. HTLM: Hyper-Text Pre-Training and Prompting of Language Models. CoRR, Vol. abs\/2107.06955 (2021). [arXiv]2107.06955 https:\/\/arxiv.org\/abs\/2107.06955"},{"key":"e_1_3_2_2_2_1","volume-title":"Proceedings of the World Congress on Engineering","volume":"3","author":"Al-Dallal Ammar","year":"2011","unstructured":"Ammar Al-Dallal and Rasha S Abdul-Wahab . 2011 . Achieving high recall and precision with HTLM documents: an innovation approach in information retrieval . In Proceedings of the World Congress on Engineering , Vol. 3 . Ammar Al-Dallal and Rasha S Abdul-Wahab. 2011. Achieving high recall and precision with HTLM documents: an innovation approach in information retrieval. In Proceedings of the World Congress on Engineering, Vol. 3."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.36"},{"key":"e_1_3_2_2_4_1","volume-title":"Longformer: The Long-Document Transformer. CoRR","author":"Beltagy Iz","year":"2020","unstructured":"Iz Beltagy , Matthew E. Peters , and Arman Cohan . 2020 . Longformer: The Long-Document Transformer. CoRR , Vol. abs\/ 2004 .05150 (2020). [arXiv]2004.05150 https:\/\/arxiv.org\/abs\/2004.05150 Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The Long-Document Transformer. CoRR, Vol. abs\/2004.05150 (2020). [arXiv]2004.05150 https:\/\/arxiv.org\/abs\/2004.05150"},{"key":"e_1_3_2_2_5_1","volume-title":"Large-scale pattern-based information extraction from the world wide web","author":"Blohm Sebastian","unstructured":"Sebastian Blohm . 2011. Large-scale pattern-based information extraction from the world wide web . KIT Scientific Publishing . Sebastian Blohm. 2011. Large-scale pattern-based information extraction from the world wide web. KIT Scientific Publishing."},{"key":"e_1_3_2_2_6_1","volume-title":"Pre-training Tasks for Embedding-based Large-scale Retrieval. In ICLR","author":"Chang Wei-Cheng","year":"2020","unstructured":"Wei-Cheng Chang , Felix X. Yu , Yin-Wen Chang , Yiming Yang , and Sanjiv Kumar . 2020 . Pre-training Tasks for Embedding-based Large-scale Retrieval. In ICLR 2020. OpenReview.net. Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020. Pre-training Tasks for Embedding-based Large-scale Retrieval. In ICLR 2020. OpenReview.net."},{"key":"e_1_3_2_2_7_1","volume-title":"ICLR","author":"Clark Kevin","year":"2020","unstructured":"Kevin Clark , Minh-Thang Luong , Quoc V. Le , and Christopher D. Manning . 2020. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators . In ICLR 2020 . OpenReview.net. Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In ICLR 2020. OpenReview.net."},{"key":"e_1_3_2_2_8_1","volume-title":"Overview of the TREC 2019 deep learning track.","author":"Craswell Nick","year":"2020","unstructured":"Nick Craswell , Bhaskar Mitra , Emine Yilmaz , Daniel Campos , and Ellen M. Voorhees . 2020 . Overview of the TREC 2019 deep learning track. ( 2020 ). Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. (2020)."},{"key":"e_1_3_2_2_9_1","volume-title":"USENIX Symposium on Internet Technologies and Systems. 241--252","author":"Cutler Michal","year":"1997","unstructured":"Michal Cutler , Yungming Shih , and Weiyi Meng . 1997 . Using the Structure of HTML Documents to Improve Retrieval .. In USENIX Symposium on Internet Technologies and Systems. 241--252 . Michal Cutler, Yungming Shih, and Weiyi Meng. 1997. Using the Structure of HTML Documents to Improve Retrieval.. In USENIX Symposium on Internet Technologies and Systems. 241--252."},{"key":"e_1_3_2_2_10_1","volume-title":"Deeper Text Understanding for IR with Contextual Neural Language Modeling. In SIGIR","author":"Dai Zhuyun","year":"2019","unstructured":"Zhuyun Dai and Jamie Callan . 2019 . Deeper Text Understanding for IR with Contextual Neural Language Modeling. In SIGIR 2019. ACM, 985--988. Zhuyun Dai and Jamie Callan. 2019. Deeper Text Understanding for IR with Contextual Neural Language Modeling. In SIGIR 2019. ACM, 985--988."},{"key":"e_1_3_2_2_11_1","volume-title":"Proceedings of the NAACL","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the NAACL 2019. ACL, 4171--4186. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the NAACL 2019. ACL, 4171--4186."},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72240-1_26"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983769"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023293820057"},{"key":"e_1_3_2_2_15_1","volume-title":"Reformer: The Efficient Transformer. In 8th International Conference on Learning Representations, ICLR 2020","author":"Kitaev Nikita","year":"2020","unstructured":"Nikita Kitaev , Lukasz Kaiser , and Anselm Levskaya . 2020 . Reformer: The Efficient Transformer. In 8th International Conference on Learning Representations, ICLR 2020 , Addis Ababa, Ethiopia , April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=rkgNKkHtvB Nikita Kitaev, Lukasz Kaiser, and Anselm Levskaya. 2020. Reformer: The Efficient Transformer. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=rkgNKkHtvB"},{"key":"e_1_3_2_2_16_1","volume-title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations, ICLR 2020","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan , Mingda Chen , Sebastian Goodman , Kevin Gimpel , Piyush Sharma , and Radu Soricut . 2020 . ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations, ICLR 2020 , Addis Ababa, Ethiopia , April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=H1eA7AEtvS Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net. https:\/\/openreview.net\/forum?id=H1eA7AEtvS"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1612"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462869"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462869"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482286"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-017-1097-2"},{"key":"e_1_3_2_2_22_1","volume-title":"WWW","author":"Mitra Bhaskar","year":"2017","unstructured":"Bhaskar Mitra , Fernando Diaz , and Nick Craswell . 2017 . Learning to Match using Local and Distributed Representations of Text for Web Search . In WWW 2017. Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to Match using Local and Distributed Representations of Text for Web Search. In WWW 2017."},{"key":"e_1_3_2_2_23_1","volume-title":"MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In NIPS","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen , Mir Rosenberg , Xia Song , 2016 . MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In NIPS 2016. Tri Nguyen, Mir Rosenberg, Xia Song, et al. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In NIPS 2016."},{"key":"e_1_3_2_2_24_1","volume-title":"Passage Re-ranking with BERT. CoRR","author":"Nogueira Rodrigo","year":"2019","unstructured":"Rodrigo Nogueira and Kyunghyun Cho . 2019. Passage Re-ranking with BERT. CoRR , Vol. abs\/ 1901 .04085 ( 2019 ). Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. CoRR, Vol. abs\/1901.04085 (2019)."},{"key":"e_1_3_2_2_25_1","volume-title":"Multi-Stage Document Ranking with BERT. CoRR","author":"Nogueira Rodrigo","year":"2019","unstructured":"Rodrigo Nogueira , Wei Yang , Kyunghyun Cho , and Jimmy Lin . 2019. Multi-Stage Document Ranking with BERT. CoRR , Vol. abs\/ 1910 .14424 ( 2019 ). Rodrigo Nogueira, Wei Yang, Kyunghyun Cho, and Jimmy Lin. 2019. Multi-Stage Document Ranking with BERT. CoRR, Vol. abs\/1910.14424 (2019)."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_2_27_1","volume-title":"Understanding the Behaviors of BERT in Ranking. CoRR","author":"Qiao Yifan","year":"2019","unstructured":"Yifan Qiao , Chenyan Xiong , Zhenghao Liu , and Zhiyuan Liu . 2019. Understanding the Behaviors of BERT in Ranking. CoRR , Vol. abs\/ 1904 .07531 ( 2019 ). showeprint[arXiv]1904.07531 http:\/\/arxiv.org\/abs\/1904.07531 Yifan Qiao, Chenyan Xiong, Zhenghao Liu, and Zhiyuan Liu. 2019. Understanding the Behaviors of BERT in Ranking. CoRR, Vol. abs\/1904.07531 (2019). showeprint[arXiv]1904.07531 http:\/\/arxiv.org\/abs\/1904.07531"},{"key":"e_1_3_2_2_28_1","volume-title":"Proceedings of Techinical re-port, OpenAI.","author":"Radford Alec","year":"2018","unstructured":"Alec Radford , Karthik Narasimhan , Tim Salimans , and Ilya Sutskever . 2018 a. Improving language understanding by generative pre-training . In Proceedings of Techinical re-port, OpenAI. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018a. Improving language understanding by generative pre-training. In Proceedings of Techinical re-port, OpenAI."},{"key":"e_1_3_2_2_29_1","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2018b. Language Models are Unsupervised Multitask Learners. (2018).  Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2018b. Language Models are Unsupervised Multitask Learners. (2018)."},{"key":"e_1_3_2_2_30_1","volume-title":"et almbox","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , Ilya Sutskever , et almbox . 2019 . Language models are unsupervised multitask learners. OpenAI blog, Vol. 1 , 8 (2019), 9. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et almbox. 2019. Language models are unsupervised multitask learners. OpenAI blog, Vol. 1, 8 (2019), 9."},{"key":"e_1_3_2_2_31_1","volume-title":"Robertson and Steve Walker","author":"Stephen","year":"1994","unstructured":"Stephen E. Robertson and Steve Walker . 1994 . Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval. In Proceedings of the SIGIR 1994. ACM\/Springer , 232--241. Stephen E. Robertson and Steve Walker. 1994. Some Simple Effective Approximations to the 2-Poisson Model for Probabilistic Weighted Retrieval. In Proceedings of the SIGIR 1994. ACM\/Springer, 232--241."},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462872"},{"key":"e_1_3_2_2_33_1","volume-title":"NIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N. Gomez , Lukasz Kaiser , and Illia Polosukhin . 2017 . Attention is All you Need . In NIPS 2017. 5998--6008. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NIPS 2017. 5998--6008."},{"key":"e_1_3_2_2_34_1","volume-title":"Linformer: Self-Attention with Linear Complexity. CoRR","author":"Wang Sinong","year":"2020","unstructured":"Sinong Wang , Belinda Z. Li , Madian Khabsa , Han Fang , and Hao Ma . 2020 . Linformer: Self-Attention with Linear Complexity. CoRR , Vol. abs\/ 2006 .04768 (2020). showeprint[arXiv]2006.04768 https:\/\/arxiv.org\/abs\/2006.04768 Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-Attention with Linear Complexity. CoRR, Vol. abs\/2006.04768 (2020). showeprint[arXiv]2006.04768 https:\/\/arxiv.org\/abs\/2006.04768"},{"key":"e_1_3_2_2_35_1","volume-title":"Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models. In SIGIR","author":"Wei Mengxi","year":"2020","unstructured":"Mengxi Wei , Yifan He , and Qiong Zhang . 2020 . Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models. In SIGIR 2020. Mengxi Wei, Yifan He, and Qiong Zhang. 2020. Robust Layout-aware IE for Visually Rich Documents with Pre-trained Language Models. In SIGIR 2020."},{"key":"e_1_3_2_2_36_1","volume-title":"Morgan Funtowicz, and Jamie Brew.","author":"Wolf Thomas","year":"2019","unstructured":"Thomas Wolf , Lysandre Debut , Victor Sanh , Julien Chaumond , Clement Delangue , Anthony Moi , Pierric Cistac , Tim Rault , R\u00e9 mi Louf , Morgan Funtowicz, and Jamie Brew. 2019 . HuggingFace's Transformers: State-of-the-art Natural Language Processing. CoRR , Vol. abs\/ 1910 .03771 (2019). [arXiv]1910.03771 http:\/\/arxiv.org\/abs\/1910.03771 Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R\u00e9 mi Louf, Morgan Funtowicz, and Jamie Brew. 2019. HuggingFace's Transformers: State-of-the-art Natural Language Processing. CoRR, Vol. abs\/1910.03771 (2019). [arXiv]1910.03771 http:\/\/arxiv.org\/abs\/1910.03771"},{"key":"e_1_3_2_2_37_1","volume-title":"End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In SIGIR","author":"Xiong Chenyan","year":"2017","unstructured":"Chenyan Xiong , Zhuyun Dai , Jamie Callan , Zhiyuan Liu , and Russell Power . 2017 . End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In SIGIR 2017. 55--64. Chenyan Xiong, Zhuyun Dai, Jamie Callan, Zhiyuan Liu, and Russell Power. 2017. End-to-End Neural Ad-hoc Ranking with Kernel Pooling. In SIGIR 2017. 55--64."},{"key":"e_1_3_2_2_38_1","unstructured":"Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu etal 2020. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. CoRR Vol. abs\/2007.00808 (2020).  Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu et al. 2020. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. CoRR Vol. abs\/2007.00808 (2020)."},{"key":"e_1_3_2_2_39_1","volume-title":"2019 c. Simple Applications of BERT for Ad Hoc Document Retrieval. CoRR","author":"Yang Wei","year":"2019","unstructured":"Wei Yang , Haotian Zhang , and Jimmy Lin . 2019 c. Simple Applications of BERT for Ad Hoc Document Retrieval. CoRR , Vol. abs\/ 1903 .10972 ( 2019 ). Wei Yang, Haotian Zhang, and Jimmy Lin. 2019 c. Simple Applications of BERT for Ad Hoc Document Retrieval. CoRR, Vol. abs\/1903.10972 (2019)."},{"key":"e_1_3_2_2_40_1","volume-title":"2019 a. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le . 2019 a. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems , Vol. 32 ( 2019 ). Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019 a. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, Vol. 32 (2019)."},{"key":"e_1_3_2_2_41_1","first-page":"5754","article-title":"b","volume":"2019","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime G. Carbonell , Ruslan Salakhutdinov , and Quoc V. Le . 2019 b . XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NIPS 2019. 5754 -- 5764 . Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019 b. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NIPS 2019. 5754--5764.","journal-title":"XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NIPS"},{"key":"e_1_3_2_2_42_1","volume-title":"Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al.","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer , Guru Guruganesh , Kumar Avinava Dubey , Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020 . Big Bird : Transformers for Longer Sequences.. In NeurIPS. Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, et al. 2020. Big Bird: Transformers for Longer Sequences.. In NeurIPS."},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130348.3130377"},{"key":"e_1_3_2_2_44_1","volume-title":"Socialformer: Social Network Inspired Long Document Modeling for Document Ranking. CoRR","author":"Zhou Yujia","year":"2022","unstructured":"Yujia Zhou , Zhicheng Dou , Huaying Yuan , and Zhengyi Ma . 2022 . Socialformer: Social Network Inspired Long Document Modeling for Document Ranking. CoRR , Vol. abs\/ 2202 .10870 (2022). Yujia Zhou, Zhicheng Dou, Huaying Yuan, and Zhengyi Ma. 2022. Socialformer: Social Network Inspired Long Document Modeling for Document Ranking. CoRR, Vol. abs\/2202.10870 (2022)."},{"key":"e_1_3_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482379"},{"key":"e_1_3_2_2_46_1","volume-title":"Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event","author":"Zhu Yutao","year":"2021","unstructured":"Yutao Zhu , Jian-Yun Nie , Zhicheng Dou , Zhengyi Ma , Xinyu Zhang , Pan Du , Xiaochen Zuo , and Hao Jiang . 2021 a . Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event , Queensland, Australia, November 1 - 5 , 2021,, Gianluca Demartini, Guido Zuccon, J. Shane Culpepper, Zi Huang, and Hanghang Tong (Eds.). ACM, 2780--2791. https:\/\/doi.org\/10.1145\/3459637.3482243 10.1145\/3459637.3482243 Yutao Zhu, Jian-Yun Nie, Zhicheng Dou, Zhengyi Ma, Xinyu Zhang, Pan Du, Xiaochen Zuo, and Hao Jiang. 2021 a. Contrastive Learning of User Behavior Sequence for Context-Aware Document Ranking. In CIKM '21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021,, Gianluca Demartini, Guido Zuccon, J. Shane Culpepper, Zi Huang, and Hanghang Tong (Eds.). ACM, 2780--2791. https:\/\/doi.org\/10.1145\/3459637.3482243"},{"key":"e_1_3_2_2_47_1","volume-title":"AAAI","author":"Zhu Yutao","year":"2021","unstructured":"Yutao Zhu , Kun Zhou , Jian-Yun Nie , Shengchao Liu , and Zhicheng Dou . 2021 b. Neural Sentence Ordering Based on Constraint Graphs . In AAAI 2021 . AAAI Press , 14656--14664. https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/17722 Yutao Zhu, Kun Zhou, Jian-Yun Nie, Shengchao Liu, and Zhicheng Dou. 2021 b. Neural Sentence Ordering Based on Constraint Graphs. In AAAI 2021. AAAI Press, 14656--14664. https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/17722"}],"event":{"name":"SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","location":"Madrid Spain","acronym":"SIGIR '22","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477495.3532086","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477495.3532086","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T14:10:36Z","timestamp":1750169436000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477495.3532086"}},"subtitle":["Pre-training with Web Pages for Information Retrieval"],"short-title":[],"issued":{"date-parts":[[2022,7,6]]},"references-count":47,"alternative-id":["10.1145\/3477495.3532086","10.1145\/3477495"],"URL":"https:\/\/doi.org\/10.1145\/3477495.3532086","relation":{},"subject":[],"published":{"date-parts":[[2022,7,6]]},"assertion":[{"value":"2022-07-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}