{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T18:31:17Z","timestamp":1774463477261,"version":"3.50.1"},"reference-count":107,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,6,1]],"date-time":"2021-06-01T00:00:00Z","timestamp":1622505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:p>When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead. Classical information retrieval systems do not answer information needs directly, but instead provide references to (hopefully authoritative) answers. Successful question answering systems offer a limited corpus created on-demand by human experts, which is neither timely nor scalable. Pre-trained language models, by contrast, are capable of directly generating prose that may be responsive to an information need, but at present they are dilettantes rather than domain experts - they do not have a true understanding of the world, they are prone to hallucinating, and crucially they are incapable of justifying their utterances by referring to supporting documents in the corpus they were trained over. This paper examines how ideas from classical information retrieval and pre-trained language models can be synthesized and evolved into systems that truly deliver on the promise of domain expert advice.<\/jats:p>","DOI":"10.1145\/3476415.3476428","type":"journal-article","created":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T16:07:00Z","timestamp":1626451620000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":84,"title":["Rethinking search"],"prefix":"10.1145","volume":"55","author":[{"given":"Donald","family":"Metzler","sequence":"first","affiliation":[{"name":"Google Research"}]},{"given":"Yi","family":"Tay","sequence":"additional","affiliation":[{"name":"Google Research"}]},{"given":"Dara","family":"Bahri","sequence":"additional","affiliation":[{"name":"Google Research"}]},{"given":"Marc","family":"Najork","sequence":"additional","affiliation":[{"name":"Google Research"}]}],"member":"320","published-online":{"date-parts":[[2021,7,16]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Towards a humanlike open-domain chatbot. arXiv preprint arXiv:2001.09977","author":"Adiwardana Daniel","year":"2020","unstructured":"Daniel Adiwardana , Minh-Thang Luong , David R So , Jamie Hall , Noah Fiedel , Romal Thoppilan , Zi Yang , Apoorv Kulshreshtha , Gaurav Nemade , Yifeng Lu , and Quoc V Le . Towards a humanlike open-domain chatbot. arXiv preprint arXiv:2001.09977 , 2020 . Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V Le. Towards a humanlike open-domain chatbot. arXiv preprint arXiv:2001.09977, 2020."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 8th International Conference on Learning Representations, ICLR '20","author":"Asai Akari","year":"2020","unstructured":"Akari Asai , Kazuma Hashimoto , Hannaneh Hajishirzi , Richard Socher , and Caiming Xiong . Learning to retrieve reasoning paths over wikipedia graph for question answering . In Proceedings of the 8th International Conference on Learning Representations, ICLR '20 , 2020 . Akari Asai, Kazuma Hashimoto, Hannaneh Hajishirzi, Richard Socher, and Caiming Xiong. Learning to retrieve reasoning paths over wikipedia graph for question answering. In Proceedings of the 8th International Conference on Learning Representations, ICLR '20, 2020."},{"key":"e_1_2_1_3_1","volume-title":"Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150","author":"Beltagy Iz","year":"2020","unstructured":"Iz Beltagy , Matthew E Peters , and Arman Cohan . Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 , 2020 . Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1038\/scientificamerican0501-34"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376746"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0169-7552(98)00110-X"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/792550.792552"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/347319.346290"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the 34th Conference on Neural Information Processing Systems, NeurIPS '20","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown , Benjamin Mann , Nick Ryder , Melanie Subbiah , Jared Kaplan , Prafulla Dhariwal , Arvind Neelakantan , Pranav Shyam , Girish Sastry , Amanda Askell , Sandhini Agarwal , Ariel Herbert-Voss , Gretchen Krueger , Tom Henighan , Rewon Child , Aditya Ramesh , Daniel M. Ziegler , Jeffrey Wu , Clemens Winter , Christopher Hesse , Mark Chen , Eric Sigler , Mateusz Litwin , Scott Gray , Benjamin Chess , Jack Clark , Christopher Berner , Sam McCandlish , Alec Radford , Ilya Sutskever , and Dario Amodei . Language models are few-shot learners . In Proceedings of the 34th Conference on Neural Information Processing Systems, NeurIPS '20 , 2020 . Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Proceedings of the 34th Conference on Neural Information Processing Systems, NeurIPS '20, 2020."},{"key":"e_1_2_1_11_1","volume-title":"Cross-document language modeling. arXiv preprint arXiv:2101.00406","author":"Caciularu Avi","year":"2021","unstructured":"Avi Caciularu , Arman Cohan , Iz Beltagy , Matthew E Peters , Arie Cattan , and Ido Dagan . Cross-document language modeling. arXiv preprint arXiv:2101.00406 , 2021 . Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E Peters, Arie Cattan, and Ido Dagan. Cross-document language modeling. arXiv preprint arXiv:2101.00406, 2021."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718501"},{"key":"e_1_2_1_13_1","volume-title":"Multi-hop question answering via reasoning chains. arXiv preprint arXiv:1910.02610","author":"Chen Jifan","year":"2019","unstructured":"Jifan Chen , Shih-ting Lin, and Greg Durrett . Multi-hop question answering via reasoning chains. arXiv preprint arXiv:1910.02610 , 2019 . Jifan Chen, Shih-ting Lin, and Greg Durrett. Multi-hop question answering via reasoning chains. arXiv preprint arXiv:1910.02610, 2019."},{"key":"e_1_2_1_14_1","volume-title":"Unifying vision-and-language tasks via text generation. arXiv preprint arXiv:2102.02779","author":"Cho Jaemin","year":"2021","unstructured":"Jaemin Cho , Jie Lei , Hao Tan , and Mohit Bansal . Unifying vision-and-language tasks via text generation. arXiv preprint arXiv:2102.02779 , 2021 . Jaemin Cho, Jie Lei, Hao Tan, and Mohit Bansal. Unifying vision-and-language tasks via text generation. arXiv preprint arXiv:2102.02779, 2021."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1078"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/1516224"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 7th International Conference on Learning Representations, ICLR '19","author":"Das Rajarshi","year":"2019","unstructured":"Rajarshi Das , Shehzaad Dhuliawala , Manzil Zaheer , and Andrew McCallum . Multi-step retriever-reader interaction for scalable open-domain question answering . In Proceedings of the 7th International Conference on Learning Representations, ICLR '19 , 2019 . Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, and Andrew McCallum. Multi-step retriever-reader interaction for scalable open-domain question answering. In Proceedings of the 7th International Conference on Learning Representations, ICLR '19, 2019."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Cao Nicola De","year":"2019","unstructured":"Nicola De Cao , Wilker Aziz , and Ivan Titov . Question answering by reasoning across documents with graph convolutional networks . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers), NAACL-HLT '19, pages 2306--2317 , 2019 . Nicola De Cao, Wilker Aziz, and Ivan Titov. Question answering by reasoning across documents with graph convolutional networks. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), NAACL-HLT '19, pages 2306--2317, 2019."},{"key":"e_1_2_1_19_1","article-title":"A continual learning survey: Defying forgetting in classification tasks","author":"Lange Matthias De","year":"2021","unstructured":"Matthias De Lange , Rahaf Aljundi , Marc Masana , Sarah Parisot , Xu Jia , Ales Leonardis , Gregory Slabaugh , and Tinne Tuytelaars . A continual learning survey: Defying forgetting in classification tasks . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 . Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . Bert : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers), NAACL-HLT '18, pages 4171--4186 , 2018 . Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), NAACL-HLT '18, pages 4171--4186, 2018."},{"key":"e_1_2_1_21_1","volume-title":"Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904","author":"Dhingra Bhuwan","year":"2017","unstructured":"Bhuwan Dhingra , Kathryn Mazaitis , and William W Cohen . Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904 , 2017 . Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904, 2017."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623623"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/2777598.2777603"},{"key":"e_1_2_1_24_1","volume-title":"An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy , Lucas Beyer , Alexander Kolesnikov , Dirk Weissenborn , Xiaohua Zhai , Thomas Unterthiner , Mostafa Dehghani , Matthias Minderer , Georg Heigold , Sylvain Gelly , Jakob Uszkoreit , and Neil Houlsby . An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 , 2020 . Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020."},{"key":"e_1_2_1_25_1","volume-title":"SearchQA: A new Q&A dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179","author":"Dunn Matthew","year":"2017","unstructured":"Matthew Dunn , Levent Sagun , Mike Higgins , V Ugur G\u00fcney , Volkan Cirik , and Kyunghyun Cho . SearchQA: A new Q&A dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179 , 2017 . Matthew Dunn, Levent Sagun, Mike Higgins, V Ugur G\u00fcney, Volkan Cirik, and Kyunghyun Cho. SearchQA: A new Q&A dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179, 2017."},{"key":"e_1_2_1_26_1","volume-title":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961","author":"Fedus William","year":"2021","unstructured":"William Fedus , Barret Zoph , and Noam Shazeer . Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961 , 2021 . William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961, 2021."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1364-6613(99)01294-2"},{"key":"e_1_2_1_28_1","first-page":"146","volume-title":"Proceedings of the 43rd European Conference on IR Research, ECIR '21","author":"Gao Luyu","year":"2021","unstructured":"Luyu Gao , Zhuyun Dai , Tongfei Chen , Zhen Fan , Benjamin Van Durme , and Jamie Callan . Complement lexical retrieval model with semantic residual embeddings . In Proceedings of the 43rd European Conference on IR Research, ECIR '21 , pages 146 -- 160 , 2021 . Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. Complement lexical retrieval model with semantic residual embeddings. In Proceedings of the 43rd European Conference on IR Research, ECIR '21, pages 146--160, 2021."},{"key":"e_1_2_1_29_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations, ICLR '15","author":"Goodfellow Ian J","year":"2015","unstructured":"Ian J Goodfellow , Jonathon Shlens , and Christian Szegedy . Explaining and harnessing adversarial examples . In Proceedings of the 3rd International Conference on Learning Representations, ICLR '15 , 2015 . Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In Proceedings of the 3rd International Conference on Learning Representations, ICLR '15, 2015."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature20101"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983769"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1181"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1110"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159689"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.487"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.164"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380231"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1147"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/324133.324140"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00023"},{"key":"e_1_2_1_44_1","volume-title":"Leveraging semantic and lexical matching to improve the recall of document retrieval systems: A hybrid approach. arXiv preprint arXiv:2010.01195","author":"Kuzi Saar","year":"2020","unstructured":"Saar Kuzi , Mingyang Zhang , Cheng Li , Michael Bendersky , and Marc Najork . Leveraging semantic and lexical matching to improve the recall of document retrieval systems: A hybrid approach. arXiv preprint arXiv:2010.01195 , 2020 . Saar Kuzi, Mingyang Zhang, Cheng Li, Michael Bendersky, and Marc Najork. Leveraging semantic and lexical matching to improve the recall of document retrieval systems: A hybrid approach. arXiv preprint arXiv:2010.01195, 2020."},{"key":"e_1_2_1_45_1","volume-title":"Interpretability and Robustness in Audio, Speech, and Language Workshop at NeurIPS","author":"Lee Katherine","year":"2018","unstructured":"Katherine Lee , Orhan Firat , Ashish Agarwal , Clara Fannjiang , and David Sussillo . Hallucinations in neural machine translation . In Interpretability and Robustness in Audio, Speech, and Language Workshop at NeurIPS 2018 , 2018. Katherine Lee, Orhan Firat, Ashish Agarwal, Clara Fannjiang, and David Sussillo. Hallucinations in neural machine translation. In Interpretability and Robustness in Audio, Speech, and Language Workshop at NeurIPS 2018, 2018."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1612"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.2200\/S00607ED2V01Y201410HLT026"},{"key":"e_1_2_1_49_1","volume-title":"Distilling dense representations for ranking using tightly-coupled teachers. arXiv preprint arXiv:2010.11386","author":"Lin Sheng-Chieh","year":"2020","unstructured":"Sheng-Chieh Lin , Jheng-Hong Yang , and Jimmy Lin . Distilling dense representations for ranking using tightly-coupled teachers. arXiv preprint arXiv:2010.11386 , 2020 . Sheng-Chieh Lin, Jheng-Hong Yang, and Jimmy Lin. Distilling dense representations for ranking using tightly-coupled teachers. arXiv preprint arXiv:2010.11386, 2020."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000016"},{"key":"e_1_2_1_51_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019 . Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692, 2019."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295377"},{"key":"e_1_2_1_53_1","volume-title":"Caiming Xiong, and Richard Socher. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730","author":"McCann Bryan","year":"2018","unstructured":"Bryan McCann , Nitish Shirish Keskar , Caiming Xiong, and Richard Socher. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730 , 2018 . Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. The natural language decathlon: Multitask learning as question answering. arXiv preprint arXiv:1806.08730, 2018."},{"key":"e_1_2_1_54_1","volume-title":"Proceedings of the 6th International Conference on Learning Representations, ICLR '18","author":"Mena Gonzalo","year":"2018","unstructured":"Gonzalo Mena , David Belanger , Scott Linderman , and Jasper Snoek . Learning latent permutations with gumbel-sinkhorn networks . In Proceedings of the 6th International Conference on Learning Representations, ICLR '18 , 2018 . Gonzalo Mena, David Belanger, Scott Linderman, and Jasper Snoek. Learning latent permutations with gumbel-sinkhorn networks. In Proceedings of the 6th International Conference on Learning Representations, ICLR '18, 2018."},{"key":"e_1_2_1_55_1","volume-title":"International Conference on Learning Representations, ICLR '13","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov , Kai Chen , Greg S. Corrado , and Jeffrey Dean . Efficient estimation of word representations in vector space . In International Conference on Learning Representations, ICLR '13 , 2013 a. Tomas Mikolov, Kai Chen, Greg S. Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space. In International Conference on Learning Representations, ICLR '13, 2013a."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999792.2999959"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1147"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3191513"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000061"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052579"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.5555\/1859417"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-017-9321-y"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1244"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.01.012"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1493"},{"key":"e_1_2_1_68_1","volume-title":"Understand your world with bing","author":"Qian Richard","year":"2013","unstructured":"Richard Qian . Understand your world with bing , 2013 . URL https:\/\/blogs.bing.com\/search\/2013\/03\/21\/understand-your-world-with-bing. Richard Qian. Understand your world with bing, 2013. URL https:\/\/blogs.bing.com\/search\/2013\/03\/21\/understand-your-world-with-bing."},{"key":"e_1_2_1_69_1","volume-title":"Language models are unsupervised multitask learners","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . Language models are unsupervised multitask learners . 2019 . Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019."},{"issue":"140","key":"e_1_2_1_70_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J Liu . Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Learning Research , 21 ( 140 ): 1 -- 67 , 2020 . Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1--67, 2020.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1251"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1540"},{"key":"e_1_2_1_74_1","volume-title":"Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics","volume":"1","author":"Liu Peter J","year":"2017","unstructured":"Abigail See, Peter J Liu , and Christopher D Manning . Get to the point: Summarization with pointer-generator networks . In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics , Volume 1 (Long Papers), ACL '17, pages 1073--1083 , 2017 . Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Volume 1 (Long Papers), ACL '17, pages 1073--1083, 2017."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767738"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1339"},{"key":"e_1_2_1_77_1","volume-title":"Introducing the Knowledge Graph: things, not strings","author":"Singhal Amit","year":"2012","unstructured":"Amit Singhal . Introducing the Knowledge Graph: things, not strings , 2012 . URL https:\/\/blog.google\/products\/search\/introducing-knowledge-graph-things-not\/. Amit Singhal. Introducing the Knowledge Graph: things, not strings, 2012. URL https:\/\/blog.google\/products\/search\/introducing-knowledge-graph-things-not\/."},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242667"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305890.3306024"},{"key":"e_1_2_1_80_1","first-page":"5940","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18","author":"Tan Chuanqi","year":"2018","unstructured":"Chuanqi Tan , Furu Wei , Nan Yang , Bowen Du , Weifeng Lv , and Ming Zhou . S-net : From answer extraction to answer generation for machine reading comprehension . In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18 , pages 5940 -- 5947 , 2018 . Chuanqi Tan, Furu Wei, Nan Yang, Bowen Du, Weifeng Lv, and Ming Zhou. S-net: From answer extraction to answer generation for machine reading comprehension. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18, pages 5940--5947, 2018."},{"key":"e_1_2_1_81_1","volume-title":"Lstm-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108","author":"Tan Ming","year":"2015","unstructured":"Ming Tan , Cicero dos Santos , Bing Xiang , and Bowen Zhou . Lstm-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108 , 2015 . Ming Tan, Cicero dos Santos, Bing Xiang, and Bowen Zhou. Lstm-based deep learning models for non-factoid answer selection. arXiv preprint arXiv:1511.04108, 2015."},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220048"},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1185"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.5555\/3327345.3327399"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1486"},{"key":"e_1_2_1_86_1","volume-title":"Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006","author":"Tay Yi","year":"2020","unstructured":"Yi Tay , Mostafa Dehghani , Samira Abnar , Yikang Shen , Dara Bahri , Philip Pham , Jinfeng Rao , Liu Yang , Sebastian Ruder , and Donald Metzler . Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006 , 2020 a. Yi Tay, Mostafa Dehghani, Samira Abnar, Yikang Shen, Dara Bahri, Philip Pham, Jinfeng Rao, Liu Yang, Sebastian Ruder, and Donald Metzler. Long range arena: A benchmark for efficient transformers. arXiv preprint arXiv:2011.04006, 2020a."},{"key":"e_1_2_1_87_1","volume-title":"Efficient transformers: A survey. arXiv preprint arXiv:2009.06732","author":"Tay Yi","year":"2020","unstructured":"Yi Tay , Mostafa Dehghani , Dara Bahri , and Donald Metzler . Efficient transformers: A survey. arXiv preprint arXiv:2009.06732 , 2020 b. Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. Efficient transformers: A survey. arXiv preprint arXiv:2009.06732, 2020b."},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-2623"},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969540"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629489"},{"key":"e_1_2_1_92_1","first-page":"22","volume-title":"Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '07","author":"Wang Mengqiu","year":"2007","unstructured":"Mengqiu Wang , Noah A Smith , and Teruko Mitamura . What is the jeopardy model? a quasi-synchronous grammar for qa . In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '07 , pages 22 -- 32 , 2007 . Mengqiu Wang, Noah A Smith, and Teruko Mitamura. What is the jeopardy model? a quasi-synchronous grammar for qa. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '07, pages 22--32, 2007."},{"key":"e_1_2_1_93_1","volume-title":"Proceedings of the 5th International Conference on Learning Representations, ICLR '17","author":"Wang Shuohang","year":"2017","unstructured":"Shuohang Wang and Jing Jiang . A compare-aggregate model for matching text sequences . In Proceedings of the 5th International Conference on Learning Representations, ICLR '17 , 2017 a. Shuohang Wang and Jing Jiang. A compare-aggregate model for matching text sequences. In Proceedings of the 5th International Conference on Learning Representations, ICLR '17, 2017a."},{"key":"e_1_2_1_94_1","volume-title":"Proceedings of the 5th International Conference on Learning Representations, ICLR '17","author":"Wang Shuohang","year":"2017","unstructured":"Shuohang Wang and Jing Jiang . Machine comprehension using match-lstm and answer pointer . In Proceedings of the 5th International Conference on Learning Representations, ICLR '17 , 2017 b. Shuohang Wang and Jing Jiang. Machine comprehension using match-lstm and answer pointer. In Proceedings of the 5th International Conference on Learning Representations, ICLR '17, 2017b."},{"key":"e_1_2_1_95_1","first-page":"5981","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18","author":"Wang Shuohang","year":"2018","unstructured":"Shuohang Wang , Mo Yu , Xiaoxiao Guo , Zhiguo Wang , Tim Klinger , Wei Zhang , Shiyu Chang , Gerry Tesauro , Bowen Zhou , and Jing Jiang . R3 : Reinforced ranker-reader for open-domain question answering . In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18 , pages 5981 -- 5988 , 2018 a. Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerry Tesauro, Bowen Zhou, and Jing Jiang. R3: Reinforced ranker-reader for open-domain question answering. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI '18, pages 5981--5988, 2018a."},{"key":"e_1_2_1_96_1","volume-title":"Dataset distillation. arXiv preprint arXiv:1811.10959","author":"Wang Tongzhou","year":"2018","unstructured":"Tongzhou Wang , Jun-Yan Zhu , Antonio Torralba , and Alexei A Efros . Dataset distillation. arXiv preprint arXiv:1811.10959 , 2018 b. Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, and Alexei A Efros. Dataset distillation. arXiv preprint arXiv:1811.10959, 2018b."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1018"},{"key":"e_1_2_1_98_1","volume-title":"Chi, and Slav Petrov. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032","author":"Webster Kellie","year":"2019","unstructured":"Kellie Webster , Xuezhi Wang , Ian Tenney , Alex Beutel , Emily Pitler , Ellie Pavlick , Jilin Chen , Ed Chi, and Slav Petrov. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032 , 2019 . Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032, 2019."},{"key":"e_1_2_1_99_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations, ICLR '15","author":"Weston Jason","year":"2015","unstructured":"Jason Weston , Sumit Chopra , and Antoine Bordes . Memory networks . In Proceedings of the 3rd International Conference on Learning Representations, ICLR '15 , 2015 . Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. In Proceedings of the 3rd International Conference on Learning Representations, ICLR '15, 2015."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052591"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080809"},{"key":"e_1_2_1_102_1","volume-title":"Proceedings of the 9th International Conference on Learning Representations, ICLR '21","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong , Chenyan Xiong , Ye Li , Kwok-Fung Tang , Jialin Liu , Paul Bennett , Junaid Ahmed , and Arnold Overwijk . Approximate nearest neighbor negative contrastive learning for dense text retrieval . In Proceedings of the 9th International Conference on Learning Representations, ICLR '21 , 2021 . Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In Proceedings of the 9th International Conference on Learning Representations, ICLR '21, 2021."},{"key":"e_1_2_1_103_1","volume-title":"mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934","author":"Xue Linting","year":"2020","unstructured":"Linting Xue , Noah Constant , Adam Roberts , Mihir Kale , Rami Al-Rfou , Aditya Siddhant , Aditya Barua , and Colin Raffel . mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934 , 2020 . Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. mt5: A massively multilingual pre-trained text-to-text transformer. arXiv preprint arXiv:2010.11934, 2020."},{"key":"e_1_2_1_104_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1237"},{"key":"e_1_2_1_105_1","volume-title":"Proceedings of the 6th International Conference on Learning Representations, ICLR '18","author":"Yu Adams Wei","year":"2018","unstructured":"Adams Wei Yu , David Dohan , Minh-Thang Luong , Rui Zhao , Kai Chen , Mohammad Norouzi , and Quoc V Le. Qanet : Combining local convolution with global self-attention for reading comprehension . In Proceedings of the 6th International Conference on Learning Representations, ICLR '18 , 2018 . Adams Wei Yu, David Dohan, Minh-Thang Luong, Rui Zhao, Kai Chen, Mohammad Norouzi, and Quoc V Le. Qanet: Combining local convolution with global self-attention for reading comprehension. In Proceedings of the 6th International Conference on Learning Representations, ICLR '18, 2018."},{"key":"e_1_2_1_106_1","volume-title":"Proceedings of the 9th International Conference on Learning Representations, ICLR '21","author":"Zhao Bo","year":"2021","unstructured":"Bo Zhao , Konda Reddy Mopuri , and Hakan Bilen . Dataset condensation with gradient matching . In Proceedings of the 9th International Conference on Learning Representations, ICLR '21 , 2021 . Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. Dataset condensation with gradient matching. In Proceedings of the 9th International Conference on Learning Representations, ICLR '21, 2021."},{"key":"e_1_2_1_107_1","volume-title":"Proceedings of the 8th International Conference on Learning Representations, ICLR '20","author":"Zhao Chen","year":"2020","unstructured":"Chen Zhao , Chenyan Xiong , Corby Rosset , Xia Song , Paul Bennett , and Saurabh Tiwary . Transformer-xh : Multi-evidence reasoning with extra hop attention . In Proceedings of the 8th International Conference on Learning Representations, ICLR '20 , 2020 . Chen Zhao, Chenyan Xiong, Corby Rosset, Xia Song, Paul Bennett, and Saurabh Tiwary. Transformer-xh: Multi-evidence reasoning with extra hop attention. In Proceedings of the 8th International Conference on Learning Representations, ICLR '20, 2020."}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476415.3476428","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3476415.3476428","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:45Z","timestamp":1750188645000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3476415.3476428"}},"subtitle":["making domain experts out of dilettantes"],"short-title":[],"issued":{"date-parts":[[2021,6]]},"references-count":107,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["10.1145\/3476415.3476428"],"URL":"https:\/\/doi.org\/10.1145\/3476415.3476428","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2021,6]]},"assertion":[{"value":"2021-07-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}