{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T00:40:42Z","timestamp":1760488842169,"version":"build-2065373602"},"reference-count":67,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"Strategic Priority Research Program of the CAS","award":["XDB0680102"],"award-info":[{"award-number":["XDB0680102"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62472408, 62372431 and 62441229"],"award-info":[{"award-number":["62472408, 62372431 and 62441229"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program of China","award":["2023YFA1011602"],"award-info":[{"award-number":["2023YFA1011602"]}]},{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association CAS","doi-asserted-by":"crossref","award":["2021100"],"award-info":[{"award-number":["2021100"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Lenovo-CAS Joint Lab Youth Scientist Project, and the Strategic Priority Research Program of the CAS","award":["XDB0680301"],"award-info":[{"award-number":["XDB0680301"]}]},{"DOI":"10.13039\/501100003246","name":"Dutch Research Council","doi-asserted-by":"crossref","award":["024.004.022, NWA.1389.20.183, and KICH3.LTP.20.006"],"award-info":[{"award-number":["024.004.022, NWA.1389.20.183, and KICH3.LTP.20.006"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]},{"name":"European Union\u2019s Horizon Europe program","award":["101070212"],"award-info":[{"award-number":["101070212"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2026,1,31]]},"abstract":"<jats:p>Knowledge-intensive language tasks (KILTs) typically require retrieving relevant documents from trustworthy corpora, e.g., Wikipedia, to produce specific answers. Very recently, a pre-trained generative retrieval model for KILTs, named CorpusBrain, was proposed and reached new state-of-the-art retrieval performance. However, most research on KILTs, including CorpusBrain, has predominantly focused on a static document collection, overlooking the dynamic nature of real-world scenarios, where new documents are continuously being incorporated into the source corpus. To address this gap, it is crucial to explore the capability of retrieval models to effectively handle the dynamic retrieval scenario inherent in KILTs.<\/jats:p>\n          <jats:p>In this work, we first introduce the continual document learning (CDL) task for KILTs and build a novel benchmark dataset named KILT++ based on the original KILT dataset for evaluation. Then, we conduct a comprehensive study of the use of pre-trained CorpusBrain on KILT++. Unlike the promising results in the stationary scenario, CorpusBrain is prone to catastrophic forgetting in the dynamic scenario, hence hampering retrieval performance. To alleviate this issue, we propose CorpusBrain++, a continual generative pre-training framework that enhances the original model along two key dimensions: (i) We employ a backbone-adapter architecture: the dynamic adapter is learned for each downstream KILT task via task-specific pre-training objectives; the backbone parameters that are task-shared are kept unchanged to offer foundational retrieval capacity. (ii) We use an experience replay strategy based on exemplar documents that are similar to new documents, to prevent catastrophic forgetting of old documents. Empirical results demonstrate the effectiveness and efficiency of CorpusBrain++ in comparison to both traditional and generative information retrieval methods.<\/jats:p>","DOI":"10.1145\/3763233","type":"journal-article","created":{"date-parts":[[2025,8,25]],"date-time":"2025-08-25T13:51:16Z","timestamp":1756129876000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["CorpusBrain++: A Continual Generative Pre-Training Framework for Knowledge-Intensive Language Tasks"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9509-8674","authenticated-orcid":false,"given":"Jiafeng","family":"Guo","sequence":"first","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0005-9465","authenticated-orcid":false,"given":"Changjiang","family":"Zhou","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4294-2541","authenticated-orcid":false,"given":"Ruqing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6235-6526","authenticated-orcid":false,"given":"Jiangui","family":"Chen","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1086-0202","authenticated-orcid":false,"given":"Maarten","family":"de Rijke","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4317-2702","authenticated-orcid":false,"given":"Yixing","family":"Fan","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5201-8195","authenticated-orcid":false,"given":"Xueqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China and University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,10,14]]},"reference":[{"key":"e_1_3_3_2_2","article-title":"On the evolution of Wikipedia","author":"Almeida Rodrigo B.","year":"2007","unstructured":"Rodrigo B. Almeida, Barzan Mozafari, and Junghoo Cho. 2007. On the evolution of Wikipedia. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM).","journal-title":"Proceedings of the International AAAI Conference on Web and Social Media (ICWSM)"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1098"},{"key":"e_1_3_3_4_2","unstructured":"Ali Ayub and Alan R. Wagner. 2021. EEC: Learning to encode and regenerate images for continual learning. arXiv:2101.04904. Retrieved from https:\/\/arxiv.org\/abs\/2101.04904"},{"key":"e_1_3_3_5_2","first-page":"31668","article-title":"Autoregressive search engines: Generating substrings as document identifiers","volume":"35","author":"Bevilacqua Michele","year":"2022","unstructured":"Michele Bevilacqua, Giuseppe Ottaviano, Patrick Lewis, Scott Yih, Sebastian Riedel, and Fabio Petroni. 2022. Autoregressive search engines: Generating substrings as document identifiers. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 35, 31668\u201331683.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1171"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614821"},{"key":"e_1_3_3_8_2","first-page":"181","volume-title":"Proceedings of the 30th ACM International Conference on Information and Knowledge Management","author":"Chen Jiangui","year":"2021","unstructured":"Jiangui Chen, Ruqing Zhang, Jiafeng Guo, Yixing Fan, and Xueqi Cheng. 2021. FedMatch: Federated learning over heterogeneous question answering data. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, 181\u2013190."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531827"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557271"},{"key":"e_1_3_3_11_2","volume-title":"Discourse Processes: Advances in Research and Theory","author":"Clark Herbert H.","year":"1977","unstructured":"Herbert H. Clark, S. Haviland, and Roy O. Freedle. 1977. Discourse production and comprehension. In Discourse Processes: Advances in Research and Theory. Roy O. Freedle (Ed.), Ablex Publishing Corporation."},{"key":"e_1_3_3_12_2","first-page":"91","article-title":"Psychological processes as linguistic explanation","author":"Clark Herbert H.","year":"1974","unstructured":"Herbert H. Clark and Susan E. Haviland. 1974. Psychological processes as linguistic explanation. In Explaining Linguistic Phenomena, 91\u2013124.","journal-title":"Explaining Linguistic Phenomena"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401204"},{"key":"e_1_3_3_14_2","unstructured":"Nicola De Cao Gautier Izacard Sebastian Riedel and Fabio Petroni. 2020. Autoregressive entity retrieval. arXiv:2010.00904. Retrieved from https:\/\/arxiv.org\/abs\/2010.00904"},{"issue":"7","key":"e_1_3_3_15_2","first-page":"3366","article-title":"A continual learning survey: Defying forgetting in classification tasks","volume":"44","author":"De Lange Matthias","year":"2021","unstructured":"Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ale\u0161 Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. 2021. A continual learning survey: Defying forgetting in classification tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 7 (2021), 3366\u20133385.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_3_16_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171\u20134186."},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1346"},{"key":"e_1_3_3_18_2","first-page":"1993 1996","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Frej Jibril","year":"2020","unstructured":"Jibril Frej, Philippe Mulhem, Didier Schwab, and Jean-Pierre Chevallet. 2020. Learning term discrimination. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, 1993\u20131996."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.247"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24004"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.102067"},{"key":"e_1_3_3_22_2","article-title":"Learning the  \\( k \\)  in  \\( k \\) -means","volume":"16","author":"Hamerly Greg","year":"2003","unstructured":"Greg Hamerly and Charles Elkan. 2003. Learning the \\( k \\) in \\( k \\) -means. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 16.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"issue":"5","key":"e_1_3_3_23_2","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1016\/S0022-5371(74)80003-4","article-title":"What\u2019s new? Acquiring new information as a process in comprehension","volume":"13","author":"Haviland Susan E.","year":"1974","unstructured":"Susan E. Haviland and Herbert H. Clark. 1974. What\u2019s new? Acquiring new information as a process in comprehension. Journal of Verbal Learning and Verbal Behavior 13, 5 (1974), 512\u2013521.","journal-title":"Journal of Verbal Learning and Verbal Behavior"},{"key":"e_1_3_3_24_2","first-page":"2790","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of the International Conference on Machine Learning. PMLR, 2790\u20132799."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1147"},{"key":"e_1_3_3_26_2","doi-asserted-by":"crossref","unstructured":"Vladimir Karpukhin Barlas O\u011fuz Sewon Min Patrick Lewis Ledell Wu Sergey Edunov Danqi Chen and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. arXiv:2004.04906. Retrieved from https:\/\/arxiv.org\/abs\/2004.04906","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_3_3_27_2","first-page":"6975","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Ke Pei","year":"2020","unstructured":"Pei Ke, Haozhe Ji, Siyang Liu, Xiaoyan Zhu, and Minlie Huang. 2020. SentiLARE: Sentiment-aware language representation learning with linguistic knowledge. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 6975\u20136988."},{"key":"e_1_3_3_28_2","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00276"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_3_32_2","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"33","author":"Lewis Patrick","year":"2020","unstructured":"Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\u00fcttler, Mike Lewis, Wen-tau Yih, Tim Rockt\u00e4schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 33, 9459\u20139474.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2773081"},{"key":"e_1_3_3_34_2","article-title":"Gradient episodic memory for continual learning","volume":"30","author":"Lopez-Paz David","year":"2017","unstructured":"David Lopez-Paz and Marc\u2019Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 30.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00810"},{"key":"e_1_3_3_36_2","unstructured":"Sanket Vaibhav Mehta Jai Gupta Yi Tay Mostafa Dehghani Vinh Q. Tran Jinfeng Rao Marc Najork Emma Strubell and Donald Metzler. 2022. DSI++: Updating transformer memory with new documents. arXiv:2212.09744. Retrieved from https:\/\/arxiv.org\/abs\/2212.09744"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3476415.3476428"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0022-2496(02)00028-7"},{"key":"e_1_3_3_39_2","article-title":"From doc2query to docTTTTTquery","author":"Nogueira Rodrigo","year":"2019","unstructured":"Rodrigo Nogueira and Jimmy Lin. 2019. From doc2query to docTTTTTquery. In An MS MARCO Passage Retrieval Task Publication. University of Waterloo.","journal-title":"An MS MARCO Passage Retrieval Task Publication"},{"key":"e_1_3_3_40_2","unstructured":"Felipe Ortega. 2009. Wikipedia: A Quantitative Analysis. Ph.D. Dissertation. Universidad Rey Juan Carlos Madrid Spain."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01158"},{"key":"e_1_3_3_42_2","doi-asserted-by":"crossref","unstructured":"Fabio Petroni Aleksandra Piktus Angela Fan Patrick Lewis Majid Yazdani Nicola De Cao James Thorne Yacine Jernite Vladimir Karpukhin Jean Maillard et al. 2020. KILT: A benchmark for knowledge intensive language tasks. arXiv:2009.02252. Retrieved from https:\/\/arxiv.org\/abs\/2009.02252","DOI":"10.18653\/v1\/2021.naacl-main.200"},{"key":"e_1_3_3_43_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_3_44_2","first-page":"29","volume-title":"Proceedings of the 1st Instructional Conference on Machine Learning","volume":"242","author":"Ramos Juan","year":"2003","unstructured":"Juan Ramos. 2003. Using TF-IDF to determine word relevance in document queries. In Proceedings of the 1st Instructional Conference on Machine Learning, Vol. 242, 29\u201348."},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.587"},{"key":"e_1_3_3_46_2","article-title":"Online structured Laplace approximations for overcoming catastrophic forgetting","volume":"31","author":"Ritter Hippolyt","year":"2018","unstructured":"Hippolyt Ritter, Aleksandar Botev, and David Barber. 2018. Online structured Laplace approximations for overcoming catastrophic forgetting. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 31.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000019"},{"key":"e_1_3_3_48_2","article-title":"Okapi at TREC-3","author":"Robertson Stephen E.","year":"1994","unstructured":"Stephen E. Robertson, Steve Walker, Susan Jones, Micheline M. Hancock-Beaulieu, and Mike Gatford. 1994. Okapi at TREC-3. In Text Retrieval Conference. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:3946054","journal-title":"Text Retrieval Conference"},{"key":"e_1_3_3_49_2","unstructured":"Andrei A. Rusu Neil C. Rabinowitz Guillaume Desjardins Hubert Soyer James Kirkpatrick Koray Kavukcuoglu Razvan Pascanu and Raia Hadsell. 2016. Progressive neural networks. arXiv:1606.04671. Retrieved from https:\/\/arxiv.org\/abs\/1606.04671"},{"key":"e_1_3_3_50_2","first-page":"4528","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Schwarz Jonathan","year":"2018","unstructured":"Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. 2018. Progress & compress: A scalable framework for continual learning. In Proceedings of the International Conference on Machine Learning. PMLR, 4528\u20134537."},{"key":"e_1_3_3_51_2","article-title":"Continual learning with deep generative replay","volume":"30","author":"Shin Hanul","year":"2017","unstructured":"Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. 2017. Continual learning with deep generative replay. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 30.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"issue":"4","key":"e_1_3_3_52_2","first-page":"35","article-title":"Modern information retrieval: A brief overview","volume":"24","author":"Singhal Amit","year":"2001","unstructured":"Amit Singhal. 2001. Modern information retrieval: A brief overview. IEEE Data Engineering Bulletin 24, 4 (2001), 35\u201343.","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_3_3_53_2","first-page":"5986","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Stickland Asa Cooper","year":"2019","unstructured":"Asa Cooper Stickland and Iain Murray. 2019. BERT and Pals: Projected attention layers for efficient adaptation in multi-task learning. In Proceedings of the International Conference on Machine Learning. PMLR, 5986\u20135995."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412047"},{"key":"e_1_3_3_55_2","unstructured":"Weiwei Sun Lingyong Yan Zheng Chen Shuaiqiang Wang Haichao Zhu Pengjie Ren Zhumin Chen Dawei Yin Maarten de Rijke and Zhaochun Ren. 2023. Learning to tokenize for generative retrieval. arXiv:2304.04171. Retrieved from https:\/\/arxiv.org\/abs\/2304.04171"},{"key":"e_1_3_3_56_2","first-page":"21831","article-title":"Transformer memory as a differentiable search index","volume":"35","author":"Tay Yi","year":"2022","unstructured":"Yi Tay, Vinh Tran, Mostafa Dehghani, Jianmo Ni, Dara Bahri, Harsh Mehta, Zhen Qin, Kai Hui, Zhe Zhao, Jai Gupta, et al. 2022. Transformer memory as a differentiable search index. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 35, 21831\u201321843.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1074"},{"key":"e_1_3_3_58_2","unstructured":"Gido M. Van de Ven and Andreas S. Tolias. 2019. Three scenarios for continual learning. arXiv:1904.07734. Retrieved from https:\/\/arxiv.org\/abs\/1904.07734"},{"key":"e_1_3_3_59_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 30 (2017).","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614993"},{"key":"e_1_3_3_61_2","first-page":"6397","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Wu Ledell","year":"2020","unstructured":"Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero-shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 6397\u20136407."},{"key":"e_1_3_3_62_2","unstructured":"Lee Xiong Chenyan Xiong Ye Li Kwok-Fung Tang Jialin Liu Paul Bennett Junaid Ahmed and Arnold Overwijk. 2020. Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv:2007.00808. Retrieved from https:\/\/arxiv.org\/abs\/2007.00808"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1259"},{"key":"e_1_3_3_64_2","first-page":"23675","article-title":"Task-free continual learning via online discrepancy distance learning","volume":"35","author":"Ye Fei","year":"2022","unstructured":"Fei Ye and Adrian G. Bors. 2022. Task-free continual learning via online discrepancy distance learning. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 35 (2022), 23675\u201323688.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_65_2","unstructured":"Soyoung Yoon Chaeeun Kim Hyunji Lee Joel Jang and Minjoon Seo. 2023. Continually updating generative retrieval on dynamic corpora. arXiv:2305.18952. Retrieved from https:\/\/arxiv.org\/abs\/2305.18952"},{"key":"e_1_3_3_66_2","first-page":"11328","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhang Jingqing","year":"2020","unstructured":"Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter Liu. 2020. Pegasus: Pre-training with extracted gap-sentences for abstractive summarization. In Proceedings of the International Conference on Machine Learning. PMLR, 11328\u201311339."},{"key":"e_1_3_3_67_2","unstructured":"Wayne Xin Zhao Jing Liu Ruiyang Ren and Ji-Rong Wen. 2022. Dense text retrieval based on pretrained language models: A survey. arXiv:2211.14876. Retrieved from https:\/\/arxiv.org\/abs\/2211.14876"},{"key":"e_1_3_3_68_2","volume-title":"2023 Conference on Empirical Methods in Natural Language Processing","author":"Zhou Yujia","year":"2023","unstructured":"Yujia Zhou, Zhicheng Dou, and Ji-Rong Wen. 2023. Enhancing generative retrieval with reinforcement learning from relevance feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763233","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,14]],"date-time":"2025-10-14T18:31:57Z","timestamp":1760466717000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763233"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,14]]},"references-count":67,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1,31]]}},"alternative-id":["10.1145\/3763233"],"URL":"https:\/\/doi.org\/10.1145\/3763233","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"type":"print","value":"1046-8188"},{"type":"electronic","value":"1558-2868"}],"subject":[],"published":{"date-parts":[[2025,10,14]]},"assertion":[{"value":"2024-03-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-29","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}