{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:13:28Z","timestamp":1750220008913,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the Youth Innovation Promotion Association CAS","award":["No. 20144310, and 2021100"],"award-info":[{"award-number":["No. 20144310, and 2021100"]}]},{"name":"the Lenovo-CAS Joint Lab Youth Scientist Project"},{"name":"the National Natural Science Foundation of China (NSFC)","award":["No. 62006218 and 61902381"],"award-info":[{"award-number":["No. 62006218 and 61902381"]}]},{"name":"the Young Elite Scientist Sponsorship Program by CAST","award":["No. YESS20200121"],"award-info":[{"award-number":["No. YESS20200121"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,17]]},"DOI":"10.1145\/3511808.3557582","type":"proceedings-article","created":{"date-parts":[[2022,10,16]],"date-time":"2022-10-16T01:22:22Z","timestamp":1665883342000},"page":"3858-3862","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Discriminative Language Model via Self-Teaching for Dense Retrieval"],"prefix":"10.1145","author":[{"given":"Lu","family":"Chen","sequence":"first","affiliation":[{"name":"CAS Key Lab of Network Data Science and Technology, ICT, CAS; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ruqing","family":"Zhang","sequence":"additional","affiliation":[{"name":"CAS Key Lab of Network Data Science and Technology, ICT, CAS; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiafeng","family":"Guo","sequence":"additional","affiliation":[{"name":"CAS Key Lab of Network Data Science and Technology, ICT, CAS; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yixing","family":"Fan","sequence":"additional","affiliation":[{"name":"CAS Key Lab of Network Data Science and Technology, ICT, CAS; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xueqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"CAS Key Lab of Network Data Science and Technology, ICT, CAS; University of Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval. arXiv preprint arXiv:2010.00768","author":"Bai Yang","year":"2020","unstructured":"Yang Bai , Xiaoguang Li , Gang Wang , Chaoliang Zhang , Lifeng Shang , Jun Xu , Zhaowei Wang , Fangshan Wang , and Qun Liu . 2020. SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval. arXiv preprint arXiv:2010.00768 ( 2020 ). Yang Bai, Xiaoguang Li, Gang Wang, Chaoliang Zhang, Lifeng Shang, Jun Xu, Zhaowei Wang, Fangshan Wang, and Qun Liu. 2020. SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval. arXiv preprint arXiv:2010.00768 (2020)."},{"key":"e_1_3_2_1_2_1","unstructured":"Payal Bajaj Daniel Campos Nick Craswell Li Deng Jianfeng Gao Xiaodong Liu Rangan Majumder Andrew McNamara Bhaskar Mitra Tri Nguyen etal 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016).  Payal Bajaj Daniel Campos Nick Craswell Li Deng Jianfeng Gao Xiaodong Liu Rangan Majumder Andrew McNamara Bhaskar Mitra Tri Nguyen et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Zhuyun Dai and Jamie Callan. 2020a. Context-aware document term weighting for ad-hoc search. In WWW.  Zhuyun Dai and Jamie Callan. 2020a. Context-aware document term weighting for ad-hoc search. In WWW.","DOI":"10.1145\/3366423.3380258"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Zhuyun Dai and Jamie Callan. 2020b. Context-aware term weighting for first stage passage retrieval. In SIGIR.  Zhuyun Dai and Jamie Callan. 2020b. Context-aware term weighting for first stage passage retrieval. In SIGIR.","DOI":"10.1145\/3397271.3401204"},{"key":"e_1_3_2_1_5_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL.","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Takashi Fukuda Masayuki Suzuki Gakuto Kurata Samuel Thomas Jia Cui and Bhuvana Ramabhadran. 2017. Efficient Knowledge Distillation from an Ensemble of Teachers. In Interspeech.  Takashi Fukuda Masayuki Suzuki Gakuto Kurata Samuel Thomas Jia Cui and Bhuvana Ramabhadran. 2017. Efficient Knowledge Distillation from an Ensemble of Teachers. In Interspeech.","DOI":"10.21437\/Interspeech.2017-614"},{"key":"e_1_3_2_1_7_1","volume-title":"Benjamin Van Durme, and Jamie Callan","author":"Gao Luyu","year":"2021","unstructured":"Luyu Gao , Zhuyun Dai , Tongfei Chen , Zhen Fan , Benjamin Van Durme, and Jamie Callan . 2021 . Complement lexical retrieval model with semantic residual embeddings. In ECIR. Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. 2021. Complement lexical retrieval model with semantic residual embeddings. In ECIR."},{"key":"e_1_3_2_1_8_1","volume-title":"Self-Knowledge Distillation in Natural Language Processing. In RANLP","author":"Hahn Sangchul","year":"2019","unstructured":"Sangchul Hahn and Heeyoul Choi . 2019 . Self-Knowledge Distillation in Natural Language Processing. In RANLP 2019. Sangchul Hahn and Heeyoul Choi. 2019. Self-Knowledge Distillation in Natural Language Processing. In RANLP 2019."},{"key":"e_1_3_2_1_9_1","unstructured":"Geoffrey Hinton Oriol Vinyals Jeff Dean etal 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).  Geoffrey Hinton Oriol Vinyals Jeff Dean et al. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Ganesh Jawahar Beno^it Sagot and Djam\u00e9 Seddah. 2019. What does BERT learn about the structure of language?. In ACL.  Ganesh Jawahar Beno^it Sagot and Djam\u00e9 Seddah. 2019. What does BERT learn about the structure of language?. In ACL.","DOI":"10.18653\/v1\/P19-1356"},{"key":"e_1_3_2_1_11_1","volume-title":"TinyBERT: Distilling BERT for Natural Language Understanding. In Findings of the Association for Computational Linguistics: EMNLP","author":"Jiao Xiaoqi","year":"2020","unstructured":"Xiaoqi Jiao , Yichun Yin , Lifeng Shang , Xin Jiang , Xiao Chen , Linlin Li , Fang Wang , and Qun Liu . 2020 . TinyBERT: Distilling BERT for Natural Language Understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. TinyBERT: Distilling BERT for Natural Language Understanding. In Findings of the Association for Computational Linguistics: EMNLP 2020."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"crossref","unstructured":"Vladimir Karpukhin Barlas Oguz Sewon Min Patrick Lewis Ledell Wu Sergey Edunov Danqi Chen and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In EMNLP.  Vladimir Karpukhin Barlas Oguz Sewon Min Patrick Lewis Ledell Wu Sergey Edunov Danqi Chen and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering. In EMNLP.","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Victor Lavrenko and W Bruce Croft. 2017. Relevance-based language models. In SIGIR Forum.  Victor Lavrenko and W Bruce Croft. 2017. Relevance-based language models. In SIGIR Forum.","DOI":"10.1145\/3130348.3130376"},{"key":"e_1_3_2_1_14_1","unstructured":"Sheng-Chieh Lin Jheng-Hong Yang and Jimmy Lin. 2021. In-batch negatives for knowledge distillation with tightly-coupled teachers for dense retrieval. In RepL4NLP.  Sheng-Chieh Lin Jheng-Hong Yang and Jimmy Lin. 2021. In-batch negatives for knowledge distillation with tightly-coupled teachers for dense retrieval. In RepL4NLP."},{"key":"e_1_3_2_1_15_1","unstructured":"Shuqi Lu Di He Chenyan Xiong Guolin Ke Waleed Malik Zhicheng Dou Paul Bennett Tie-Yan Liu and Arnold Overwijk. 2021. Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder. In EMNLP.  Shuqi Lu Di He Chenyan Xiong Guolin Ke Waleed Malik Zhicheng Dou Paul Bennett Tie-Yan Liu and Arnold Overwijk. 2021. Less is More: Pretrain a Strong Siamese Encoder for Dense Text Retrieval Using a Weak Decoder. In EMNLP."},{"key":"e_1_3_2_1_16_1","volume-title":"dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181","author":"Luan Yi","year":"2020","unstructured":"Yi Luan , Jacob Eisenstein , Kristina Toutanova , and Michael Collins . 2020. Sparse , dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181 ( 2020 ). Yi Luan, Jacob Eisenstein, Kristina Toutanova, and Michael Collins. 2020. Sparse, dense, and attentional representations for text retrieval. arXiv preprint arXiv:2005.00181 (2020)."},{"key":"e_1_3_2_1_17_1","volume-title":"Overview of the TREC 2019 deep learning track.","author":"Nick Craswell","year":"2021","unstructured":"Craswell Nick , Mitra Bhaskar , Yilmaz Emine , Campos Daniel , and Ellen Voorhees M. 2021 . Overview of the TREC 2019 deep learning track. (2021). Craswell Nick, Mitra Bhaskar, Yilmaz Emine, Campos Daniel, and Ellen Voorhees M. 2021. Overview of the TREC 2019 deep learning track. (2021)."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR.  Stephen E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR.","DOI":"10.1007\/978-1-4471-2099-5_24"},{"key":"e_1_3_2_1_19_1","volume":"198","author":"Salton Gerard","unstructured":"Gerard Salton and Michael J McGill. 198 3. Introduction to modern information retrieval. mcgraw-hill. Gerard Salton and Michael J McGill. 1983. Introduction to modern information retrieval. mcgraw-hill.","journal-title":"Michael J McGill."},{"key":"e_1_3_2_1_20_1","volume-title":"a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108","author":"Sanh Victor","year":"2019","unstructured":"Victor Sanh , Lysandre Debut , Julien Chaumond , and Thomas Wolf . 2019. DistilBERT , a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 ( 2019 ). Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)."},{"key":"e_1_3_2_1_21_1","volume-title":"Visualizing data using t-SNE. JMLR","author":"der Maaten Laurens Van","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton . 2008. Visualizing data using t-SNE. JMLR ( 2008 ). Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. JMLR (2008)."},{"key":"e_1_3_2_1_22_1","volume-title":"Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957","author":"Wang Wenhui","year":"2020","unstructured":"Wenhui Wang , Furu Wei , Li Dong , Hangbo Bao , Nan Yang , and Ming Zhou . 2020 . Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957 (2020). Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957 (2020)."},{"key":"e_1_3_2_1_23_1","volume-title":"Improving bert fine-tuning via self-ensemble and self-distillation. arXiv preprint arXiv:2002.10345","author":"Xu Yige","year":"2020","unstructured":"Yige Xu , Xipeng Qiu , Ligao Zhou , and Xuanjing Huang . 2020. Improving bert fine-tuning via self-ensemble and self-distillation. arXiv preprint arXiv:2002.10345 ( 2020 ). Yige Xu, Xipeng Qiu, Ligao Zhou, and Xuanjing Huang. 2020. Improving bert fine-tuning via self-ensemble and self-distillation. arXiv preprint arXiv:2002.10345 (2020)."},{"key":"e_1_3_2_1_24_1","volume-title":"ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer. arXiv preprint arXiv:2105.11741","author":"Yan Yuanmeng","year":"2021","unstructured":"Yuanmeng Yan , Rumei Li , Sirui Wang , Fuzheng Zhang , Wei Wu , and Weiran Xu. 2021. ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer. arXiv preprint arXiv:2105.11741 ( 2021 ). Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021. ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer. arXiv preprint arXiv:2105.11741 (2021)."},{"key":"e_1_3_2_1_25_1","volume-title":"Optimizing Dense Retrieval Model Training with Hard Negatives. arXiv preprint arXiv:2104.08051","author":"Zhan Jingtao","year":"2021","unstructured":"Jingtao Zhan , Jiaxin Mao , Yiqun Liu , Jiafeng Guo , Min Zhang , and Shaoping Ma. 2021. Optimizing Dense Retrieval Model Training with Hard Negatives. arXiv preprint arXiv:2104.08051 ( 2021 ). Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021. Optimizing Dense Retrieval Model Training with Hard Negatives. arXiv preprint arXiv:2104.08051 (2021)."},{"key":"e_1_3_2_1_26_1","volume-title":"RepBERT: Contextualized text embeddings for first-stage retrieval. arXiv preprint arXiv:2006.15498","author":"Zhan Jingtao","year":"2020","unstructured":"Jingtao Zhan , Jiaxin Mao , Yiqun Liu , Min Zhang , and Shaoping Ma. 2020. RepBERT: Contextualized text embeddings for first-stage retrieval. arXiv preprint arXiv:2006.15498 ( 2020 ). Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Min Zhang, and Shaoping Ma. 2020. RepBERT: Contextualized text embeddings for first-stage retrieval. arXiv preprint arXiv:2006.15498 (2020)."},{"key":"e_1_3_2_1_27_1","volume-title":"Self-distillation: Towards efficient and compact neural networks.","author":"Zhang Linfeng","year":"2021","unstructured":"Linfeng Zhang , Chenglong Bao , and Kaisheng Ma . 2021 . Self-distillation: Towards efficient and compact neural networks. (2021). Linfeng Zhang, Chenglong Bao, and Kaisheng Ma. 2021. Self-distillation: Towards efficient and compact neural networks. (2021)."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00381"},{"key":"e_1_3_2_1_29_1","unstructured":"Wangchunshu Zhou Canwen Xu and Julian McAuley. 2022. BERT Learns to Teach: Knowledge Distillation with Meta Learning. In ACL.  Wangchunshu Zhou Canwen Xu and Julian McAuley. 2022. BERT Learns to Teach: Knowledge Distillation with Meta Learning. In ACL."}],"event":{"name":"CIKM '22: The 31st ACM International Conference on Information and Knowledge Management","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Atlanta GA USA","acronym":"CIKM '22"},"container-title":["Proceedings of the 31st ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557582","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511808.3557582","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:09Z","timestamp":1750182669000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":29,"alternative-id":["10.1145\/3511808.3557582","10.1145\/3511808"],"URL":"https:\/\/doi.org\/10.1145\/3511808.3557582","relation":{},"subject":[],"published":{"date-parts":[[2022,10,17]]},"assertion":[{"value":"2022-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}