{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T05:10:12Z","timestamp":1755839412606,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":54,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,17]]},"DOI":"10.1145\/3511808.3557145","type":"proceedings-article","created":{"date-parts":[[2022,10,16]],"date-time":"2022-10-16T01:22:22Z","timestamp":1665883342000},"page":"3756-3765","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Approximated Doubly Robust Search Relevance Estimation"],"prefix":"10.1145","author":[{"given":"Lixin","family":"Zou","sequence":"first","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Changying","family":"Hao","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hengyi","family":"Cai","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuaiqiang","family":"Wang","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijng, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Suqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhicong","family":"Cheng","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenwen","family":"Ye","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simiu","family":"Gu","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dawei","family":"Yin","sequence":"additional","affiliation":[{"name":"Baidu Inc., Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Addressing Trust Bias for Unbiased Learning-to-Rank. In WWW'19","author":"Agarwal Aman","year":"2019","unstructured":"Aman Agarwal , Xuanhui Wang , Cheng Li , Michael Bendersky , and Marc Najork . 2019 . Addressing Trust Bias for Unbiased Learning-to-Rank. In WWW'19 . Aman Agarwal, Xuanhui Wang, Cheng Li, Michael Bendersky, and Marc Najork. 2019. Addressing Trust Bias for Unbiased Learning-to-Rank. In WWW'19."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_2_1","DOI":"10.1145\/3209978.3209986"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_3_1","DOI":"10.1145\/1645953.1646033"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_4_1","DOI":"10.1145\/1526709.1526711"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_5_1","DOI":"10.1145\/3477495.3531986"},{"unstructured":"Aleksandr Chuklin Ilya Markov and Maarten de Rijke. [n. d.]. Click models for web search. Synthesis lectures on information concepts retrieval and services (n. .  Aleksandr Chuklin Ilya Markov and Maarten de Rijke. [n. d.]. Click models for web search. Synthesis lectures on information concepts retrieval and services (n. .","key":"e_1_3_2_1_6_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_7_1","DOI":"10.1145\/1341531.1341545"},{"key":"e_1_3_2_1_8_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL'19","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL'19 . Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL'19."},{"key":"e_1_3_2_1_9_1","volume-title":"ICML'00","author":"Domingos Pedro","year":"2000","unstructured":"Pedro Domingos . 2000 . A unified bias-variance decomposition . In ICML'00 . Morgan Kaufmann Stanford. Pedro Domingos. 2000. A unified bias-variance decomposition. In ICML'00. Morgan Kaufmann Stanford."},{"key":"e_1_3_2_1_10_1","volume-title":"Doubly Robust Policy Evaluation and Learning. In ICML'11","author":"Dud\u00edk Miroslav","year":"2011","unstructured":"Miroslav Dud\u00edk , John Langford , and Lihong Li . 2011 . Doubly Robust Policy Evaluation and Learning. In ICML'11 . Miroslav Dud\u00edk, John Langford, and Lihong Li. 2011. Doubly Robust Policy Evaluation and Learning. In ICML'11."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_11_1","DOI":"10.1145\/1390334.1390392"},{"key":"e_1_3_2_1_12_1","volume-title":"ERR. Rank: An algorithm based on learning to rank for direct optimization of Expected Reciprocal Rank. Applied Intelligence","author":"Ghanbari Elham","year":"2019","unstructured":"Elham Ghanbari and Azadeh Shakery . 2019. ERR. Rank: An algorithm based on learning to rank for direct optimization of Expected Reciprocal Rank. Applied Intelligence ( 2019 ). Elham Ghanbari and Azadeh Shakery. 2019. ERR. Rank: An algorithm based on learning to rank for direct optimization of Expected Reciprocal Rank. Applied Intelligence (2019)."},{"key":"e_1_3_2_1_13_1","volume-title":"A survey of quantization methods for efficient neural network inference. arXiv:2103.13630","author":"Gholami Amir","year":"2021","unstructured":"Amir Gholami , Sehoon Kim , Zhen Dong , Zhewei Yao , Michael W Mahoney , and Kurt Keutzer . 2021. A survey of quantization methods for efficient neural network inference. arXiv:2103.13630 ( 2021 ). Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2021. A survey of quantization methods for efficient neural network inference. arXiv:2103.13630 (2021)."},{"unstructured":"Huifeng Guo Jinkai Yu Qing Liu Ruiming Tang and Yuzhou Zhang. 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems. In RecSys'19.  Huifeng Guo Jinkai Yu Qing Liu Ruiming Tang and Yuzhou Zhang. 2019. PAL: a position-bias aware learning framework for CTR prediction in live recommender systems. In RecSys'19.","key":"e_1_3_2_1_14_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_15_1","DOI":"10.1145\/3404835.3462917"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_16_1","DOI":"10.1609\/aaai.v35i14.17532"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_17_1","DOI":"10.1145\/3308558.3313447"},{"key":"e_1_3_2_1_18_1","volume-title":"TKDE'05","author":"Huang Jin","year":"2005","unstructured":"Jin Huang and Charles X Ling . 2005 . Using AUC and accuracy in evaluating learning algorithms . TKDE'05 (2005). Jin Huang and Charles X Ling. 2005. Using AUC and accuracy in evaluating learning algorithms. TKDE'05 (2005)."},{"key":"e_1_3_2_1_19_1","volume-title":"Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR'20","author":"Humeau Samuel","year":"2020","unstructured":"Samuel Humeau , Kurt Shuster , Marie-Anne Lachaux , and Jason Weston . 2020 . Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR'20 . Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring. In ICLR'20."},{"key":"e_1_3_2_1_20_1","volume-title":"SIGIR'17","author":"J\u00e4rvelin K.","year":"2017","unstructured":"K. J\u00e4rvelin and Jaana Kek\u00e4l\u00e4inen . 2017 . IR evaluation methods for retrieving highly relevant documents . In SIGIR'17 . K. J\u00e4rvelin and Jaana Kek\u00e4l\u00e4inen. 2017. IR evaluation methods for retrieving highly relevant documents. In SIGIR'17."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_21_1","DOI":"10.1145\/775047.775067"},{"doi-asserted-by":"crossref","unstructured":"Thorsten Joachims Laura Granka Bing Pan Helene Hembrooke and Geri Gay. 2017. Accurately interpreting clickthrough data as implicit feedback. In Acm Sigir Forum.  Thorsten Joachims Laura Granka Bing Pan Helene Hembrooke and Geri Gay. 2017. Accurately interpreting clickthrough data as implicit feedback. In Acm Sigir Forum.","key":"e_1_3_2_1_22_1","DOI":"10.1145\/3130332.3130334"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_23_1","DOI":"10.1145\/3018661.3018699"},{"doi-asserted-by":"crossref","unstructured":"J.D.Y. Kang and J.L. Schafer. 2007. Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. STATISTICAL SCIENCE (2007) 523.  J.D.Y. Kang and J.L. Schafer. 2007. Demystifying Double Robustness: A Comparison of Alternative Strategies for Estimating a Population Mean from Incomplete Data. STATISTICAL SCIENCE (2007) 523.","key":"e_1_3_2_1_24_1","DOI":"10.1214\/07-STS227"},{"key":"e_1_3_2_1_25_1","volume-title":"NIPS'17","author":"Ke Guolin","year":"2017","unstructured":"Guolin Ke , Qi Meng , Thomas Finley , Taifeng Wang , Wei Chen , Weidong Ma , Qiwei Ye , and Tie-Yan Liu . 2017 . Lightgbm: A highly efficient gradient boosting decision tree . NIPS'17 (2017). Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. NIPS'17 (2017)."},{"key":"e_1_3_2_1_26_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba . 2015 . Adam : A Method for Stochastic Optimization. In ICLR '15. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR'15."},{"key":"e_1_3_2_1_27_1","volume-title":"Disentangled Deep Multivariate Hawkes Process for Learning Event Sequences. In ICDM'21","author":"Lin Xixun","year":"2021","unstructured":"Xixun Lin , Jiangxia Cao , Peng Zhang , Chuan Zhou , Zhao Li , Jia Wu , and Bin Wang . 2021 . Disentangled Deep Multivariate Hawkes Process for Learning Event Sequences. In ICDM'21 . IEEE. Xixun Lin, Jiangxia Cao, Peng Zhang, Chuan Zhou, Zhao Li, Jia Wu, and Bin Wang. 2021. Disentangled Deep Multivariate Hawkes Process for Learning Event Sequences. In ICDM'21. IEEE."},{"key":"e_1_3_2_1_28_1","volume-title":"Model-based Unbiased Learning to Rank. arXiv:2207.11785","author":"Luo Dan","year":"2022","unstructured":"Dan Luo , Lixin Zou , Qingyao Ai , Zhiyu Chen , Dawei Yin , and Brian D Davison . 2022. Model-based Unbiased Learning to Rank. arXiv:2207.11785 ( 2022 ). Dan Luo, Lixin Zou, Qingyao Ai, Zhiyu Chen, Dawei Yin, and Brian D Davison. 2022. Model-based Unbiased Learning to Rank. arXiv:2207.11785 (2022)."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_29_1","DOI":"10.1145\/3437963.3441777"},{"volume-title":"Introduction to Information Retrieval","author":"Manning Christopher D.","unstructured":"Christopher D. Manning , Prabhakar Raghavan , and Hinrich Sch\u00fctze . 2008. Introduction to Information Retrieval . Cambridge University Press . Christopher D. Manning, Prabhakar Raghavan, and Hinrich Sch\u00fctze. 2008. Introduction to Information Retrieval. Cambridge University Press.","key":"e_1_3_2_1_30_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_31_1","DOI":"10.1145\/3341981.3344242"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_32_1","DOI":"10.1145\/3209978.3210060"},{"key":"e_1_3_2_1_33_1","volume-title":"WWW'99","author":"Page Lawrence","year":"1999","unstructured":"Lawrence Page , Sergey Brin , Rajeev Motwani , and Terry Winograd . 1999 . The PageRank Citation Ranking : Bringing Order to the Web . In WWW'99 . Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd. 1999. The PageRank Citation Ranking : Bringing Order to the Web. In WWW'99."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_34_1","DOI":"10.1145\/1242572.1242643"},{"doi-asserted-by":"crossref","unstructured":"Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr. (2009).  Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Found. Trends Inf. Retr. (2009).","key":"e_1_3_2_1_35_1","DOI":"10.1561\/1500000019"},{"volume-title":"The probabilistic relevance framework: BM25 and beyond","author":"Robertson Stephen","unstructured":"Stephen Robertson and Hugo Zaragoza . 2009. The probabilistic relevance framework: BM25 and beyond . Now Publishers Inc . Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Now Publishers Inc.","key":"e_1_3_2_1_36_1"},{"doi-asserted-by":"crossref","unstructured":"Yuta Saito. 2020. Doubly robust estimator for ranking metrics with post-click conversions. In RecSys'20.  Yuta Saito. 2020. Doubly robust estimator for ranking metrics with post-click conversions. In RecSys'20.","key":"e_1_3_2_1_37_1","DOI":"10.1145\/3383313.3412262"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_38_1","DOI":"10.1145\/2567948.2577348"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_39_1","DOI":"10.1609\/aaai.v34i05.6428"},{"key":"e_1_3_2_1_40_1","volume-title":"ACL'17","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N. Gomez , L. Kaiser , and Illia Polosukhin . 2017 . Attention is All you Need . In ACL'17 . Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, L. Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In ACL'17."},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_41_1","DOI":"10.1145\/2484028.2484036"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_42_1","DOI":"10.1145\/2911451.2911537"},{"key":"e_1_3_2_1_43_1","volume-title":"ICML'19","author":"Wang Xiaojie","year":"2019","unstructured":"Xiaojie Wang , Rui Zhang , Yu Sun , and Jianzhong Qi . 2019 . Doubly robust joint learning for recommendation on data missing not at random . In ICML'19 . PMLR. Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly robust joint learning for recommendation on data missing not at random. In ICML'19. PMLR."},{"unstructured":"Ronald J Williams. 199","key":"e_1_3_2_1_44_1"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_45_1","DOI":"10.1145\/3437963.3441751"},{"doi-asserted-by":"crossref","unstructured":"Tao Yang Chen Luo Hanqing Lu Parth Gupta Bing Yin and Qingyao Ai. 2022. Can clicks be both labels and features? Unbiased Behavior Feature Collection and Uncertainty-aware Learning to Rank. (2022).  Tao Yang Chen Luo Hanqing Lu Parth Gupta Bing Yin and Qingyao Ai. 2022. Can clicks be both labels and features? Unbiased Behavior Feature Collection and Uncertainty-aware Learning to Rank. (2022).","key":"e_1_3_2_1_46_1","DOI":"10.1145\/3477495.3531948"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_47_1","DOI":"10.1145\/2939672.2939677"},{"doi-asserted-by":"crossref","unstructured":"Bowen Yuan Yaxu Liu Jui-Yang Hsia Zhenhua Dong and Chih-Jen Lin. 2020. Unbiased Ad click prediction for position-aware advertising systems. In RecSys'20.  Bowen Yuan Yaxu Liu Jui-Yang Hsia Zhenhua Dong and Chih-Jen Lin. 2020. Unbiased Ad click prediction for position-aware advertising systems. In RecSys'20.","key":"e_1_3_2_1_48_1","DOI":"10.1145\/3383313.3412241"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_49_1","DOI":"10.1145\/3366423.3380037"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_50_1","DOI":"10.1145\/3340531.3412044"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_51_1","DOI":"10.1145\/3292500.3330668"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_52_1","DOI":"10.1145\/3336191.3371801"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_53_1","DOI":"10.1145\/3397271.3401181"},{"doi-asserted-by":"publisher","key":"e_1_3_2_1_54_1","DOI":"10.1145\/3447548.3467147"}],"event":{"sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"acronym":"CIKM '22","name":"CIKM '22: The 31st ACM International Conference on Information and Knowledge Management","location":"Atlanta GA USA"},"container-title":["Proceedings of the 31st ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557145","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511808.3557145","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:57Z","timestamp":1750188657000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557145"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":54,"alternative-id":["10.1145\/3511808.3557145","10.1145\/3511808"],"URL":"https:\/\/doi.org\/10.1145\/3511808.3557145","relation":{},"subject":[],"published":{"date-parts":[[2022,10,17]]},"assertion":[{"value":"2022-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}