{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:34:09Z","timestamp":1750221249942,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T00:00:00Z","timestamp":1530057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,6,27]]},"DOI":"10.1145\/3209978.3210050","type":"proceedings-article","created":{"date-parts":[[2018,7,2]],"date-time":"2018-07-02T12:12:40Z","timestamp":1530533560000},"page":"705-714","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Offline Comparative Evaluation with Incremental, Minimally-Invasive Online Feedback"],"prefix":"10.1145","author":[{"given":"Ben","family":"Carterette","sequence":"first","affiliation":[{"name":"University of Delaware, Newark, DE, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Praveen","family":"Chandar","sequence":"additional","affiliation":[{"name":"Spotify, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,6,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498824"},{"key":"e_1_3_2_1_2_1","volume-title":"Counterfactual reasoning and learning systems - the example of computational advertising. Journal of Machine Learning Research ()","author":"Bottou L.","year":"2013","unstructured":"L. Bottou , J. Peters , J. Q. Candela , D. X. Charles , M. Chickering , E. Portugaly , D. Ray , P. Y. Simard , and E. Snelson . Counterfactual reasoning and learning systems - the example of computational advertising. Journal of Machine Learning Research () , 2013 . L. Bottou, J. Peters, J. Q. Candela, D. X. Charles, M. Chickering, E. Portugaly, D. Ray, P. Y. Simard, and E. Snelson. Counterfactual reasoning and learning systems - the example of computational advertising. Journal of Machine Learning Research (), 2013."},{"key":"e_1_3_2_1_3_1","first-page":"217","volume-title":"NIPS","author":"Carterette B.","year":"2008","unstructured":"B. Carterette and R. Jones . Evaluating search engines by modeling the relationship between relevance and clicks. In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors , NIPS , pages 217 -- 224 . 2008 . B. Carterette and R. Jones. Evaluating search engines by modeling the relationship between relevance and clicks. In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, NIPS, pages 217--224. 2008."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2094072.2094078"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526711"},{"key":"e_1_3_2_1_6_1","volume-title":"Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services, 7(3):1--115","author":"Chuklin A.","year":"2015","unstructured":"A. Chuklin , I. Markov , and M. de Rijke . Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services, 7(3):1--115 , 2015 . A. Chuklin, I. Markov, and M. de Rijke. Click Models for Web Search. Synthesis Lectures on Information Concepts, Retrieval, and Services, 7(3):1--115, 2015."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291009"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341545"},{"issue":"4","key":"e_1_3_2_1_9_1","first-page":"485","article-title":"Doubly Robust Policy Evaluation and Optimization","volume":"29","author":"Dud'\u0131k M.","year":"2014","unstructured":"M. Dud'\u0131k , D. Erhan , J. Langford , and L. Li . Doubly Robust Policy Evaluation and Optimization . Statistical Science , 29 ( 4 ): 485 -- 511 , 2014 . M. Dud'\u0131k, D. Erhan, J. Langford, and L. Li. Doubly Robust Policy Evaluation and Optimization. Statistical Science, 29(4):485--511, 2014.","journal-title":"Statistical Science"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390392"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000051"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2063576.2063618"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1229179.1229181"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018699"},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society","volume":"28","author":"Keane M. T.","year":"2006","unstructured":"M. T. Keane and M. O'Brien . Click Models for Web Search . In Proceedings of the Annual Meeting of the Cognitive Science Society , volume 28 , 2006 . M. T. Keane and M. O'Brien. Click Models for Web Search. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 28, 2006."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-008-0114-1"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3133015"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2742562"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-56608-5_28"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2851613.2851692"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963463"},{"key":"e_1_3_2_1_24_1","volume-title":"A practical sampling strategy for efficient retrieval evaluation","author":"Pavlu V.","year":"2007","unstructured":"V. Pavlu and J. Aslam . A practical sampling strategy for efficient retrieval evaluation . 2007 . V. Pavlu and J. Aslam. A practical sampling strategy for efficient retrieval evaluation. 2007."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148263"},{"key":"e_1_3_2_1_26_1","volume-title":"Proceedings of the 21st National Conference on Artificial Intelligence -","volume":"2","author":"Radlinski F.","year":"2006","unstructured":"F. Radlinski and T. Joachims . Minimally invasive randomization for collecting unbiased preferences from clickthrough logs . In Proceedings of the 21st National Conference on Artificial Intelligence - Volume 2 , AAAI'06, 2006 . F. Radlinski and T. Joachims. Minimally invasive randomization for collecting unbiased preferences from clickthrough logs. In Proceedings of the 21st National Conference on Artificial Intelligence - Volume 2, AAAI'06, 2006."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458092"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1242572.1242643"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/70.1.41"},{"key":"e_1_3_2_1_30_1","volume-title":"TREC : Experiment and Evaluation in Information Retrieval","author":"Voorhees E. M.","year":"2005","unstructured":"E. M. Voorhees and D. K. Harman . TREC : Experiment and Evaluation in Information Retrieval . MIT Press , 2005 . E. M. Voorhees and D. K. Harman. TREC : Experiment and Evaluation in Information Retrieval. MIT Press, 2005."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911537"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390437"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772793"}],"event":{"name":"SIGIR '18: The 41st International ACM SIGIR conference on research and development in Information Retrieval","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Ann Arbor MI USA","acronym":"SIGIR '18"},"container-title":["The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209978.3210050","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3209978.3210050","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:07:49Z","timestamp":1750212469000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209978.3210050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,27]]},"references-count":33,"alternative-id":["10.1145\/3209978.3210050","10.1145\/3209978"],"URL":"https:\/\/doi.org\/10.1145\/3209978.3210050","relation":{},"subject":[],"published":{"date-parts":[[2018,6,27]]},"assertion":[{"value":"2018-06-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}