{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:15:08Z","timestamp":1775283308896,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":43,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T00:00:00Z","timestamp":1691107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,6]]},"DOI":"10.1145\/3580305.3599447","type":"proceedings-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T18:13:58Z","timestamp":1691172838000},"page":"1154-1163","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Off-Policy Evaluation of Ranking Policies under Diverse User Behavior"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-6378-4365","authenticated-orcid":false,"given":"Haruka","family":"Kiyohara","sequence":"first","affiliation":[{"name":"Hanjuku-Kaso Co., Ltd., Tokyo, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9017-3105","authenticated-orcid":false,"given":"Masatoshi","family":"Uehara","sequence":"additional","affiliation":[{"name":"Cornell University, New York, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0314-3384","authenticated-orcid":false,"given":"Yusuke","family":"Narita","sequence":"additional","affiliation":[{"name":"Yale University, New Haven, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6767-3662","authenticated-orcid":false,"given":"Nobuyuki","family":"Shimizu","sequence":"additional","affiliation":[{"name":"Yahoo Japan Corporation, Tokyo, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9257-9063","authenticated-orcid":false,"given":"Yasuo","family":"Yamamoto","sequence":"additional","affiliation":[{"name":"Yahoo Japan Corporation, Tokyo, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4357-5835","authenticated-orcid":false,"given":"Yuta","family":"Saito","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,8,4]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1510489113"},{"key":"e_1_3_2_2_2_1","volume-title":"Classification and regression trees","author":"Breiman Leo","unstructured":"Leo Breiman , Jerome H Friedman , Richard A Olshen , and Charles J Stone . 2017. Classification and regression trees . Routledge . Leo Breiman, Jerome H Friedman, Richard A Olshen, and Charles J Stone. 2017. Classification and regression trees. Routledge."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1526709.1526711"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371819"},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-02294-4"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1341531.1341545"},{"key":"e_1_3_2_2_7_1","volume-title":"Hybrid Interest Modeling for Long-tailed Users. arXiv preprint arXiv:2012.14770","author":"Deng Lifang","year":"2020","unstructured":"Lifang Deng , Jin Niu , Angulia Yang , Qidi Xu , Xiang Fu , Jiandong Zhang , and Anxiang Zeng . 2020. Hybrid Interest Modeling for Long-tailed Users. arXiv preprint arXiv:2012.14770 ( 2020 ). Lifang Deng, Jin Niu, Angulia Yang, Qidi Xu, Xiang Fu, Jiandong Zhang, and Anxiang Zeng. 2020. Hybrid Interest Modeling for Long-tailed Users. arXiv preprint arXiv:2012.14770 (2020)."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Maria Dimakopoulou Nikos Vlassis and Tony Jebara. 2019. Marginal Posterior Sampling for Slate Bandits.. In IJCAI. 2223--2229.  Maria Dimakopoulou Nikos Vlassis and Tony Jebara. 2019. Marginal Posterior Sampling for Slate Bandits.. In IJCAI. 2223--2229.","DOI":"10.24963\/ijcai.2019\/308"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/3104482.3104620"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390392"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159687"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1498759.1498818"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1963405.1963412"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018699"},{"key":"e_1_3_2_2_16_1","volume-title":"Proceedings of the 38th International Conference on Machine Learning","volume":"139","author":"Kallus Nathan","year":"2021","unstructured":"Nathan Kallus , Yuta Saito , and Masatoshi Uehara . 2021 . Optimal Off-Policy Evaluation from Multiple Logging Policies . In Proceedings of the 38th International Conference on Machine Learning , Vol. 139 . PMLR, 5247--5256. Nathan Kallus, Yuta Saito, and Masatoshi Uehara. 2021. Optimal Off-Policy Evaluation from Multiple Logging Policies. In Proceedings of the 38th International Conference on Machine Learning, Vol. 139. PMLR, 5247--5256."},{"key":"e_1_3_2_2_17_1","volume-title":"Proceedings of the Conference on Health, Inference, and Learning","volume":"174","author":"Keramati Ramtin","year":"2022","unstructured":"Ramtin Keramati , Omer Gottesman , Leo Anthony Celi , Finale Doshi-Velez , and Emma Brunskill . 2022 . Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation . In Proceedings of the Conference on Health, Inference, and Learning , Vol. 174 . 397--410. Ramtin Keramati, Omer Gottesman, Leo Anthony Celi, Finale Doshi-Velez, and Emma Brunskill. 2022. Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation. In Proceedings of the Conference on Health, Inference, and Learning, Vol. 174. 397--410."},{"key":"e_1_3_2_2_18_1","volume-title":"Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation. arXiv preprint arXiv:2109.08331","author":"Kiyohara Haruka","year":"2021","unstructured":"Haruka Kiyohara , Kosuke Kawakami , and Yuta Saito . 2021. Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation. arXiv preprint arXiv:2109.08331 ( 2021 ). Haruka Kiyohara, Kosuke Kawakami, and Yuta Saito. 2021. Accelerating Offline Reinforcement Learning Application in Real-Time Bidding and Recommendation: Potential Use of Simulation. arXiv preprint arXiv:2109.08331 (2021)."},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498380"},{"key":"e_1_3_2_2_20_1","volume-title":"Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv preprint arXiv:2005.01643","author":"Levine Sergey","year":"2020","unstructured":"Sergey Levine , Aviral Kumar , George Tucker , and Justin Fu. 2020. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv preprint arXiv:2005.01643 ( 2020 ). Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv preprint arXiv:2005.01643 (2020)."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935878"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220028"},{"key":"e_1_3_2_2_23_1","volume-title":"Constructing Click Models for Mobile Search. In The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 775--784","author":"Mao Jiaxin","year":"2018","unstructured":"Jiaxin Mao , Cheng Luo , Min Zhang , and Shaoping Ma . 2018 . Constructing Click Models for Mobile Search. In The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 775--784 . Jiaxin Mao, Cheng Luo, Min Zhang, and Shaoping Ma. 2018. Constructing Click Models for Mobile Search. In The 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. 775--784."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403229"},{"key":"e_1_3_2_2_25_1","volume-title":"Proceedings of the 17th International Conference on Machine Learning. 759--766","author":"Precup Doina","unstructured":"Doina Precup , Richard S. Sutton , and Satinder P. Singh . 2000. Eligibility Traces for Off-Policy Policy Evaluation . In Proceedings of the 17th International Conference on Machine Learning. 759--766 . Doina Precup, Richard S. Sutton, and Satinder P. Singh. 2000. Eligibility Traces for Off-Policy Policy Evaluation. In Proceedings of the 17th International Conference on Machine Learning. 759--766."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3409256.3409812"},{"key":"e_1_3_2_2_27_1","volume-title":"Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation. arXiv preprint arXiv:2008.07146","author":"Saito Yuta","year":"2020","unstructured":"Yuta Saito , Shunsuke Aihara , Megumi Matsutani , and Yusuke Narita . 2020a. Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation. arXiv preprint arXiv:2008.07146 ( 2020 ). Yuta Saito, Shunsuke Aihara, Megumi Matsutani, and Yusuke Narita. 2020a. Open Bandit Dataset and Pipeline: Towards Realistic and Reproducible Off-Policy Evaluation. arXiv preprint arXiv:2008.07146 (2020)."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460231.3473320"},{"key":"e_1_3_2_2_29_1","volume-title":"Proceedings of the 39th International Conference on Machine Learning. 19089--19122","author":"Saito Yuta","year":"2022","unstructured":"Yuta Saito and Thorsten Joachims . 2022 . Off-Policy Evaluation for Large Action Spaces via Embeddings . In Proceedings of the 39th International Conference on Machine Learning. 19089--19122 . Yuta Saito and Thorsten Joachims. 2022. Off-Policy Evaluation for Large Action Spaces via Embeddings. In Proceedings of the 39th International Conference on Machine Learning. 19089--19122."},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401282"},{"key":"e_1_3_2_2_31_1","volume-title":"Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling. arXiv preprint arXiv:2305.08062","author":"Saito Yuta","year":"2023","unstructured":"Yuta Saito , Qingyang Ren , and Thorsten Joachims . 2023. Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling. arXiv preprint arXiv:2305.08062 ( 2023 ). Yuta Saito, Qingyang Ren, and Thorsten Joachims. 2023. Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling. arXiv preprint arXiv:2305.08062 (2023)."},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3460231.3474245"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371783"},{"key":"e_1_3_2_2_34_1","first-page":"2217","article-title":"Learning from Logged Implicit Exploration Data","volume":"23","author":"Strehl Alex","year":"2010","unstructured":"Alex Strehl , John Langford , Lihong Li , and Sham M Kakade . 2010 . Learning from Logged Implicit Exploration Data . In Advances in Neural Information Processing Systems , Vol. 23. 2217 -- 2225 . Alex Strehl, John Langford, Lihong Li, and Sham M Kakade. 2010. Learning from Logged Implicit Exploration Data. In Advances in Neural Information Processing Systems, Vol. 23. 2217--2225.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_35_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning","volume":"119","author":"Su Yi","year":"2020","unstructured":"Yi Su , Maria Dimakopoulou , Akshay Krishnamurthy , and Miroslav Dud\u00edk . 2020 . Doubly Robust Off-Policy Evaluation with Shrinkage . In Proceedings of the 37th International Conference on Machine Learning , Vol. 119 . PMLR, 9167--9176. Yi Su, Maria Dimakopoulou, Akshay Krishnamurthy, and Miroslav Dud\u00edk. 2020. Doubly Robust Off-Policy Evaluation with Shrinkage. In Proceedings of the 37th International Conference on Machine Learning, Vol. 119. PMLR, 9167--9176."},{"key":"e_1_3_2_2_36_1","first-page":"3632","article-title":"Off-Policy Evaluation for Slate Recommendation","volume":"30","author":"Swaminathan Adith","year":"2017","unstructured":"Adith Swaminathan , Akshay Krishnamurthy , Alekh Agarwal , Miro Dudik , John Langford , Damien Jose , and Imed Zitouni . 2017 . Off-Policy Evaluation for Slate Recommendation . In Advances in Neural Information Processing Systems , Vol. 30. 3632 -- 3642 . Adith Swaminathan, Akshay Krishnamurthy, Alekh Agarwal, Miro Dudik, John Langford, Damien Jose, and Imed Zitouni. 2017. Off-Policy Evaluation for Slate Recommendation. In Advances in Neural Information Processing Systems, Vol. 30. 3632--3642.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_37_1","volume-title":"Policy-Adaptive Estimator Selection for Off-Policy Evaluation. arXiv preprint arXiv:2211.13904","author":"Udagawa Takuma","year":"2022","unstructured":"Takuma Udagawa , Haruka Kiyohara , Yusuke Narita , Yuta Saito , and Kei Tateno . 2022. Policy-Adaptive Estimator Selection for Off-Policy Evaluation. arXiv preprint arXiv:2211.13904 ( 2022 ). Takuma Udagawa, Haruka Kiyohara, Yusuke Narita, Yuta Saito, and Kei Tateno. 2022. Policy-Adaptive Estimator Selection for Off-Policy Evaluation. arXiv preprint arXiv:2211.13904 (2022)."},{"key":"e_1_3_2_2_38_1","volume-title":"Fernando Amat Gil, and Ashok Chandrashekar","author":"Vlassis Nikos","year":"2021","unstructured":"Nikos Vlassis , Fernando Amat Gil, and Ashok Chandrashekar . 2021 . Off-Policy Evaluation of Slate Policies under Bayes Risk . arXiv preprint arXiv:2101.02553 (2021). Nikos Vlassis, Fernando Amat Gil, and Ashok Chandrashekar. 2021. Off-Policy Evaluation of Slate Policies under Bayes Risk. arXiv preprint arXiv:2101.02553 (2021)."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911537"},{"key":"e_1_3_2_2_40_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning. ICML, 3589--3597","author":"Wang Yu-Xiang","year":"2017","unstructured":"Yu-Xiang Wang , Alekh Agarwal , and Miroslav Dudik . 2017 . Optimal and Adaptive Off-policy Evaluation in Contextual Bandits , In Proceedings of the 34th International Conference on Machine Learning. ICML, 3589--3597 . Yu-Xiang Wang, Alekh Agarwal, and Miroslav Dudik. 2017. Optimal and Adaptive Off-policy Evaluation in Contextual Bandits, In Proceedings of the 34th International Conference on Machine Learning. ICML, 3589--3597."},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2124295.2124334"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3511950"},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449918"}],"event":{"name":"KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Long Beach CA USA","acronym":"KDD '23","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599447","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3580305.3599447","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:36Z","timestamp":1750178256000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3580305.3599447"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,4]]},"references-count":43,"alternative-id":["10.1145\/3580305.3599447","10.1145\/3580305"],"URL":"https:\/\/doi.org\/10.1145\/3580305.3599447","relation":{},"subject":[],"published":{"date-parts":[[2023,8,4]]},"assertion":[{"value":"2023-08-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}