{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T17:36:55Z","timestamp":1790357815126,"version":"4.1.0"},"publisher-location":"New York, NY, USA","reference-count":44,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,10,17]],"date-time":"2022-10-17T00:00:00Z","timestamp":1665964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,10,17]]},"DOI":"10.1145\/3511808.3557083","type":"proceedings-article","created":{"date-parts":[[2022,10,16]],"date-time":"2022-10-16T01:29:57Z","timestamp":1665883797000},"page":"3614-3623","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Offline Reinforcement Learning for Mobile Notifications"],"prefix":"10.1145","author":[{"given":"Yiping","family":"Yuan","sequence":"first","affiliation":[{"name":"LinkedIn Corporation, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ajith","family":"Muralidharan","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Preetam","family":"Nandy","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miao","family":"Cheng","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prakruthi","family":"Prabhakar","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,10,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2020408.2020435"},{"key":"e_1_3_2_1_2_1","unstructured":"Rishabh Agarwal Dale Schuurmans and Mohammad Norouzi. 2019. Striving for simplicity in off-policy deep reinforcement learning. (2019).  Rishabh Agarwal Dale Schuurmans and Mohammad Norouzi. 2019. Striving for simplicity in off-policy deep reinforcement learning. (2019)."},{"key":"e_1_3_2_1_3_1","volume-title":"Bayesian Optimization for Balancing Metrics in Recommender Systems. International Joint Conference on Artificial Intelligence (2020","author":"Basu Kinjal","year":"2020","unstructured":"Kinjal Basu , Cyrus DiCiccio , Brendan Gavin , Viral Gupta , and Yunbo Ouyang . 2020 . Bayesian Optimization for Balancing Metrics in Recommender Systems. International Joint Conference on Artificial Intelligence (2020 ). https:\/\/sites.google.com\/view\/ijcai2020-linkedin-bayesopt\/home Kinjal Basu, Cyrus DiCiccio, Brendan Gavin, Viral Gupta, and Yunbo Ouyang. 2020. Bayesian Optimization for Balancing Metrics in Recommender Systems. International Joint Conference on Artificial Intelligence (2020). https:\/\/sites.google.com\/view\/ijcai2020-linkedin-bayesopt\/home"},{"key":"e_1_3_2_1_4_1","volume-title":"abs\/1606.01540","author":"Brockman Greg","year":"2016","unstructured":"Greg Brockman , Vicki Cheung , Ludwig Pettersson , Jonas Schneider , John Schulman , Jie Tang , and Wojciech Zaremba . 2016. Open AI Gym . Co RR , Vol . abs\/1606.01540 ( 2016 ). arxiv: 1606.01540 http:\/\/arxiv.org\/abs\/1606.01540 Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. CoRR, Vol. abs\/1606.01540 (2016). arxiv: 1606.01540 http:\/\/arxiv.org\/abs\/1606.01540"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290999"},{"key":"e_1_3_2_1_6_1","volume-title":"International Conference on Machine Learning. PMLR","author":"Fujimoto Scott","year":"2019","unstructured":"Scott Fujimoto , David Meger , and Doina Precup . 2019 . Off-policy deep reinforcement learning without exploration . In International Conference on Machine Learning. PMLR , 2052--2062. Scott Fujimoto, David Meger, and Doina Precup. 2019. Off-policy deep reinforcement learning without exploration. In International Conference on Machine Learning. PMLR, 2052--2062."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219880"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132849"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939692"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330754"},{"key":"e_1_3_2_1_11_1","unstructured":"Eugene Ie Vihan Jain Jing Wang Sanmit Narvekar Ritesh Agarwal Rui Wu Heng-Tze Cheng Morgane Lustman Vince Gatto Paul Covington etal 2019. Reinforcement learning for slate-based recommender systems: A tractable decomposition and practical methodology. arXiv preprint arXiv:1905.12767 (2019).  Eugene Ie Vihan Jain Jing Wang Sanmit Narvekar Ritesh Agarwal Rui Wu Heng-Tze Cheng Morgane Lustman Vince Gatto Paul Covington et al. 2019. Reinforcement learning for slate-based recommender systems: A tractable decomposition and practical methodology. arXiv preprint arXiv:1905.12767 (2019)."},{"key":"e_1_3_2_1_12_1","volume-title":"International Conference on Machine Learning. PMLR, 652--661","author":"Jiang Nan","year":"2016","unstructured":"Nan Jiang and Lihong Li . 2016 . Doubly robust off-policy value evaluation for reinforcement learning . In International Conference on Machine Learning. PMLR, 652--661 . Nan Jiang and Lihong Li. 2016. Doubly robust off-policy value evaluation for reinforcement learning. In International Conference on Machine Learning. PMLR, 652--661."},{"key":"e_1_3_2_1_13_1","volume-title":"Conservative q-learning for offline reinforcement learning. arXiv preprint arXiv:2006.04779","author":"Kumar Aviral","year":"2020","unstructured":"Aviral Kumar , Aurick Zhou , George Tucker , and Sergey Levine . 2020. Conservative q-learning for offline reinforcement learning. arXiv preprint arXiv:2006.04779 ( 2020 ). Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. 2020. Conservative q-learning for offline reinforcement learning. arXiv preprint arXiv:2006.04779 (2020)."},{"key":"e_1_3_2_1_14_1","volume-title":"Reinforcement learning","author":"Lange Sascha","unstructured":"Sascha Lange , Thomas Gabel , and Martin Riedmiller . 2012. Batch reinforcement learning . In Reinforcement learning . Springer , 45--73. Sascha Lange, Thomas Gabel, and Martin Riedmiller. 2012. Batch reinforcement learning. In Reinforcement learning. Springer, 45--73."},{"key":"e_1_3_2_1_15_1","volume-title":"Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643","author":"Levine Sergey","year":"2020","unstructured":"Sergey Levine , Aviral Kumar , George Tucker , and Justin Fu. 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 ( 2020 ). Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline reinforcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 (2020)."},{"key":"e_1_3_2_1_16_1","volume-title":"Hado Van Hasselt, and Richard S Sutton","author":"Mahmood Ashique Rupam","year":"2014","unstructured":"Ashique Rupam Mahmood , Hado Van Hasselt, and Richard S Sutton . 2014 . Weighted importance sampling for off-policy learning with linear function approximation. In NIPS. 3014--3022. Ashique Rupam Mahmood, Hado Van Hasselt, and Richard S Sutton. 2014. Weighted importance sampling for off-policy learning with linear function approximation. In NIPS. 3014--3022."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807544"},{"key":"e_1_3_2_1_18_1","volume-title":"Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602","author":"Mnih Volodymyr","year":"2013","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Alex Graves , Ioannis Antonoglou , Daan Wierstra , and Martin Riedmiller . 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 ( 2013 ). Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137770"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/PERCOM.2015.7146515"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2807517"},{"key":"e_1_3_2_1_22_1","volume-title":"Attention and engagement-awareness in the wild: A large-scale study with adaptive notifications. In 2017 ieee international conference on pervasive computing and communications (percom)","author":"Okoshi Tadashi","unstructured":"Tadashi Okoshi , Kota Tsubouchi , Masaya Taji , Takanori Ichikawa , and Hideyuki Tokuda . 2017. Attention and engagement-awareness in the wild: A large-scale study with adaptive notifications. In 2017 ieee international conference on pervasive computing and communications (percom) . IEEE , 100--110. Tadashi Okoshi, Kota Tsubouchi, Masaya Taji, Takanori Ichikawa, and Hideyuki Tokuda. 2017. Attention and engagement-awareness in the wild: A large-scale study with adaptive notifications. In 2017 ieee international conference on pervasive computing and communications (percom). IEEE, 100--110."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330732"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2632048.2632062"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130956"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2628363.2628364"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2556973"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2750858.2804252"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/0303-2647(95)01551-5"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014902"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"key":"e_1_3_2_1_32_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton Richard S","unstructured":"Richard S Sutton and Andrew G Barto . 1998. Reinforcement learning: An introduction . MIT press Cambridge . Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction. MIT press Cambridge."},{"key":"e_1_3_2_1_33_1","volume-title":"Off-policy evaluation for slate recommendation. arXiv preprint arXiv:1605.04812","author":"Swaminathan Adith","year":"2016","unstructured":"Adith Swaminathan , Akshay Krishnamurthy , Alekh Agarwal , Miroslav Dud'ik , John Langford , Damien Jose , and Imed Zitouni . 2016. Off-policy evaluation for slate recommendation. arXiv preprint arXiv:1605.04812 ( 2016 ). Adith Swaminathan, Akshay Krishnamurthy, Alekh Agarwal, Miroslav Dud'ik, John Langford, Damien Jose, and Imed Zitouni. 2016. Off-policy evaluation for slate recommendation. arXiv preprint arXiv:1605.04812 (2016)."},{"key":"e_1_3_2_1_34_1","volume-title":"International Conference on Machine Learning. PMLR, 2139--2148","author":"Thomas Philip","year":"2016","unstructured":"Philip Thomas and Emma Brunskill . 2016 . Data-efficient off-policy policy evaluation for reinforcement learning . In International Conference on Machine Learning. PMLR, 2139--2148 . Philip Thomas and Emma Brunskill. 2016. Data-efficient off-policy policy evaluation for reinforcement learning. In International Conference on Machine Learning. PMLR, 2139--2148."},{"key":"e_1_3_2_1_35_1","volume-title":"Deep reinforcement learning and the deadly triad. arXiv preprint arXiv:1812.02648","author":"Hasselt Hado Van","year":"2018","unstructured":"Hado Van Hasselt , Yotam Doron , Florian Strub , Matteo Hessel , Nicolas Sonnerat , and Joseph Modayil . 2018. Deep reinforcement learning and the deadly triad. arXiv preprint arXiv:1812.02648 ( 2018 ). Hado Van Hasselt, Yotam Doron, Florian Strub, Matteo Hessel, Nicolas Sonnerat, and Joseph Modayil. 2018. Deep reinforcement learning and the deadly triad. arXiv preprint arXiv:1812.02648 (2018)."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403128"},{"key":"e_1_3_2_1_38_1","volume-title":"Proceedings of The 33rd International Conference on Machine Learning. 1995--2003","author":"Wang Ziyu","year":"2016","unstructured":"Ziyu Wang , Tom Schaul , Matteo Hessel , Hado Hasselt , Marc Lanctot , and Nando Freitas . 2016 . Dueling Network Architectures for Deep Reinforcement Learning . In Proceedings of The 33rd International Conference on Machine Learning. 1995--2003 . Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016. Dueling Network Architectures for Deep Reinforcement Learning. In Proceedings of The 33rd International Conference on Machine Learning. 1995--2003."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3133025"},{"key":"e_1_3_2_1_40_1","volume-title":"Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. arXiv preprint arXiv:1906.03393","author":"Xie Tengyang","year":"2019","unstructured":"Tengyang Xie , Yifei Ma , and Yu-Xiang Wang . 2019. Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. arXiv preprint arXiv:1906.03393 ( 2019 ). Tengyang Xie, Yifei Ma, and Yu-Xiang Wang. 2019. Towards optimal off-policy evaluation for reinforcement learning with marginalized importance sampling. arXiv preprint arXiv:1906.03393 (2019)."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290981"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219906"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219886"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330668"}],"event":{"name":"CIKM '22: The 31st ACM International Conference on Information and Knowledge Management","location":"Atlanta GA USA","acronym":"CIKM '22","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 31st ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557083","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511808.3557083","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:56Z","timestamp":1750188656000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511808.3557083"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,17]]},"references-count":44,"alternative-id":["10.1145\/3511808.3557083","10.1145\/3511808"],"URL":"https:\/\/doi.org\/10.1145\/3511808.3557083","relation":{},"subject":[],"published":{"date-parts":[[2022,10,17]]},"assertion":[{"value":"2022-10-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}