{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T06:48:40Z","timestamp":1783579720133,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":45,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T00:00:00Z","timestamp":1595635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,25]]},"DOI":"10.1145\/3397271.3401147","type":"proceedings-article","created":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T07:50:08Z","timestamp":1595663408000},"page":"931-940","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":162,"title":["Self-Supervised Reinforcement Learning for Recommender Systems"],"prefix":"10.1145","author":[{"given":"Xin","family":"Xin","sequence":"first","affiliation":[{"name":"University of Glasgow, Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexandros","family":"Karatzoglou","sequence":"additional","affiliation":[{"name":"Google, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ioannis","family":"Arapakis","sequence":"additional","affiliation":[{"name":"Telefonica Research, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joemon M.","family":"Jose","sequence":"additional","affiliation":[{"name":"University of Glasgow, Galsgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,7,25]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Science","volume":"153","author":"Bellman Richard","year":"1966","unstructured":"Richard Bellman . 1966 . Dynamic programming . Science , Vol. 153 , 3731 (1966), 34--37. Richard Bellman. 1966. Dynamic programming. Science, Vol. 153, 3731 (1966), 34--37."},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the Twelfth Irish Conference on Artificial Intelligence and Cognitive Science","author":"Bradley Keith","year":"2001","unstructured":"Keith Bradley and Barry Smyth . 2001 . Improving recommendation diversity . In Proceedings of the Twelfth Irish Conference on Artificial Intelligence and Cognitive Science , Maynooth, Ireland. Citeseer, 85--94. Keith Bradley and Barry Smyth. 2001. Improving recommendation diversity. In Proceedings of the Twelfth Irish Conference on Artificial Intelligence and Cognitive Science, Maynooth, Ireland. Citeseer, 85--94."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290999"},{"key":"e_1_3_2_1_4_1","volume-title":"Generative Adversarial User Model for Reinforcement Learning Based Recommendation System. In International Conference on Machine Learning. 1052--1061","author":"Chen Xinshi","year":"2019","unstructured":"Xinshi Chen , Shuang Li , Hui Li , Shaohua Jiang , Yuan Qi , and Le Song . 2019 b . Generative Adversarial User Model for Reinforcement Learning Based Recommendation System. In International Conference on Machine Learning. 1052--1061 . Xinshi Chen, Shuang Li, Hui Li, Shaohua Jiang, Yuan Qi, and Le Song. 2019 b. Generative Adversarial User Model for Reinforcement Learning Based Recommendation System. In International Conference on Machine Learning. 1052--1061."},{"key":"e_1_3_2_1_5_1","unstructured":"Chen Cheng Haiqin Yang Michael R Lyu and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In Twenty-Third international joint conference on Artificial Intelligence.  Chen Cheng Haiqin Yang Michael R Lyu and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In Twenty-Third international joint conference on Artificial Intelligence."},{"key":"e_1_3_2_1_6_1","volume-title":"Dzmitry Bahdanau, and Yoshua Bengio.","author":"Cho Kyunghyun","year":"2014","unstructured":"Kyunghyun Cho , Bart Van Merri\u00ebnboer , Dzmitry Bahdanau, and Yoshua Bengio. 2014 . On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014). Kyunghyun Cho, Bart Van Merri\u00ebnboer, Dzmitry Bahdanau, and Yoshua Bengio. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259 (2014)."},{"key":"e_1_3_2_1_7_1","volume-title":"Off-policy deep reinforcement learning without exploration. arXiv preprint arXiv:1812.02900","author":"Fujimoto Scott","year":"2018","unstructured":"Scott Fujimoto , David Meger , and Doina Precup . 2018. Off-policy deep reinforcement learning without exploration. arXiv preprint arXiv:1812.02900 ( 2018 ). Scott Fujimoto, David Meger, and Doina Precup. 2018. Off-policy deep reinforcement learning without exploration. arXiv preprint arXiv:1812.02900 (2018)."},{"key":"e_1_3_2_1_8_1","volume-title":"Exact-K Recommendation via Maximal Clique Optimization. arXiv preprint arXiv:1905.07089","author":"Gong Yu","year":"2019","unstructured":"Yu Gong , Yu Zhu , Lu Duan , Qingwen Liu , Ziyu Guan , Fei Sun , Wenwu Ou , and Kenny Q Zhu . 2019. Exact-K Recommendation via Maximal Clique Optimization. arXiv preprint arXiv:1905.07089 ( 2019 ). Yu Gong, Yu Zhu, Lu Duan, Qingwen Liu, Ziyu Guan, Fei Sun, Wenwu Ou, and Kenny Q Zhu. 2019. Exact-K Recommendation via Maximal Clique Optimization. arXiv preprint arXiv:1905.07089 (2019)."},{"key":"e_1_3_2_1_9_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.  Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680."},{"key":"e_1_3_2_1_10_1","unstructured":"Hado V Hasselt. 2010. Double Q-learning. In Advances in Neural Information Processing Systems. 2613--2621.  Hado V Hasselt. 2010. Double Q-learning. In Advances in Neural Information Processing Systems. 2613--2621."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959152"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0030"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271761"},{"key":"e_1_3_2_1_14_1","volume-title":"Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939","author":"Hidasi Bal\u00e1zs","year":"2015","unstructured":"Bal\u00e1zs Hidasi , Alexandros Karatzoglou , Linas Baltrunas , and Domonkos Tikk . 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 ( 2015 ). Bal\u00e1zs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-015-0417-y"},{"key":"e_1_3_2_1_16_1","unstructured":"Jonathan Ho and Stefano Ermon. 2016. Generative adversarial imitation learning. In Advances in neural information processing systems. 4565--4573.  Jonathan Ho and Stefano Ermon. 2016. Generative adversarial imitation learning. In Advances in neural information processing systems. 4565--4573."},{"key":"e_1_3_2_1_17_1","volume-title":"International Conference on Machine Learning. 2760--2769","author":"Ho Jonathan","year":"2016","unstructured":"Jonathan Ho , Jayesh Gupta , and Stefano Ermon . 2016 . Model-free imitation learning with policy optimization . In International Conference on Machine Learning. 2760--2769 . Jonathan Ho, Jayesh Gupta, and Stefano Ermon. 2016. Model-free imitation learning with policy optimization. In International Conference on Machine Learning. 2760--2769."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219846"},{"key":"e_1_3_2_1_19_1","unstructured":"Eugene Ie Vihan Jain Jing Wang Sanmit Narvekar Ritesh Agarwal Rui Wu Heng-Tze Cheng Tushar Chandra and Craig Boutilier. 2019. SlateQ: A tractable decomposition for reinforcement learning with recommendation sets. (2019).  Eugene Ie Vihan Jain Jing Wang Sanmit Narvekar Ritesh Agarwal Rui Wu Heng-Tze Cheng Tushar Chandra and Craig Boutilier. 2019. SlateQ: A tractable decomposition for reinforcement learning with recommendation sets. (2019)."},{"key":"e_1_3_2_1_20_1","volume-title":"ACM Transactions on Information Systems (TOIS)","volume":"20","author":"Jaana Kalervo","year":"2002","unstructured":"Kalervo J\"arvelin and Jaana Kek\"al\"ainen. 2002 . Cumulated gain-based evaluation of IR techniques . ACM Transactions on Information Systems (TOIS) , Vol. 20 , 4 (2002), 422--446. Kalervo J\"arvelin and Jaana Kek\"al\"ainen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), Vol. 20, 4 (2002), 422--446."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00035"},{"key":"e_1_3_2_1_22_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_3_2_1_23_1","unstructured":"Vijay R Konda and John N Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008--1014.  Vijay R Konda and John N Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008--1014."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935878"},{"key":"e_1_3_2_1_27_1","volume-title":"Nature","volume":"518","author":"Mnih Volodymyr","year":"2015","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Andrei A Rusu , Joel Veness , Marc G Bellemare , Alex Graves , Martin Riedmiller , Andreas K Fidjeland , Georg Ostrovski , 2015 . Human-level control through deep reinforcement learning . Nature , Vol. 518 , 7540 (2015), 529. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. 2015. Human-level control through deep reinforcement learning. Nature, Vol. 518, 7540 (2015), 529."},{"key":"e_1_3_2_1_28_1","unstructured":"R\u00e9mi Munos Tom Stepleton Anna Harutyunyan and Marc Bellemare. 2016. Safe and efficient off-policy reinforcement learning. In Advances in Neural Information Processing Systems. 1054--1062.  R\u00e9mi Munos Tom Stepleton Anna Harutyunyan and Marc Bellemare. 2016. Safe and efficient off-policy reinforcement learning. In Advances in Neural Information Processing Systems. 1054--1062."},{"key":"e_1_3_2_1_29_1","volume-title":"Aidan Clark, Seb Noury, et al.","author":"Parisotto Emilio","year":"2019","unstructured":"Emilio Parisotto , H Francis Song , Jack W Rae , Razvan Pascanu , Caglar Gulcehre , Siddhant M Jayakumar , Max Jaderberg , Raphael Lopez Kaufman , Aidan Clark, Seb Noury, et al. 2019 . Stabilizing Transformers for Reinforcement Learning . arXiv preprint arXiv:1910.06764 (2019). Emilio Parisotto, H Francis Song, Jack W Rae, Razvan Pascanu, Caglar Gulcehre, Siddhant M Jayakumar, Max Jaderberg, Raphael Lopez Kaufman, Aidan Clark, Seb Noury, et al. 2019. Stabilizing Transformers for Reinforcement Learning. arXiv preprint arXiv:1910.06764 (2019)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2010.127"},{"key":"e_1_3_2_1_31_1","volume-title":"Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452--461","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle , Christoph Freudenthaler , Zeno Gantner , and Lars Schmidt-Thieme . 2009 . BPR: Bayesian personalized ranking from implicit feedback . In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452--461 . Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, 452--461."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772773"},{"key":"e_1_3_2_1_33_1","volume-title":"RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising. arXiv preprint arXiv:1808.00720","author":"Rohde David","year":"2018","unstructured":"David Rohde , Stephen Bonner , Travis Dunlop , Flavian Vasile , and Alexandros Karatzoglou . 2018. RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising. arXiv preprint arXiv:1808.00720 ( 2018 ). David Rohde, Stephen Bonner, Travis Dunlop, Flavian Vasile, and Alexandros Karatzoglou. 2018. RecoGym: A Reinforcement Learning Environment for the problem of Product Recommendation in Online Advertising. arXiv preprint arXiv:1808.00720 (2018)."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330933"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/1046920.1088715"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014902"},{"key":"e_1_3_2_1_37_1","volume-title":"Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al.","author":"Silver David","year":"2016","unstructured":"David Silver , Aja Huang , Chris J Maddison , Arthur Guez , Laurent Sifre , George Van Den Driessche , Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016 . Mastering the game of Go with deep neural networks and tree search. nature, Vol. 529 , 7587 (2016), 484. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of Go with deep neural networks and tree search. nature, Vol. 529, 7587 (2016), 484."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159656"},{"key":"e_1_3_2_1_39_1","volume-title":"Behavioral cloning from observation. arXiv preprint arXiv:1805.01954","author":"Torabi Faraz","year":"2018","unstructured":"Faraz Torabi , Garrett Warnell , and Peter Stone . 2018. Behavioral cloning from observation. arXiv preprint arXiv:1805.01954 ( 2018 ). Faraz Torabi, Garrett Warnell, and Peter Stone. 2018. Behavioral cloning from observation. arXiv preprint arXiv:1805.01954 (2018)."},{"key":"e_1_3_2_1_40_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_3_2_1_41_1","volume-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning","author":"Williams Ronald J","year":"1992","unstructured":"Ronald J Williams . 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning , Vol. 8 , 3--4 ( 1992 ), 229--256. Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, Vol. 8, 3--4 (1992), 229--256."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290975"},{"key":"e_1_3_2_1_43_1","unstructured":"Fajie Yuan Xin Xin Xiangnan He Guibing Guo Weinan Zhang Chua Tat-Seng and Joemon M Jose. 2018. fBGD: Learning embeddings from positive unlabeled data with BGD. (2018).  Fajie Yuan Xin Xin Xiangnan He Guibing Guo Weinan Zhang Chua Tat-Seng and Joemon M Jose. 2018. fBGD: Learning embeddings from positive unlabeled data with BGD. (2018)."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219886"},{"key":"e_1_3_2_1_45_1","volume-title":"Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems. arXiv preprint arXiv:1902.05570","author":"Zou Lixin","year":"2019","unstructured":"Lixin Zou , Long Xia , Zhuoye Ding , Jiaxing Song , Weidong Liu , and Dawei Yin . 2019. Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems. arXiv preprint arXiv:1902.05570 ( 2019 ). Lixin Zou, Long Xia, Zhuoye Ding, Jiaxing Song, Weidong Liu, and Dawei Yin. 2019. Reinforcement Learning to Optimize Long-term User Engagement in Recommender Systems. arXiv preprint arXiv:1902.05570 (2019)."}],"event":{"name":"SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval","location":"Virtual Event China","acronym":"SIGIR '20","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401147","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397271.3401147","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:43Z","timestamp":1750200103000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401147"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,25]]},"references-count":45,"alternative-id":["10.1145\/3397271.3401147","10.1145\/3397271"],"URL":"https:\/\/doi.org\/10.1145\/3397271.3401147","relation":{},"subject":[],"published":{"date-parts":[[2020,7,25]]},"assertion":[{"value":"2020-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}