{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T06:15:59Z","timestamp":1775283359837,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":37,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T00:00:00Z","timestamp":1631491200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Amazon.com"},{"name":"Center for Intelligent Information Retrieval at University of Massachusetts Amherst"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,13]]},"DOI":"10.1145\/3460231.3474271","type":"proceedings-article","created":{"date-parts":[[2021,9,13]],"date-time":"2021-09-13T21:45:04Z","timestamp":1631569504000},"page":"220-229","source":"Crossref","is-referenced-by-count":5,"title":["Large-scale Interactive Conversational Recommendation System using Actor-Critic Framework"],"prefix":"10.1145","author":[{"given":"Ali","family":"Montazeralghaem","sequence":"first","affiliation":[{"name":"University of Massachusetts Amherst, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Allan","sequence":"additional","affiliation":[{"name":"University of Massachusetts Amherst, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philip S.","family":"Thomas","sequence":"additional","affiliation":[{"name":"University of Massachusetts Amherst, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,9,13]]},"reference":[{"key":"e_1_3_2_2_1_1","unstructured":"Qingyao Ai Yongfeng Zhang Keping Bi Xu Chen and W\u00a0Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In SIGIR\u201917. 645\u2013654.  Qingyao Ai Yongfeng Zhang Keping Bi Xu Chen and W\u00a0Bruce Croft. 2017. Learning a hierarchical embedding model for personalized product search. In SIGIR\u201917. 645\u2013654."},{"key":"e_1_3_2_2_2_1","volume-title":"Dynamic programming. Science 153, 3731","author":"Bellman Richard","year":"1966","unstructured":"Richard Bellman . 1966. Dynamic programming. Science 153, 3731 ( 1966 ), 34\u201337. Richard Bellman. 1966. Dynamic programming. Science 153, 3731 (1966), 34\u201337."},{"key":"e_1_3_2_2_3_1","unstructured":"Keping Bi Qingyao Ai Yongfeng Zhang and W\u00a0Bruce Croft. 2019. Conversational product search based on negative feedback. In CIKM\u201919. 359\u2013368.  Keping Bi Qingyao Ai Yongfeng Zhang and W\u00a0Bruce Croft. 2019. Conversational product search based on negative feedback. In CIKM\u201919. 359\u2013368."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"crossref","unstructured":"Haokun Chen Xinyi Dai Han Cai Weinan Zhang Xuejian Wang Ruiming Tang Yuzhou Zhang and Yong Yu. 2019. Large-scale interactive recommendation with tree-structured policy gradient. In AAAI\u201919 Vol.\u00a033. 3312\u20133320.  Haokun Chen Xinyi Dai Han Cai Weinan Zhang Xuejian Wang Ruiming Tang Yuzhou Zhang and Yong Yu. 2019. Large-scale interactive recommendation with tree-structured policy gradient. In AAAI\u201919 Vol.\u00a033. 3312\u20133320.","DOI":"10.1609\/aaai.v33i01.33013312"},{"key":"e_1_3_2_2_5_1","unstructured":"Konstantina Christakopoulou Filip Radlinski and Katja Hofmann. 2016. Towards conversational recommender systems. In SIGKDD\u201916. 815\u2013824.  Konstantina Christakopoulou Filip Radlinski and Katja Hofmann. 2016. Towards conversational recommender systems. In SIGKDD\u201916. 815\u2013824."},{"key":"e_1_3_2_2_6_1","unstructured":"Gabriel Dulac-Arnold Richard Evans Hado van Hasselt Peter Sunehag Timothy Lillicrap Jonathan Hunt Timothy Mann Theophane Weber Thomas Degris and Ben Coppin. 2015. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679(2015).  Gabriel Dulac-Arnold Richard Evans Hado van Hasselt Peter Sunehag Timothy Lillicrap Jonathan Hunt Timothy Mann Theophane Weber Thomas Degris and Ben Coppin. 2015. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679(2015)."},{"key":"e_1_3_2_2_7_1","volume-title":"An Axiomatic Study of Query Terms Order in Ad-hoc Retrieval. In European Conference on Information Retrieval. Springer, 196\u2013202","author":"Imani Ayyoob","year":"2019","unstructured":"Ayyoob Imani , Amir Vakili , Ali Montazer , and Azadeh Shakery . 2019 . An Axiomatic Study of Query Terms Order in Ad-hoc Retrieval. In European Conference on Information Retrieval. Springer, 196\u2013202 . Ayyoob Imani, Amir Vakili, Ali Montazer, and Azadeh Shakery. 2019. An Axiomatic Study of Query Terms Order in Ad-hoc Retrieval. In European Conference on Information Retrieval. Springer, 196\u2013202."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-15719-7_26"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_2_10_1","unstructured":"Tom Kenter and Maarten de Rijke. 2017. Attentive memory networks: Efficient machine reading for conversational search. arXiv preprint arXiv:1712.07229(2017).  Tom Kenter and Maarten de Rijke. 2017. Attentive memory networks: Efficient machine reading for conversational search. arXiv preprint arXiv:1712.07229(2017)."},{"key":"e_1_3_2_2_11_1","volume-title":"Kingma and Jimmy Ba","author":"P.","year":"2015","unstructured":"Diederik\u00a0 P. Kingma and Jimmy Ba . 2015 . Adam : A Method for Stochastic Optimization. In ICLR\u201915 (San Diego, CA, USA) . Diederik\u00a0P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR\u201915 (San Diego, CA, USA)."},{"key":"e_1_3_2_2_12_1","unstructured":"Vijay\u00a0R Konda and John\u00a0N Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008\u20131014.  Vijay\u00a0R Konda and John\u00a0N Tsitsiklis. 2000. Actor-critic algorithms. In Advances in neural information processing systems. 1008\u20131014."},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371769"},{"key":"e_1_3_2_2_14_1","unstructured":"Raymond Li Samira\u00a0Ebrahimi Kahou Hannes Schulz Vincent Michalski Laurent Charlin and Chris Pal. 2018. Towards deep conversational recommendations. In Advances in neural information processing systems. 9725\u20139735.  Raymond Li Samira\u00a0Ebrahimi Kahou Hannes Schulz Vincent Michalski Laurent Charlin and Chris Pal. 2018. Towards deep conversational recommendations. In Advances in neural information processing systems. 9725\u20139735."},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45439-5_30"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401099"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914768"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080728"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"crossref","unstructured":"Jeffrey Pennington Richard Socher and Christopher\u00a0D. Manning. 2014. GloVe: Global Vectors for Word Representation. In EMNLP\u201914. 1532\u20131543.  Jeffrey Pennington Richard Socher and Christopher\u00a0D. Manning. 2014. GloVe: Global Vectors for Word Representation. In EMNLP\u201914. 1532\u20131543.","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Jay\u00a0M Ponte and W\u00a0Bruce Croft. 1998. A language modeling approach to information retrieval. In SIGIR\u201998. 275\u2013281.  Jay\u00a0M Ponte and W\u00a0Bruce Croft. 1998. A language modeling approach to information retrieval. In SIGIR\u201998. 275\u2013281.","DOI":"10.1145\/290941.291008"},{"key":"e_1_3_2_2_21_1","unstructured":"Ivaylo Popov Nicolas Heess Timothy Lillicrap Roland Hafner Gabriel Barth-Maron Matej Vecerik Thomas Lampe Yuval Tassa Tom Erez and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073(2017).  Ivaylo Popov Nicolas Heess Timothy Lillicrap Roland Hafner Gabriel Barth-Maron Matej Vecerik Thomas Lampe Yuval Tassa Tom Erez and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073(2017)."},{"key":"e_1_3_2_2_22_1","volume-title":"SIGIR\u201994","author":"Robertson E","unstructured":"Stephen\u00a0 E Robertson and Steve Walker . 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval . In SIGIR\u201994 . Springer , 232\u2013241. Stephen\u00a0E Robertson and Steve Walker. 1994. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In SIGIR\u201994. Springer, 232\u2013241."},{"key":"e_1_3_2_2_23_1","volume-title":"Learning representations by back-propagating errors. nature 323, 6088","author":"Rumelhart E","year":"1986","unstructured":"David\u00a0 E Rumelhart , Geoffrey\u00a0 E Hinton , and Ronald\u00a0 J Williams . 1986. Learning representations by back-propagating errors. nature 323, 6088 ( 1986 ), 533. David\u00a0E Rumelhart, Geoffrey\u00a0E Hinton, and Ronald\u00a0J Williams. 1986. Learning representations by back-propagating errors. nature 323, 6088 (1986), 533."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0269888903000638"},{"key":"e_1_3_2_2_25_1","volume-title":"Julian Schrittwieser, Ioannis Antonoglou","author":"Silver David","year":"2016","unstructured":"David Silver , Aja Huang , Chris\u00a0 J Maddison , Arthur Guez , Laurent Sifre , George Van Den\u00a0Driessche , Julian Schrittwieser, Ioannis Antonoglou , Veda Panneershelvam, Marc Lanctot , 2016 . Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2016), 484. David Silver, Aja Huang, Chris\u00a0J Maddison, Arthur Guez, Laurent Sifre, George Van Den\u00a0Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, 2016. Mastering the game of Go with deep neural networks and tree search. nature 529, 7587 (2016), 484."},{"key":"e_1_3_2_2_26_1","volume-title":"Mastering the game of Go without human knowledge. Nature 550, 7676","author":"Silver David","year":"2017","unstructured":"David Silver , Julian Schrittwieser , Karen Simonyan , Ioannis Antonoglou , Aja Huang , Arthur Guez , Thomas Hubert , Lucas Baker , Matthew Lai , Adrian Bolton , 2017. Mastering the game of Go without human knowledge. Nature 550, 7676 ( 2017 ), 354. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, 2017. Mastering the game of Go without human knowledge. Nature 550, 7676 (2017), 354."},{"key":"e_1_3_2_2_27_1","volume-title":"Mastering the game of go without human knowledge. Nature 550, 7676","author":"Silver David","year":"2017","unstructured":"David Silver , Julian Schrittwieser , Karen Simonyan , Ioannis Antonoglou , Aja Huang , Arthur Guez , Thomas Hubert , Lucas Baker , Matthew Lai , Adrian Bolton , 2017. Mastering the game of go without human knowledge. Nature 550, 7676 ( 2017 ), 354\u2013359. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, 2017. Mastering the game of go without human knowledge. Nature 550, 7676 (2017), 354\u2013359."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210002"},{"key":"e_1_3_2_2_29_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton S","unstructured":"Richard\u00a0 S Sutton and Andrew\u00a0 G Barto . 2018. Reinforcement learning: An introduction . MIT press . Richard\u00a0S Sutton and Andrew\u00a0G Barto. 2018. Reinforcement learning: An introduction. MIT press."},{"key":"e_1_3_2_2_30_1","volume-title":"Introduction to reinforcement learning. Vol.\u00a0135","author":"Sutton S","unstructured":"Richard\u00a0 S Sutton , Andrew\u00a0 G Barto , 1998. Introduction to reinforcement learning. Vol.\u00a0135 . MIT press Cambridge . Richard\u00a0S Sutton, Andrew\u00a0G Barto, 1998. Introduction to reinforcement learning. Vol.\u00a0135. MIT press Cambridge."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Christophe Van\u00a0Gysel Maarten de Rijke and Evangelos Kanoulas. 2016. Learning latent vector spaces for product search. In CIKM\u201916. 165\u2013174.  Christophe Van\u00a0Gysel Maarten de Rijke and Evangelos Kanoulas. 2016. Learning latent vector spaces for product search. In CIKM\u201916. 165\u2013174.","DOI":"10.1145\/2983323.2983702"},{"key":"e_1_3_2_2_32_1","volume-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3-4","author":"Williams J","year":"1992","unstructured":"Ronald\u00a0 J Williams . 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3-4 ( 1992 ), 229\u2013256. Ronald\u00a0J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3-4 (1992), 229\u2013256."},{"key":"e_1_3_2_2_33_1","unstructured":"Liu Yang Hamed Zamani Yongfeng Zhang Jiafeng Guo and W\u00a0Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. arXiv preprint arXiv:1707.05409(2017).  Liu Yang Hamed Zamani Yongfeng Zhang Jiafeng Guo and W\u00a0Bruce Croft. 2017. Neural matching models for question retrieval and next question prediction in conversation. arXiv preprint arXiv:1707.05409(2017)."},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Yongfeng Zhang Xu Chen Qingyao Ai Liu Yang and W\u00a0Bruce Croft. 2018. Towards conversational search and recommendation: System ask user respond. In CIKM\u201918. 177\u2013186.  Yongfeng Zhang Xu Chen Qingyao Ai Liu Yang and W\u00a0Bruce Croft. 2018. Towards conversational search and recommendation: System ask user respond. In CIKM\u201918. 177\u2013186.","DOI":"10.1145\/3269206.3271776"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609579"},{"key":"e_1_3_2_2_36_1","volume-title":"Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 1027\u20131030","author":"Zhang Yongfeng","year":"2014","unstructured":"Yongfeng Zhang , Haochen Zhang , Min Zhang , Yiqun Liu , and Shaoping Ma . 2014 . Do users rate or review? Boost phrase-level sentiment labeling with review-level sentiment classification . In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 1027\u20131030 . Yongfeng Zhang, Haochen Zhang, Min Zhang, Yiqun Liu, and Shaoping Ma. 2014. Do users rate or review? Boost phrase-level sentiment labeling with review-level sentiment classification. In Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval. 1027\u20131030."},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240374"}],"event":{"name":"RecSys '21: Fifteenth ACM Conference on Recommender Systems","location":"Amsterdam Netherlands","acronym":"RecSys '21","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGAI ACM Special Interest Group on Artificial Intelligence","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGIR ACM Special Interest Group on Information Retrieval","SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGecom Special Interest Group on Economics and Computation"]},"container-title":["Fifteenth ACM Conference on Recommender Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460231.3474271","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3460231.3474271","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:12:17Z","timestamp":1750191137000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3460231.3474271"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,13]]},"references-count":37,"alternative-id":["10.1145\/3460231.3474271","10.1145\/3460231"],"URL":"https:\/\/doi.org\/10.1145\/3460231.3474271","relation":{},"subject":[],"published":{"date-parts":[[2021,9,13]]}}}