{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T05:20:49Z","timestamp":1784092849500,"version":"3.55.0"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,10]],"date-time":"2024-04-10T00:00:00Z","timestamp":1712707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Recomm. Syst."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>\n            A key distinguishing feature of conversational recommender systems over traditional recommender systems is theirability to elicit user preferences using natural language. Currently, the predominant approach to preference elicitation is to ask questions directly about items or item attributes. Users searching for recommendations may not have deep knowledge of the available options in a given domain. As such, they might not be aware of key attributes or desirable values for them. However, in many settings, talking about the\n            <jats:italic>planned use<\/jats:italic>\n            of items does not present any difficulties, even for those that are new to a domain. In this article, we propose a novel approach to preference elicitation by asking implicit questions based on item usage. As one of the main contributions of this work, we develop a multi-stage data annotation protocol using crowdsourcing, to create a high-quality labeled training dataset. Another main contribution is the development of four models for the question generation task: two template-based baseline models and two neural text-to-text models. The template-based models use heuristically extracted common patterns found in the training data, while the neural models use the training data to learn to generate questions automatically. Using common metrics from machine translation for automatic evaluation, we show that our approaches are effective in generating elicitation questions, even with limited training data. We further employ human evaluation for comparing the generated questions using both pointwise and pairwise evaluation designs. We find that the human evaluation results are consistent with the automatic ones, allowing us to draw conclusions about the quality of the generated questions with certainty. Finally, we provide a detailed analysis of cases where the models show their limitations.\n          <\/jats:p>","DOI":"10.1145\/3629981","type":"journal-article","created":{"date-parts":[[2023,10,27]],"date-time":"2023-10-27T22:05:27Z","timestamp":1698444327000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Generating Usage-related Questions for Preference Elicitation in Conversational Recommender Systems"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0859-9762","authenticated-orcid":false,"given":"Ivica","family":"Kostric","sequence":"first","affiliation":[{"name":"University of Stavanger, Stavanger, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2762-721X","authenticated-orcid":false,"given":"Krisztian","family":"Balog","sequence":"additional","affiliation":[{"name":"University of Stavanger, Stavanger, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1386-9288","authenticated-orcid":false,"given":"Filip","family":"Radlinski","sequence":"additional","affiliation":[{"name":"Google, London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,10]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1145\/3331184.3331265","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Aliannejadi Mohammad","year":"2019","unstructured":"Mohammad Aliannejadi, Hamed Zamani, Fabio Crestani, and W. Bruce Croft. 2019. Asking clarifying questions in open\u2013 domain information\u2013 seeking conversations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.475\u2013484."},{"issue":"11","key":"e_1_3_2_3_2","first-page":"34","article-title":"Conversational search (dagstuhl seminar 19461)","volume":"9","author":"Anand Avishek","year":"2020","unstructured":"Avishek Anand, Lawrence Cavedon, Hideo Joho, Mark Sanderson, and Benno Stein. 2020. Conversational search (dagstuhl seminar 19461). Dagstuhl Reports 9, 11 (2020), 34\u201383.","journal-title":"Dagstuhl Reports"},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"890","DOI":"10.1145\/3404835.3462893","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Balog Krisztian","year":"2021","unstructured":"Krisztian Balog, Filip Radlinski, and Alexandros Karatzoglou. 2021. On interpretation and measurement of soft attributes for recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.890\u2013899."},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1145\/3295750.3298929","volume-title":"Proceedings of the 2019 Conference on Human Information Interaction and Retrieval.","author":"Benetka Jan R.","year":"2019","unstructured":"Jan R. Benetka, John Krumm, and Paul N. Bennett. 2019. Understanding context for tasks and activities. In Proceedings of the 2019 Conference on Human Information Interaction and Retrieval.133\u2013142."},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/978-3-642-41338-4_3","volume-title":"Proceedings of the Semantic Web \u2013 ISWC 2013.","author":"Blanco Roi","year":"2013","unstructured":"Roi Blanco, Berkant Barla Cambazoglu, Peter Mika, and Nicolas Torzec. 2013. Entity recommendations in web search. In Proceedings of the Semantic Web \u2013 ISWC 2013.33\u201348."},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","first-page":"5016","DOI":"10.18653\/v1\/D18-1547","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.","author":"Budzianowski Pawe\\l","year":"2018","unstructured":"Pawe\\l Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, I\u00f1igo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Ga\u0161i\u0107. 2018. MultiWOZ - a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.5016\u20135026."},{"key":"e_1_3_2_8_2","first-page":"249","volume-title":"Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics","author":"Callison-Burch Chris","year":"2006","unstructured":"Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the role of bleu in machine translation research. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. 249\u2013256."},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/978-3-540-78646-7_5","volume-title":"Proceedings of the IR Research, 30th European Conference on Advances in Information Retrieval.","author":"Carterette Ben","year":"2008","unstructured":"Ben Carterette, Paul N. Bennett, David Maxwell Chickering, and Susan T. Dumais. 2008. Here or there: Preference judgments for relevance. In Proceedings of the IR Research, 30th European Conference on Advances in Information Retrieval.16\u201327."},{"issue":"1","key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1007\/s11257-011-9108-6","article-title":"Critiquing-based recommenders: Survey and emerging trends","volume":"22","author":"Chen Li","year":"2012","unstructured":"Li Chen and Pearl Pu. 2012. Critiquing-based recommenders: Survey and emerging trends. User Modeling and User-Adapted Interaction 22, 1-2 (2012), 125\u2013150.","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"e_1_3_2_11_2","first-page":"1724","volume-title":"Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.","author":"Cho Kyunghyun","year":"2014","unstructured":"Kyunghyun Cho, Bart van Merri\u00ebnboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder\u2013 decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 1724\u20131734."},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1145\/3219819.3219894","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Christakopoulou Konstantina","year":"2018","unstructured":"Konstantina Christakopoulou, Alex Beutel, Rui Li, Sagar Jain, and Ed H. Chi. 2018. Q&R: A two\u2013 stage approach toward interactive recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 139\u2013148."},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1145\/2939672.2939746","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Christakopoulou Konstantina","year":"2016","unstructured":"Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards conversational recommender systems. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.815\u2013824."},{"key":"e_1_3_2_14_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171\u20134186."},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.aiopen.2021.06.002","article-title":"Advances and challenges in conversational recommender systems: A survey","volume":"2","author":"Gao Chongming","year":"2021","unstructured":"Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. AI Open 2 (2021), 100\u2013126.","journal-title":"AI Open"},{"key":"e_1_3_2_16_2","first-page":"1371","volume-title":"Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Gao Jianfeng","year":"2018","unstructured":"Jianfeng Gao, Michel Galley, and Lihong Li. 2018. Neural approaches to conversational AI. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval.1371\u20131374."},{"key":"e_1_3_2_17_2","first-page":"3405","volume-title":"Proceedings of the 29th ACM International Conference on Information and Knowledge Management.","author":"Habib Javeria","year":"2020","unstructured":"Javeria Habib, Shuo Zhang, and Krisztian Balog. 2020. IAI MovieBot: A conversational movie recommender system. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management.3405\u20133408."},{"key":"e_1_3_2_18_2","first-page":"355","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR \u201917)","author":"He Xiangnan","year":"2017","unstructured":"Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR \u201917). 355\u2013364."},{"issue":"8","key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter S.","year":"1997","unstructured":"S. Hochreiter and J. Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (1997), 1735\u20131780.","journal-title":"Neural Computation"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1145\/3383313.3412252","volume-title":"Proceedings of the 14th ACM Conference on Recommender Systems.","author":"Huang Jin","year":"2020","unstructured":"Jin Huang, Harrie Oosterhuis, Maarten de Rijke, and Herke van Hoof. 2020. Keeping dataset biases out of the simulation: A debiased simulator for reinforcement learning based recommender systems. In Proceedings of the 14th ACM Conference on Recommender Systems.190\u2013199."},{"key":"e_1_3_2_21_2","first-page":"447","volume-title":"Proceedings of the 12th ACM International Conference on Web Search and Data Mining.","author":"Jagerman Rolf","year":"2019","unstructured":"Rolf Jagerman, Ilya Markov, and Maarten de Rijke. 2019. When people change their mind: Off\u2013 policy evaluation in non\u2013 stationary recommendation environments. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining.447\u2013455."},{"issue":"5","key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3453154","article-title":"A survey on conversational recommender systems","volume":"54","author":"Jannach Dietmar","year":"2022","unstructured":"Dietmar Jannach, Ahtsham Manzoor, Wanling Cai, and Li Chen. 2022. A survey on conversational recommender systems. Computing Surveys 54, 5 (2022), 1\u201336.","journal-title":"Computing Surveys"},{"key":"e_1_3_2_23_2","volume-title":"Speech and Language Processing","author":"Jurafsky Dan","year":"2020","unstructured":"Dan Jurafsky and James H. Martin. 2020. Speech and Language Processing. (3rd ed.). Draft. https:\/\/web.stanford.edu\/jurafsky\/slp3\/"},{"key":"e_1_3_2_24_2","first-page":"388","volume-title":"Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.","author":"Koehn Philipp","year":"2004","unstructured":"Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.388\u2013395."},{"key":"e_1_3_2_25_2","first-page":"724","volume-title":"Proceedings of the 15th ACM Conference on Recommender Systems.","author":"Kostric Ivica","year":"2021","unstructured":"Ivica Kostric, Krisztian Balog, and Filip Radlinski. 2021. Soliciting user preferences in conversational recommender systems via usage-related questions. In Proceedings of the 15th ACM Conference on Recommender Systems.724\u2013729."},{"key":"e_1_3_2_26_2","first-page":"228","volume-title":"Proceedings of the 2nd Workshop on Statistical Machine Translation.","author":"Lavie Alon","year":"2007","unstructured":"Alon Lavie and Abhaya Agarwal. 2007. Meteor: An automatic metric for MT evaluation with high levels of correlation with human judgments. In Proceedings of the 2nd Workshop on Statistical Machine Translation.228\u2013231."},{"key":"e_1_3_2_27_2","first-page":"1073","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Lee Hoyeop","year":"2019","unstructured":"Hoyeop Lee, Jinbae Im, Seongwon Jang, Hyunsouk Cho, and Sehee Chung. 2019. MeLU: Meta\u2013 learned user preference estimator for cold\u2013 start recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.1073\u20131082."},{"key":"e_1_3_2_28_2","first-page":"304","volume-title":"Proceedings of the 13th International Conference on Web Search and Data Mining.","author":"Lei Wenqiang","year":"2020","unstructured":"Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining.304\u2013312."},{"key":"e_1_3_2_29_2","first-page":"2073","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Lei Wenqiang","year":"2020","unstructured":"Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, and Tat\u2013Seng Chua. 2020. Interactive path reasoning on graph for conversational recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2073\u20132083."},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","first-page":"7871","DOI":"10.18653\/v1\/2020.acl-main.703","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.","author":"Lewis Mike","year":"2020","unstructured":"Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.7871\u20137880."},{"key":"e_1_3_2_31_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Li Raymond","year":"2018","unstructured":"Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In Proceedings of the Advances in Neural Information Processing Systems."},{"key":"e_1_3_2_32_2","unstructured":"Yuan-Hong Liao Amlan Kar and Sanja Fidler. 2021. Towards Good Practices for Efficiently Annotating Large-Scale Image Classification Datasets. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 4350\u20134359."},{"key":"e_1_3_2_33_2","first-page":"74","volume-title":"Proceedings of the Text Summarization Branches Out.","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Proceedings of the Text Summarization Branches Out.74\u201381."},{"key":"e_1_3_2_34_2","unstructured":"Sheng-Chieh Lin Jheng-Hong Yang Rodrigo Nogueira Ming-Feng Tsai Chuan-Ju Wang and Jimmy Lin. 2020. Conversational Question Reformulation via Sequence-to-Sequence Architectures and Pretrained Language Models. arXiv:2004.01909. Retrieved from https:\/\/arxiv.org\/abs\/2004.01909"},{"key":"e_1_3_2_35_2","first-page":"2122","volume-title":"Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.","author":"Liu Chia-Wei","year":"2016","unstructured":"Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2122\u20132132."},{"key":"e_1_3_2_36_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_2_37_2","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Loshchilov Ilya","year":"2022","unstructured":"Ilya Loshchilov and Frank Hutter. 2022. Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1145\/1963405.1963456","volume-title":"Proceedings of the 20th International Conference on World Wide Web.","author":"Lu Yue","year":"2011","unstructured":"Yue Lu, Malu Castellanos, Umeshwar Dayal, and ChengXiang Zhai. 2011. Automatic construction of a context\u2013 aware sentiment lexicon: An optimization approach. In Proceedings of the 20th International Conference on World Wide Web.347\u2013356."},{"key":"e_1_3_2_39_2","first-page":"2535","volume-title":"Proceedings of the Web Conference 2020.","author":"Luo Kai","year":"2020","unstructured":"Kai Luo, Scott Sanner, Ga Wu, Hanze Li, and Hojin Yang. 2020. Latent linear critiquing for conversational recommender systems. In Proceedings of the Web Conference 2020.2535\u20132541."},{"key":"e_1_3_2_40_2","first-page":"1269","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Luo Kai","year":"2020","unstructured":"Kai Luo, Hojin Yang, Ga Wu, and Scott Sanner. 2020. Deep critiquing for VAE\u2013 based recommender systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1269\u20131278."},{"key":"e_1_3_2_41_2","first-page":"1552","volume-title":"Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.","author":"Mairesse Fran\u00e7ois","year":"2010","unstructured":"Fran\u00e7ois Mairesse, Milica Ga\u0161i\u0107, Filip Jur\u010d\u00ed\u010dek, Simon Keizer, Blaise Thomson, Kai Yu, and Steve Young. 2010. Phrase-based statistical language generation using graphical models and active learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.1552\u20131561."},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"55","DOI":"10.3115\/v1\/P14-5010","volume-title":"Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations.","author":"Manning Christopher","year":"2014","unstructured":"Christopher Manning, Mihai Surdeanu, John Bauer, Jenny Finkel, Steven Bethard, and David McClosky. 2014. The stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 55\u201360."},{"key":"e_1_3_2_43_2","first-page":"515","volume-title":"Proceedings of the 15th ACM Conference on Recommender Systems.","author":"Manzoor Ahtsham","year":"2021","unstructured":"Ahtsham Manzoor and Dietmar Jannach. 2021. Generation-based vs. retrieval-based conversational recommendation: A user-centric comparison. In Proceedings of the 15th ACM Conference on Recommender Systems.515\u2013520."},{"key":"e_1_3_2_44_2","article-title":"Towards retrieval-based conversational recommendation","volume":"109","author":"Manzoor Ahtsham","year":"2022","unstructured":"Ahtsham Manzoor and Dietmar Jannach. 2022. Towards retrieval-based conversational recommendation. Information Systems 109, C (2022), 102083.","journal-title":"Information Systems"},{"key":"e_1_3_2_45_2","first-page":"630","volume-title":"Proceedings of the 2nd International Conference on Neural Information Processing Systems.","author":"Morgan N.","year":"1989","unstructured":"N. Morgan and H. Bourlard. 1989. Generalization and parameter estimation in feedforward nets: Some experiments. In Proceedings of the 2nd International Conference on Neural Information Processing Systems.630\u2013637."},{"issue":"3","key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"370","DOI":"10.2307\/2344614","article-title":"Generalized linear models","volume":"135","author":"Nelder J. A.","year":"1972","unstructured":"J. A. Nelder and R. W. M. Wedderburn. 1972. Generalized linear models. Journal of the Royal Statistical Society: Series A (General) 135, 3 (1972), 370\u2013384.","journal-title":"Journal of the Royal Statistical Society: Series A (General)"},{"key":"e_1_3_2_47_2","first-page":"188","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.","author":"Ni Jianmo","year":"2019","unstructured":"Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing.188\u2013197."},{"key":"e_1_3_2_48_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting on Association for Computational Linguistics.","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. 311\u2013318."},{"key":"e_1_3_2_49_2","first-page":"2227","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).","author":"Peters Matthew E.","year":"2018","unstructured":"Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).2227\u20132237."},{"key":"e_1_3_2_50_2","unstructured":"Ronak Pradeep Rodrigo Nogueira and Jimmy Lin. 2021. The Expando-Mono-Duo Design Pattern for Text Ranking with Pretrained Sequence-to-Sequence Models. arXiv:2101.05667. Retrieved from https:\/\/arxiv.org\/abs\/2101.05667"},{"key":"e_1_3_2_51_2","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2019. Language models are unsupervised multitask learners."},{"issue":"140","key":"e_1_3_2_52_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21, 140 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_53_2","first-page":"2737","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).","author":"Rao Sudha","year":"2018","unstructured":"Sudha Rao and Hal Daum\u00e9 III. 2018. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).2737\u20132746."},{"key":"e_1_3_2_54_2","first-page":"143","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).","author":"Rao Sudha","year":"2019","unstructured":"Sudha Rao and Hal Daum\u00e9 III. 2019. Answer-based adversarial training for generating clarification questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).143\u2013155."},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1145\/3366423.3380193","volume-title":"Proceedings of the Web Conference 2020.","author":"Rosset Corbin","year":"2020","unstructured":"Corbin Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, and Paul Bennett. 2020. Leading conversational search by suggesting useful questions. In Proceedings of the Web Conference 2020.1160\u20131170."},{"issue":"2","key":"e_1_3_2_56_2","first-page":"26:1\u201326:39","article-title":"A survey of evaluation metrics used for NLG systems","volume":"55","author":"Sai Ananya B.","year":"2022","unstructured":"Ananya B. Sai, Akash Kumar Mohankumar, and Mitesh M. Khapra. 2022. A survey of evaluation metrics used for NLG systems. Computing Surveys 55, 2 (2022), 26:1\u201326:39.","journal-title":"Computing Surveys"},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1145\/371920.372071","volume-title":"Proceedings of the 10th International Conference on World Wide Web.","author":"Sarwar Badrul","year":"2001","unstructured":"Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th International Conference on World Wide Web.285\u2013295."},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1145\/3471158.3472257","volume-title":"Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval.","author":"Sekuli\u0107 Ivan","year":"2021","unstructured":"Ivan Sekuli\u0107, Mohammad Aliannejadi, and Fabio Crestani. 2021. Towards facet-driven generation of clarifying questions for conversational search. In Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval.167\u2013175."},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1145\/3240323.3240352","volume-title":"Proceedings of the 12th ACM Conference on Recommender Systems.","author":"Sepliarskaia Anna","year":"2018","unstructured":"Anna Sepliarskaia, Julia Kiseleva, Filip Radlinski, and Maarten de Rijke. 2018. Preference elicitation as an optimization problem. In Proceedings of the 12th ACM Conference on Recommender Systems.172\u2013180."},{"key":"e_1_3_2_60_2","first-page":"235","volume-title":"Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Sun Yueming","year":"2018","unstructured":"Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval.235\u2013244."},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-02262-3","volume-title":"Faceted Search","author":"Tunkelang Daniel","year":"2009","unstructured":"Daniel Tunkelang. 2009. Faceted Search. Vol. 5, Morgan & Claypool Publishers."},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","first-page":"101151","DOI":"10.1016\/j.csl.2020.101151","article-title":"Human evaluation of automatically generated text: Current trends and best practice guidelines","volume":"67","author":"Lee Chris van der","year":"2021","unstructured":"Chris van der Lee, Albert Gatt, Emiel van Miltenburg, and Emiel Krahmer. 2021. Human evaluation of automatically generated text: Current trends and best practice guidelines. Computer Speech and Language 67 (2021), 101151.","journal-title":"Computer Speech and Language"},{"issue":"5","key":"e_1_3_2_63_2","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1109\/TKDE.2017.2652461","article-title":"Dynamic facet ordering for faceted product search engines","volume":"29","author":"Vandic Damir","year":"2017","unstructured":"Damir Vandic, Steven Aanen, Flavius Frasincar, and Uzay Kaymak. 2017. Dynamic facet ordering for faceted product search engines. IEEE Transactions on Knowledge and Data Engineering 29, 5 (2017), 1004\u20131016.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_64_2","first-page":"6000","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems.","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 6000\u20136010."},{"issue":"8","key":"e_1_3_2_65_2","doi-asserted-by":"crossref","first-page":"1569","DOI":"10.1109\/TKDE.2018.2866041","article-title":"Online interactive collaborative filtering using multi-armed bandit with dependent arms","volume":"31","author":"Wang Qing","year":"2019","unstructured":"Qing Wang, Chunqiu Zeng, Wubai Zhou, Tao Li, S. S. Iyengar, Larisa Shwartz, and Genady Ya. Grabarnik. 2019. Online interactive collaborative filtering using multi-armed bandit with dependent arms. IEEE Transactions on Knowledge and Data Engineering 31, 8 (2019), 1569\u20131580.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"issue":"7","key":"e_1_3_2_66_2","first-page":"154:1\u2013154:38","article-title":"A survey on session-based recommender systems","volume":"54","author":"Wang Shoujin","year":"2021","unstructured":"Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Orgun, and Defu Lian. 2021. A survey on session-based recommender systems. Computing Surveys 54, 7 (2021), 154:1\u2013154:38.","journal-title":"Computing Surveys"},{"key":"e_1_3_2_67_2","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1145\/3437963.3441800","volume-title":"Proceedings of the 14th ACM International Conference on Web Search and Data Mining.","author":"Wang Wenjie","year":"2021","unstructured":"Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2021. Denoising implicit feedback for recommendation. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining.373\u2013381."},{"key":"e_1_3_2_68_2","doi-asserted-by":"crossref","first-page":"2193","DOI":"10.18653\/v1\/P18-1204","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).","author":"Wang Yansen","year":"2018","unstructured":"Yansen Wang, Chenyi Liu, Minlie Huang, and Liqiang Nie. 2018. Learning to ask questions in open-domain conversational systems with typed decoders. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).2193\u20132203."},{"issue":"2","key":"e_1_3_2_69_2","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1162\/neco.1989.1.2.270","article-title":"A learning algorithm for continually running fully recurrent neural networks","volume":"1","author":"Williams Ronald J.","year":"1989","unstructured":"Ronald J. Williams and David Zipser. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural Computation 1, 2 (1989), 270\u2013280.","journal-title":"Neural Computation"},{"key":"e_1_3_2_70_2","first-page":"137","volume-title":"Proceedings of the 13th ACM Conference on Recommender Systems.","author":"Wu Ga","year":"2019","unstructured":"Ga Wu, Kai Luo, Scott Sanner, and Harold Soh. 2019. Deep language\u2013 based critiquing for recommender systems. In Proceedings of the 13th ACM Conference on Recommender Systems.137\u2013145."},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1145\/3331184.3331388","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Wu Wei","year":"2019","unstructured":"Wei Wu and Rui Yan. 2019. Deep chit-chat: Deep learning for chatbots. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval.1413\u20131414."},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","first-page":"2592","DOI":"10.1145\/3366423.3380011","volume-title":"Proceedings of the Web Conference 2020.","author":"Yang Liu","year":"2020","unstructured":"Liu Yang, Minghui Qiu, Chen Qu, Cen Chen, Jiafeng Guo, Yongfeng Zhang, W. Bruce Croft, and Haiqing Chen. 2020. IART: Intent-aware response ranking with transformers in information-seeking conversation systems. In Proceedings of the Web Conference 2020.2592\u20132598."},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1145\/3219819.3219890","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Ying Rex","year":"2018","unstructured":"Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.974\u2013983."},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"1512","DOI":"10.1145\/3394486.3403202","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.","author":"Zhang Shuo","year":"2020","unstructured":"Shuo Zhang and Krisztian Balog. 2020. Evaluating conversational recommender systems via user simulation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.1512\u20131520."},{"key":"e_1_3_2_75_2","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1145\/3269206.3271776","volume-title":"Proceedings of the 27th ACM International Conference on Information and Knowledge Management.","author":"Zhang Yongfeng","year":"2018","unstructured":"Yongfeng Zhang, Xu Chen, Qingyao Ai, Liu Yang, and W. Bruce Croft. 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management.177\u2013186."},{"key":"e_1_3_2_76_2","first-page":"83","volume-title":"Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Zhang Yongfeng","year":"2014","unstructured":"Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. 2014. Explicit factor models for explainable recommendation based on phrase-level sentiment analysis. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval.83\u201392."},{"key":"e_1_3_2_77_2","first-page":"1027","volume-title":"Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval.","author":"Zhang Yongfeng","year":"2014","unstructured":"Yongfeng Zhang, Haochen Zhang, Min Zhang, Yiqun Liu, and Shaoping Ma. 2014. Do users rate or review? Boost phrase\u2013 level sentiment labeling with review\u2013 level sentiment classification. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval.1027\u20131030."},{"key":"e_1_3_2_78_2","first-page":"1411","volume-title":"Proceedings of the 22nd ACM International Conference on Information and Knowledge Management.","author":"Zhao Xiaoxue","year":"2013","unstructured":"Xiaoxue Zhao, Weinan Zhang, and Jun Wang. 2013. Interactive collaborative filtering. In Proceedings of the 22nd ACM International Conference on Information and Knowledge Management.1411\u20131420."},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","unstructured":"Jafar Afzali Aleksander Mark Drzewiecki Krisztian Balog and Shuo Zhang. 2023. UserSimCRS: A User Simulation Toolkit for Evaluating Conversational Recommender Systems. In Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining (WSDM\u201923). Association for Computing Machinery 1160\u20131163. 10.1145\/3539597.3573029","DOI":"10.1145\/3539597.3573029"}],"container-title":["ACM Transactions on Recommender Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3629981","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3629981","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:00Z","timestamp":1750291020000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3629981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,10]]},"references-count":78,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3629981"],"URL":"https:\/\/doi.org\/10.1145\/3629981","relation":{},"ISSN":["2770-6699"],"issn-type":[{"value":"2770-6699","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,10]]},"assertion":[{"value":"2022-12-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-05","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}