{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T19:53:07Z","timestamp":1770493987960,"version":"3.49.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,8,18]],"date-time":"2023-08-18T00:00:00Z","timestamp":1692316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100013105","name":"Shanghai Rising-Star Program","doi-asserted-by":"crossref","award":["23QA1403100"],"award-info":[{"award-number":["23QA1403100"]}],"id":[{"id":"10.13039\/501100013105","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Natural Science Foundation of Shanghai","award":["21ZR1421900"],"award-info":[{"award-number":["21ZR1421900"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["72192832"],"award-info":[{"award-number":["72192832"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Program for Innovative Research Team of Shanghai University of Finance and Economics, Ant Group, and China Mobile Research Institute"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>Conversational recommender system (CRS) enhances the recommender system by acquiring the latest user preference through dialogues, where an agent needs to decide \u201cwhether to ask or recommend\u201d, \u201cwhich attributes to ask\u201d, and \u201cwhich items to recommend\u201d in each round. To explore these questions, reinforcement learning is adopted in most CRS frameworks. However, existing studies somewhat ignore to consider the connection between the previous rounds and the current round of the conversation, which might lead to the lack of prior knowledge and inaccurate decisions. In this view, we propose to facilitate the connections between different rounds of conversations in a dialogue session through deep transformer-based multi-channel meta-reinforcement learning, so that the CRS agent can decide each action\/decision based on previous states, actions, and their rewards. Besides, to better utilize a user\u2019s historical preferences, we propose a more dynamic and personalized graph structure to support the conversation module and the recommendation module. Experiment results on five real-world datasets and an online evaluation with real users in an industrial environment validate the improvement of our method over the state-of-the-art approaches and the effectiveness of our designs.<\/jats:p>","DOI":"10.1145\/3604804","type":"journal-article","created":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T09:15:43Z","timestamp":1686993343000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Meta-CRS: A Dynamic Meta-Learning Approach for Effective Conversational Recommender System"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5780-4751","authenticated-orcid":false,"given":"Yuxin","family":"Ni","sequence":"first","affiliation":[{"name":"Nanyang Technological University, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-9493-7213","authenticated-orcid":false,"given":"Yunwen","family":"Xia","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9788-6634","authenticated-orcid":false,"given":"Hui","family":"Fang","sequence":"additional","affiliation":[{"name":"RIIS, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6939-5055","authenticated-orcid":false,"given":"Chong","family":"Long","sequence":"additional","affiliation":[{"name":"Ant Group, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9593-7861","authenticated-orcid":false,"given":"Xinyu","family":"Kong","sequence":"additional","affiliation":[{"name":"Ant Group, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3957-2427","authenticated-orcid":false,"given":"Daqian","family":"Li","sequence":"additional","affiliation":[{"name":"Ant Group, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0840-7526","authenticated-orcid":false,"given":"Dong","family":"Yang","sequence":"additional","affiliation":[{"name":"Ant Group, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8996-7581","authenticated-orcid":false,"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,8,18]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations","author":"Antoniou Antreas","year":"2019","unstructured":"Antreas Antoniou, Harri Edwards, and Amos Storkey. 2019. How to train your MAML. In Proceedings of the 7th International Conference on Learning Representations."},{"key":"e_1_3_2_3_2","unstructured":"Jacob Beck Risto Vuorio Evan Zheran Liu Zheng Xiong Luisa Zintgraf Chelsea Finn and Shimon Whiteson. 2023. A survey of meta-reinforcement learning. arXiv:2301.08028. Retrieved from https:\/\/arxiv.org\/abs\/2301.08028."},{"key":"e_1_3_2_4_2","unstructured":"Antoine Bordes Nicolas Usunier Alberto Garcia-Dur\u00e1n Jason Weston and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2 (NIPS\u201913) . Curran Associates Inc. Red Hook NY 2787\u20132795."},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Qibin Chen Junyang Lin Yichang Zhang Ming Ding Yukuo Cen Hongxia Yang and Jie Tang. 2019. Towards knowledge-based recommender dialog system. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919) 1803\u20131813.","DOI":"10.18653\/v1\/D19-1189"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219894"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939746"},{"key":"e_1_3_2_8_2","first-page":"222","volume-title":"Proceedings of the 16th ACM International Conference on Web Search and Data Mining","author":"Chu Zhendong","year":"2023","unstructured":"Zhendong Chu, Hongning Wang, Yun Xiao, Bo Long, and Lingfei Wu. 2023. Meta policy learning for cold-start conversational recommendation. In Proceedings of the 16th ACM International Conference on Web Search and Data Mining. 222\u2013230."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3225109"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462913"},{"key":"e_1_3_2_11_2","first-page":"1597","volume-title":"Proceedings of the 2017 IEEE 60th International Midwest Symposium on Circuits and Systems","author":"Dey Rahul","year":"2017","unstructured":"Rahul Dey and Fathi M. Salem. 2017. Gate-variants of gated recurrent unit (GRU) neural networks. In Proceedings of the 2017 IEEE 60th International Midwest Symposium on Circuits and Systems. IEEE, 1597\u20131600."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1045"},{"key":"e_1_3_2_13_2","unstructured":"Yan Duan John Schulman Xi Chen Peter L. Bartlett Ilya Sutskever and Pieter Abbeel. 2016. RL \\(^2\\) : Fast reinforcement learning via slow reinforcement learning. arXiv:1611.02779. Retrieved from https:\/\/arxiv.org\/abs\/1611.02779."},{"key":"e_1_3_2_14_2","first-page":"1126","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Finn Chelsea","year":"2017","unstructured":"Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning. PMLR, 1126\u20131135."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2021.06.002"},{"key":"e_1_3_2_16_2","unstructured":"Alex Graves Greg Wayne and Ivo Danihelka. 2014. Neural turing machines. arXiv:1410.5401. Retrieved from https:\/\/arxiv.org\/abs\/1410.5401."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294869"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"139","DOI":"10.18653\/v1\/D18-2024","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Han Xu","year":"2018","unstructured":"Xu Han, Shulin Cao, Xin Lv, Yankai Lin, Zhiyuan Liu, Maosong Sun, and Juanzi Li. 2018. Openke: An open toolkit for knowledge embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 139\u2013144."},{"issue":"9","key":"e_1_3_2_19_2","first-page":"5149","article-title":"Meta-learning in neural networks: A survey","volume":"44","author":"Hospedales Timothy","year":"2021","unstructured":"Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2021. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9 (2021), 5149\u20135169.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531844"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3453154"},{"key":"e_1_3_2_22_2","volume-title":"Proceedings of the ICML Deep Learning Workshop","author":"Koch Gregory","year":"2015","unstructured":"Gregory Koch, Richard Zemel, Ruslan Salakhutdinov, et\u00a0al. 2015. Siamese neural networks for one-shot image recognition. In Proceedings of the ICML Deep Learning Workshop. Lille."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330859"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371769"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403258"},{"key":"e_1_3_2_26_2","first-page":"9725","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Li Raymond","year":"2018","unstructured":"Raymond Li, Samira Ebrahimi Kahou, Hannes Schulz, Vincent Michalski, Laurent Charlin, and Chris Pal. 2018. Towards deep conversational recommendations. In Proceedings of the Advances in Neural Information Processing Systems. 9725\u20139735."},{"issue":"4","key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3446427","article-title":"Seamlessly unifying attributes and items: Conversational recommendation for cold-start users","volume":"39","author":"Li Shijun","year":"2021","unstructured":"Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, and Tat-Seng Chua. 2021. Seamlessly unifying attributes and items: Conversational recommendation for cold-start users. ACM Transactions on Information Systems 39, 4 (2021), 1\u201329.","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_28_2","unstructured":"Zhenguo Li Fengwei Zhou Fei Chen and Hang Li. 2017. Meta-sgd: Learning to learn quickly for few-shot learning. arXiv:1707.09835. Retrieved from https:\/\/arxiv.org\/abs\/1707.09835."},{"key":"e_1_3_2_29_2","unstructured":"Lizi Liao Ryuichi Takanobu Yunshan Ma Xun Yang Minlie Huang and Tat-Seng Chua. 2019. Deep conversational recommender in travel. arXiv:1907.00710. Retrieved from https:\/\/arxiv.org\/abs\/1907.00710."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.98"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2557069"},{"key":"e_1_3_2_32_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Mishra Nikhil","year":"2018","unstructured":"Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. 2018. A simple neural attentive meta-learner. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3406499.3415079"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.18653\/v1\/W19-5941","volume-title":"Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue","author":"Radlinski Filip","year":"2019","unstructured":"Filip Radlinski, Krisztian Balog, Bill Byrne, and Karthik Krishnamoorthi. 2019. Coached conversational preference elicitation: A case study in understanding movie preferences. In Proceedings of the 20th Annual SIGdial Meeting on Discourse and Dialogue. 353\u2013360."},{"key":"e_1_3_2_35_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Ravi Sachin","year":"2017","unstructured":"Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks","author":"Rehurek Radim","year":"2010","unstructured":"Radim Rehurek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. Citeseer."},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394592"},{"key":"e_1_3_2_38_2","volume-title":"Proceedings of the ICLR","author":"Schaul Tom","year":"2016","unstructured":"Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. 2016. Prioritized experience replay. In Proceedings of the ICLR."},{"key":"e_1_3_2_39_2","volume-title":"Evolutionary Principles in Self-referential Learning, or on Learning How to Learn: The Meta-Meta-... Hook","author":"Schmidhuber J\u00fcrgen","year":"1987","unstructured":"J\u00fcrgen Schmidhuber. 1987. Evolutionary Principles in Self-referential Learning, or on Learning How to Learn: The Meta-Meta-... Hook. Ph. D. Dissertation. Technische Universit\u00e4t M\u00fcnchen."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007383707642"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210002"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00131"},{"key":"e_1_3_2_43_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT press."},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/978-1-4615-5529-2_1","volume-title":"Proceedings of the Learning to Learn","author":"Thrun Sebastian","year":"1998","unstructured":"Sebastian Thrun and Lorien Pratt. 1998. Learning to learn: Introduction and overview. In Proceedings of the Learning to Learn. Springer, 3\u201317."},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1145\/3350546.3352500","volume-title":"Proceedings of the 2019 IEEE\/WIC\/ACM International Conference on Web Intelligence","author":"Tsumita Daisuke","year":"2019","unstructured":"Daisuke Tsumita and Tomohiro Takagi. 2019. Dialogue based recommender system that flexibly mixes utterances and recommendations. In Proceedings of the 2019 IEEE\/WIC\/ACM International Conference on Web Intelligence. IEEE, 51\u201358."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"e_1_3_2_47_2","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez ?ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917) . Curran Associates Inc. Red Hook NY 6000\u20136010."},{"key":"e_1_3_2_48_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Veli\u010dkovi\u0107 Petar","year":"2018","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li\u00f2, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_49_2","first-page":"3630","article-title":"Matching networks for one shot learning","volume":"29","year":"2016","unstructured":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Koray kavukcuoglu, and Daan Wierstra. 2016. Matching networks for one shot learning. Advances in Neural Information Processing Systems 29 (2016), 3630\u20133638.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_50_2","volume-title":"Proceedings of the CogSci","author":"Wang Jane X.","year":"2017","unstructured":"Jane X. Wang, Zeb Kurth-Nelson, Hubert Soyer, Joel Z. Leibo, Dhruva Tirumala, R\u00e9mi Munos, Charles Blundell, Dharshan Kumaran, and Matt M. Botvinick. 2017. Learning to reinforcement learn. In Proceedings of the CogSci."},{"key":"e_1_3_2_51_2","volume-title":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","author":"Wang Xiaolei","year":"2022","unstructured":"Xiaolei Wang, Kun Zhou, Ji-Rong Wen, and Wayne Xin Zhao. 2022. Towards unified conversational recommender systems via knowledge-enhanced prompt learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery & Data Mining."},{"key":"e_1_3_2_52_2","unstructured":"Yanan Wang Yong Ge Li Li Rui Chen and Tong Xu. 2020. Offline meta-level model-based reinforcement learning approach for cold-start recommendation. arXiv:2012.02476. Retrieved from https:\/\/arxiv.org\/abs\/2012.02476."},{"key":"e_1_3_2_53_2","first-page":"1995","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Wang Ziyu","year":"2016","unstructured":"Ziyu Wang, Tom Schaul, Matteo Hessel, Hado Hasselt, Marc Lanctot, and Nando Freitas. 2016. Dueling network architectures for deep reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1995\u20132003."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1369"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"5288","DOI":"10.18653\/v1\/2020.coling-main.463","volume-title":"Proceedings of the 28th International Conference on Computational Linguistics","author":"Xu Hu","year":"2020","unstructured":"Hu Xu, Seungwhan Moon, Honglei Liu, Bing Liu, Pararth Shah, and S. Yu Philip. 2020. User memory reasoning for conversational recommendation. In Proceedings of the 28th International Conference on Computational Linguistics. 5288\u20135308."},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330991"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380148"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512088"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411954"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403143"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401180"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531852"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401181"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604804","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3604804","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:04Z","timestamp":1750178764000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3604804"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,18]]},"references-count":62,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3604804"],"URL":"https:\/\/doi.org\/10.1145\/3604804","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,18]]},"assertion":[{"value":"2022-07-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-07","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}