{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:16:08Z","timestamp":1750220168485,"version":"3.41.0"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,11,16]],"date-time":"2022-11-16T00:00:00Z","timestamp":1668556800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Web"],"published-print":{"date-parts":[[2022,11,30]]},"abstract":"<jats:p>Many recent recommendation systems leverage the large quantity of reviews placed by users on items. However, it is both challenging and important to accurately measure the usefulness of such reviews for effective recommendation. In particular, users have been shown to exhibit distinct preferences over different types of reviews (e.g., preferring longer versus shorter or recent versus old reviews), indicating that users might differ in their viewpoints on what makes the reviews useful. Yet, there have been limited studies that account for the personalised usefulness of reviews when estimating the users\u2019 preferences. In this article, we propose a novel neural model, called BanditProp, which addresses this gap in the literature. It first models reviews according to both their content and associated properties (e.g., length, sentiment and recency). Thereafter, it constructs a multi-task learning (MTL) framework to model the reviews\u2019 content encoded with various properties.In such an MTL framework, each task corresponds to producing recommendations focusing on an individual property. Next, we address the selection of the features from reviews with different review properties as a bandit problem using multinomial rewards. We propose a neural contextual bandit algorithm (i.e., ConvBandit) and examine its effectiveness in comparison to eight existing bandit algorithms in addressing the bandit problem. Our extensive experiments on two well-known Amazon and Yelp datasets show that BanditProp can significantly outperform one classic and six existing state-of-the-art recommendation baselines. Moreover, BanditProp using ConvBandit\nconsistently outperforms the use of other bandit algorithms over the two used datasets. In particular, we experimentally demonstrate the effectiveness of our proposed customised multinomial rewards in comparison to binary rewards, when addressing our bandit problem.<\/jats:p>\n          <jats:p\/>","DOI":"10.1145\/3532859","type":"journal-article","created":{"date-parts":[[2022,7,21]],"date-time":"2022-07-21T12:18:55Z","timestamp":1658405935000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["BanditProp: Bandit Selection of Review Properties for Effective Recommendation"],"prefix":"10.1145","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5936-9919","authenticated-orcid":false,"given":"Xi","family":"Wang","sequence":"first","affiliation":[{"name":"University of Glasgow, Glasgow, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4701-3223","authenticated-orcid":false,"given":"Iadh","family":"Ounis","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3143-279X","authenticated-orcid":false,"given":"Craig","family":"Macdonald","sequence":"additional","affiliation":[{"name":"University of Glasgow, Glasgow, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11,16]]},"reference":[{"key":"e_1_3_2_2_2","volume-title":"Proceedings of the ICLR","author":"Agarwal Rishabh","year":"2019","unstructured":"Rishabh Agarwal, Chen Liang, Dale Schuurmans, and Mohammad Norouzi. 2019. Learning to generalize from sparse and underspecified rewards. In Proceedings of the ICLR."},{"key":"e_1_3_2_3_2","unstructured":"Sanae Amani and Christos Thrampoulidis. 2021. UCB-based algorithms for multinomial logistic regression bandits. Retrieved from https:\/\/arxiv.org\/abs\/2103.11489."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3320435.3323466"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007472513967"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098170"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/203"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3346984"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/B978-1-55860-307-3.50012-5"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186070"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/296"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/521"},{"key":"e_1_3_2_13_2","volume-title":"Proceedings of the AAAI","author":"Cicirello Vincent A.","year":"2005","unstructured":"Vincent A. Cicirello and Stephen F. Smith. 2005. The max K-armed bandit: A new model of exploration applied to search heuristic selection. In Proceedings of the AAAI."},{"key":"e_1_3_2_14_2","volume-title":"Proceedings of the NAACL-HLT","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL-HLT."},{"key":"e_1_3_2_15_2","volume-title":"Proceedings of the ANT","author":"Mignon Alexandre dos Santos","year":"2017","unstructured":"Alexandre dos Santos Mignon and Ricardo Luis de Azevedo da Rocha. 2017. An adaptive implementation of \\( \\epsilon \\) -greedy in reinforcement learning. In Proceedings of the ANT."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1177\/0047287513481274"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3309546"},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the NeurIPS","author":"Guo Dalin","year":"2018","unstructured":"Dalin Guo and Angela J. Yu. 2018. Why so gloomy? A Bayesian explanation of human pessimism bias in the multi-armed bandit task. In Proceedings of the NeurIPS."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883037"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/2806416.2806504"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/571"},{"issue":"9","key":"e_1_3_2_22_2","doi-asserted-by":"crossref","DOI":"10.1109\/MCOM.2013.6588663","article-title":"Understanding user behavior in online social networks: A survey","volume":"51","author":"Jin Long","year":"2013","unstructured":"Long Jin, Yang Chen, Tianyi Wang, Pan Hui, and Athanasios V. Vasilakos. 2013. Understanding user behavior in online social networks: A survey. IEEE Commun. Mag. 51, 9 (2013).","journal-title":"IEEE Commun. Mag."},{"key":"e_1_3_2_23_2","volume-title":"Proceedings of the AISTATS","author":"Kaufmann Emilie","year":"2012","unstructured":"Emilie Kaufmann, Olivier Capp\u00e9, and Aur\u00e9lien Garivier. 2012. On Bayesian upper confidence bounds for bandit problems. In Proceedings of the AISTATS."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959165"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1181"},{"key":"e_1_3_2_26_2","volume-title":"Proceedings of the ICLR","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the ICLR."},{"issue":"1","key":"e_1_3_2_27_2","article-title":"Algorithms for multi-armed bandit problems","author":"Kuleshov Volodymyr","year":"2000","unstructured":"Volodymyr Kuleshov and Doina Precup. 2000. Algorithms for multi-armed bandit problems. Mach. Learn. Res.1 (2000).","journal-title":"Mach. Learn. Res."},{"key":"e_1_3_2_28_2","volume-title":"Proceedings of the ICML","author":"Kveton Branislav","year":"2019","unstructured":"Branislav Kveton, Csaba Szepesv\u00e1ri, Sharan Vaswani, Zheng Wen, Tor Lattimore, and Mohammad Ghavamzadeh. 2019. Garbage in, reward out: Bootstrapping exploration in multi-armed bandits. In Proceedings of the ICML."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/397"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645728"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.07.057"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240365"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482028"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401264"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2507163"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1401"},{"key":"e_1_3_2_38_2","volume-title":"Proceedings of the ACML","author":"Ning Xia","year":"2010","unstructured":"Xia Ning and George Karypis. 2010. Multi-task learning for recommender system. In Proceedings of the ACML."},{"key":"e_1_3_2_39_2","volume-title":"Proceedings of the NeurIPS","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et\u00a0al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the NeurIPS."},{"key":"e_1_3_2_40_2","volume-title":"Proceedings of the UAI","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the UAI."},{"key":"e_1_3_2_41_2","volume-title":"Proceedings of the ALT","author":"Riou Charles","year":"2020","unstructured":"Charles Riou and Junya Honda. 2020. Bandit algorithms based on thompson sampling for bounded reward distributions. In Proceedings of the ALT."},{"key":"e_1_3_2_42_2","volume-title":"Proceedings of the ICLR","author":"Riquelme Carlos","year":"2018","unstructured":"Carlos Riquelme, George Tucker, and Jasper Snoek. 2018. Deep Bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. In Proceedings of the ICLR."},{"key":"e_1_3_2_43_2","unstructured":"Sebastian Ruder. 2017. An overview of multi-task learning in deep neural networks. Retrieved from https:\/\/arxiv.org\/abs\/1706.05098."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1561\/2200000070"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401281"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403121"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347040"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109890"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.413"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21383"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401123"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1287\/isre.14.1.47.14767"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159656"},{"key":"e_1_3_2_54_2","volume-title":"Proceedings of the NeurIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the NeurIPS."},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3471158.3472233"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411978"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3450038"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.6125"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240355"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914683"},{"key":"e_1_3_2_61_2","unstructured":"Tom Zahavy and Shie Mannor. 2019. Deep neural linear bandits: Overcoming catastrophic forgetting through likelihood matching. Retrieved from https:\/\/arxiv.org\/abs\/1901.08612."},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132892"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-010-0001-0"},{"key":"e_1_3_2_64_2","unstructured":"Xiangyu Zhao Liang Zhang Long Xia Zhuoye Ding Dawei Yin and Jiliang Tang. 2018. Deep reinforcement learning for list-wise recommendations. Retrieved from https:\/\/arxiv.org\/abs\/1801.00209."},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018665"},{"key":"e_1_3_2_66_2","volume-title":"Proceedings of the ICML","author":"Zhou Dongruo","year":"2020","unstructured":"Dongruo Zhou, Lihong Li, and Quanquan Gu. 2020. Neural contextual bandits with UCB-based exploration. In Proceedings of the ICML."}],"container-title":["ACM Transactions on the Web"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3532859","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3532859","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:37Z","timestamp":1750186837000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3532859"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,16]]},"references-count":65,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,11,30]]}},"alternative-id":["10.1145\/3532859"],"URL":"https:\/\/doi.org\/10.1145\/3532859","relation":{},"ISSN":["1559-1131","1559-114X"],"issn-type":[{"type":"print","value":"1559-1131"},{"type":"electronic","value":"1559-114X"}],"subject":[],"published":{"date-parts":[[2022,11,16]]},"assertion":[{"value":"2021-11-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-07-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}