{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T11:23:44Z","timestamp":1783682624928,"version":"3.55.0"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,2,16]],"date-time":"2023-02-16T00:00:00Z","timestamp":1676505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Hong Kong RGC\/GRF","award":["RGC\/HKBU12201620"],"award-info":[{"award-number":["RGC\/HKBU12201620"]}]},{"name":"NSF","award":["IIS-1910154, 2007907, and 2046457"],"award-info":[{"award-number":["IIS-1910154, 2007907, and 2046457"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"<jats:p>Explaining to users why some items are recommended is critical, as it can help users to make better decisions, increase their satisfaction, and gain their trust in recommender systems (RS). However, existing explainable RS usually consider explanation as a side output of the recommendation model, which has two problems: (1) It is difficult to evaluate the produced explanations, because they are usually model-dependent, and (2) as a result, how the explanations impact the recommendation performance is less investigated.<\/jats:p>\n          <jats:p>In this article, explaining recommendations is formulated as a ranking task and learned from data, similarly to item ranking for recommendation. This makes it possible for standard evaluation of explanations via ranking metrics (e.g., Normalized Discounted Cumulative Gain). Furthermore, this article extends traditional item ranking to an item\u2013explanation joint-ranking formalization to study if purposely selecting explanations could reach certain learning goals, e.g., improving recommendation performance. A great challenge, however, is that the sparsity issue in the user-item-explanation data would be inevitably severer than that in traditional user\u2013item interaction data, since not every user\u2013item pair can be associated with all explanations. To mitigate this issue, this article proposes to perform two sets of matrix factorization by considering the ternary relationship as two groups of binary relationships. Experiments on three large datasets verify the solution\u2019s effectiveness on both explanation ranking and item recommendation.<\/jats:p>","DOI":"10.1145\/3569423","type":"journal-article","created":{"date-parts":[[2022,10,25]],"date-time":"2022-10-25T13:23:46Z","timestamp":1666704226000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["On the Relationship between Explanation and Recommendation: Learning to Rank Explanations for Improved Performance"],"prefix":"10.1145","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5631-2519","authenticated-orcid":false,"given":"Lei","family":"Li","sequence":"first","affiliation":[{"name":"Hong Kong Baptist University, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2633-8555","authenticated-orcid":false,"given":"Yongfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Rutgers University, New Brunswick, New Jersey, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5842-838X","authenticated-orcid":false,"given":"Li","family":"Chen","sequence":"additional","affiliation":[{"name":"Hong Kong Baptist University, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,2,16]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1145\/3397271.3401032","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Balog Krisztian","year":"2020","unstructured":"Krisztian Balog and Filip Radlinski. 2020. Measuring recommendation explanation quality: The conflicting goals of explanations. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 329\u2013338."},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1145\/2736277.2741077","volume-title":"Proceedings of the 24th International Conference on World Wide Web","author":"Bhargava Preeti","year":"2015","unstructured":"Preeti Bhargava, Thomas Phan, Jiayu Zhou, and Juhan Lee. 2015. Who, what, when, and where: Multi-dimensional collaborative recommendations using tensor factorization on sparse user-generated data. In Proceedings of the 24th International Conference on World Wide Web. 130\u2013140."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02310791"},{"key":"e_1_3_2_5_2","first-page":"288","volume-title":"Proceedings of the 11th ACM Conference on Recommender Systems","author":"Catherine Rose","year":"2017","unstructured":"Rose Catherine and William Cohen. 2017. Transnets: Learning to transform for recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems. 288\u2013296."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186070"},{"key":"e_1_3_2_7_2","volume-title":"Proceedings of SIGIR\u201919 Workshop on ExplainAble Recommendation and Search","author":"Chen Hanxiong","year":"2019","unstructured":"Hanxiong Chen, Xu Chen, Shaoyun Shi, and Yongfeng Zhang. 2019. Generate natural language explanations for recommendation. In Proceedings of SIGIR\u201919 Workshop on ExplainAble Recommendation and Search. ACM."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441762"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313469"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331254"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911549"},{"key":"e_1_3_2_12_2","first-page":"53","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"33","author":"Chen Xu","year":"2019","unstructured":"Xu Chen, Yongfeng Zhang, and Zheng Qin. 2019. Dynamic explainable recommendation based on neural attentive models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 53\u201360."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/296"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1137\/S0895479896305696"},{"key":"e_1_3_2_15_2","volume-title":"Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013622"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3449848"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/2806416.2806504"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/358916.358995"},{"key":"e_1_3_2_20_2","first-page":"4018","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201917)","author":"Huang Jizhou","year":"2017","unstructured":"Jizhou Huang, Wei Zhang, Shiqi Zhao, Shiqiang Ding, and Haifeng Wang. 2017. Learning to explain entity relationships by pairwise ranking with convolutional neural networks. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI\u201917). 4018\u20134025."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11280-021-00912-4"},{"issue":"3","key":"e_1_3_2_22_2","first-page":"909","article-title":"Coupled graphs and tensor factorization for recommender systems and community detection","volume":"33","author":"Ioannidis Vassilis N.","year":"2019","unstructured":"Vassilis N. Ioannidis, Ahmed S. Zamzam, Georgios B. Giannakis, and Nicholas D. Sidiropoulos. 2019. Coupled graphs and tensor factorization for recommender systems and community detection. IEEE Trans. Knowl. Data Eng. 33, 3 (2019), 909\u2013920.","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"e_1_3_2_23_2","first-page":"506","volume-title":"Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery","author":"J\u00e4schke Robert","year":"2007","unstructured":"Robert J\u00e4schke, Leandro Marinho, Andreas Hotho, Lars Schmidt-Thieme, and Gerd Stumme. 2007. Tag recommendations in folksonomies. In Proceedings of the European Conference on Principles of Data Mining and Knowledge Discovery. Springer, 506\u2013514."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_3_2_25_2","first-page":"2427","volume-title":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IICAI\u201920)","author":"Le Trung-Hoang","year":"2020","unstructured":"Trung-Hoang Le, Hady W. Lauw, and C. Bessiere. 2020. Synthesizing aspect-driven recommendation explanations from reviews. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IICAI\u201920). 2427\u20132434."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10844-020-00631-8"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411992"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463248"},{"key":"e_1_3_2_29_2","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics","author":"Li Lei","year":"2021","unstructured":"Lei Li, Yongfeng Zhang, and Li Chen. 2021. Personalized transformer for explainable recommendation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080822"},{"key":"e_1_3_2_31_2","first-page":"74","volume-title":"Text Summarization Branches Out","author":"Lin Chin-Yew","year":"2004","unstructured":"Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text Summarization Branches Out. 74\u201381."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14267-3"},{"key":"e_1_3_2_33_2","first-page":"773","volume-title":"Proceedings of the World Wide Web Conference","author":"Lu Yichao","year":"2018","unstructured":"Yichao Lu, Ruihai Dong, and Barry Smyth. 2018. Coevolutionary recommendation model: Mutual learning between ratings and reviews. In Proceedings of the World Wide Web Conference. 773\u2013782."},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1145\/3240323.3240354","volume-title":"Proceedings of the 12th ACM Conference on Recommender Systems","author":"McInerney James","year":"2018","unstructured":"James McInerney, Benjamin Lacker, Samantha Hansen, Karl Higley, Hugues Bouchard, Alois Gruson, and Rishabh Mehrotra. 2018. Explore, exploit, and explain: Personalizing explainable recommendations with bandits. In Proceedings of the 12th ACM Conference on Recommender Systems. 31\u201339."},{"key":"e_1_3_2_35_2","first-page":"1257","volume-title":"Advances in Neural Information Processing Systems","author":"Mnih Andriy","year":"2007","unstructured":"Andriy Mnih and Russ R. Salakhutdinov. 2007. Probabilistic matrix factorization. In Advances in Neural Information Processing Systems. 1257\u20131264."},{"key":"e_1_3_2_36_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311\u2013318."},{"key":"e_1_3_2_37_2","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/1718487.1718498"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/192844.192905"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372071"},{"key":"e_1_3_2_41_2","first-page":"297","volume-title":"Proceedings of the 11th ACM Conference on Recommender Systems","author":"Seo Sungyong","year":"2017","unstructured":"Sungyong Seo, Jing Huang, Hao Yang, and Yan Liu. 2017. Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In Proceedings of the 11th ACM Conference on Recommender Systems. 297\u2013305."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220088"},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1007\/978-1-4899-7637-6_10","volume-title":"Recommender Systems Handbook (2nd ed.)","author":"Tintarev Nava","year":"2015","unstructured":"Nava Tintarev and Judith Masthoff. 2015. Explaining recommendations: Design and evaluation. In Recommender Systems Handbook (2nd ed.), Bracha Shapira (Ed.). Springer, Chapter 10, 353\u2013382."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02289464"},{"key":"e_1_3_2_45_2","first-page":"564","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Voskarides Nikos","year":"2015","unstructured":"Nikos Voskarides, Edgar Meij, Manos Tsagkias, Maarten De Rijke, and Wouter Weerkamp. 2015. Learning to explain entity relationships in knowledge graphs. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 564\u2013574."},{"key":"e_1_3_2_46_2","first-page":"165","volume-title":"Proceedings of the Special Interest Group on Information Retrieval (SIGIR\u201918)","author":"Wang Nan","year":"2018","unstructured":"Nan Wang, Hongning Wang, Yiling Jia, and Yue Yin. 2018. Explainable recommendation via multi-task learning in opinionated text data. In Proceedings of the Special Interest Group on Information Retrieval (SIGIR\u201918). ACM, 165\u2013174."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512168"},{"key":"e_1_3_2_48_2","first-page":"587","volume-title":"Proceedings of the IEEE International Conference on Data Mining (ICDM)","author":"Wang Xiting","year":"2018","unstructured":"Xiting Wang, Yiru Chen, Jie Yang, Le Wu, Zhengtao Wu, and Xing Xie. 2018. A reinforcement learning framework for explainable recommendation. In Proceedings of the IEEE International Conference on Data Mining (ICDM). IEEE, 587\u2013596."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331203"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380071"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512031"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441726"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000066"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2609579"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569423","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3569423","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:56Z","timestamp":1750182536000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3569423"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,16]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3569423"],"URL":"https:\/\/doi.org\/10.1145\/3569423","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,16]]},"assertion":[{"value":"2022-03-08","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-10-04","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}