{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,30]],"date-time":"2026-09-30T21:51:07Z","timestamp":1790805067892,"version":"4.1.0"},"publisher-location":"New York, NY, USA","reference-count":29,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,9,22]],"date-time":"2020-09-22T00:00:00Z","timestamp":1600732800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Community's Horizon 2020 research and innovation programme","award":["779747"],"award-info":[{"award-number":["779747"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,9,22]]},"DOI":"10.1145\/3383313.3418479","type":"proceedings-article","created":{"date-parts":[[2020,9,18]],"date-time":"2020-09-18T22:28:22Z","timestamp":1600468102000},"page":"681-686","source":"Crossref","is-referenced-by-count":100,"title":["Exploring Data Splitting Strategies for the Evaluation of Recommendation Models"],"prefix":"10.1145","author":[{"given":"Zaiqiao","family":"Meng","sequence":"first","affiliation":[{"name":"University of Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"McCreadie","sequence":"additional","affiliation":[{"name":"School of Computing Science University of Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Craig","family":"Macdonald","sequence":"additional","affiliation":[{"name":"School of Computing cience University of Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Iadh","family":"Ounis","sequence":"additional","affiliation":[{"name":"University of Glasgow, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,9,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"crossref","unstructured":"Ting Bai Lixin Zou Wayne\u00a0Xin Zhao Pan Du Weidong Liu Jian-Yun Nie and Ji-Rong Wen. 2019. CTRec: A Long-Short Demands Evolution Model for Continuous-Time Recommendation. In SIGIR. 675\u2013684. Ting Bai Lixin Zou Wayne\u00a0Xin Zhao Pan Du Weidong Liu Jian-Yun Nie and Ji-Rong Wen. 2019. CTRec: A Long-Short Demands Evolution Model for Continuous-Time Recommendation. In SIGIR. 675\u2013684.","DOI":"10.1145\/3331184.3331199"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Pedro\u00a0G Campos Fernando D\u00edez and Manuel S\u00e1nchez-Monta\u00f1\u00e9s. 2011. Towards a more realistic evaluation: testing the ability to predict future tastes of matrix factorization-based recommenders. In RecSys. 309\u2013312. Pedro\u00a0G Campos Fernando D\u00edez and Manuel S\u00e1nchez-Monta\u00f1\u00e9s. 2011. Towards a more realistic evaluation: testing the ability to predict future tastes of matrix factorization-based recommenders. In RecSys. 309\u2013312.","DOI":"10.1145\/2043932.2043990"},{"key":"e_1_3_2_1_3_1","unstructured":"Roc\u00edo Ca\u00f1amares Pablo Castells and Alistair Moffat. 2020. Offline evaluation options for recommender systems. Information Retrieval Journal(2020) 1\u201324. Roc\u00edo Ca\u00f1amares Pablo Castells and Alistair Moffat. 2020. Offline evaluation options for recommender systems. Information Retrieval Journal(2020) 1\u201324."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Maurizio\u00a0Ferrari Dacrema Paolo Cremonesi and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In RecSys. 101\u2013109. Maurizio\u00a0Ferrari Dacrema Paolo Cremonesi and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In RecSys. 101\u2013109.","DOI":"10.1145\/3298689.3347058"},{"key":"e_1_3_2_1_5_1","unstructured":"Xiangnan He Lizi Liao Hanwang Zhang Liqiang Nie Xia Hu and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173\u2013182. Xiangnan He Lizi Liao Hanwang Zhang Liqiang Nie Xia Hu and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW. 173\u2013182."},{"key":"e_1_3_2_1_6_1","unstructured":"Haoji Hu and Xiangnan He. 2019. Sets2Sets: Learning from Sequential Sets with Neural Networks. In SIGKDD. 1491\u20131499. Haoji Hu and Xiangnan He. 2019. Sets2Sets: Learning from Sequential Sets with Neural Networks. In SIGKDD. 1491\u20131499."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. 197\u2013206. Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In ICDM. 197\u2013206.","DOI":"10.1109\/ICDM.2018.00035"},{"key":"e_1_3_2_1_8_1","unstructured":"Daniel\u00a0D Lee and H\u00a0Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In NeurIPS. 556\u2013562. Daniel\u00a0D Lee and H\u00a0Sebastian Seung. 2001. Algorithms for non-negative matrix factorization. In NeurIPS. 556\u2013562."},{"key":"e_1_3_2_1_9_1","unstructured":"Jiacheng Li Yujie Wang and Julian McAuley. 2020. Time Interval Aware Self-Attention for Sequential Recommendation. In WSDM. 322\u2013330. Jiacheng Li Yujie Wang and Julian McAuley. 2020. Time Interval Aware Self-Attention for Sequential Recommendation. In WSDM. 322\u2013330."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Dawen Liang Rahul\u00a0G Krishnan Matthew\u00a0D Hoffman and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In WWW. 689\u2013698. Dawen Liang Rahul\u00a0G Krishnan Matthew\u00a0D Hoffman and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In WWW. 689\u2013698.","DOI":"10.1145\/3178876.3186150"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Jarana Manotumruksa Craig Macdonald and Iadh Ounis. 2018. A Contextual Attention Recurrent Architecture for Context-aware Venue recommendation. In SIGIR. 555\u2013564. Jarana Manotumruksa Craig Macdonald and Iadh Ounis. 2018. A Contextual Attention Recurrent Architecture for Context-aware Venue recommendation. In SIGIR. 555\u2013564.","DOI":"10.1145\/3209978.3210042"},{"key":"e_1_3_2_1_12_1","unstructured":"Zaiqiao Meng Richard McCreadie Craig Macdonald and Iadh Ounis. 2019. Variational Bayesian Context-aware Representation for Grocery Recommendation. In CARS2.0@RecSys. Zaiqiao Meng Richard McCreadie Craig Macdonald and Iadh Ounis. 2019. Variational Bayesian Context-aware Representation for Grocery Recommendation. In CARS2.0@RecSys."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Zaiqiao Meng Richard McCreadie Craig Macdonald Iadh Ounis Shangsong Liang Siwei Liu Guangtao Zeng Liang Junha Yucheng Liang Qiang Zhang Xi Wang and Wu Yaxiong. 2020. BETA-Rec: Build Evaluate and Tune Automated Recommender Systems. (2020). Zaiqiao Meng Richard McCreadie Craig Macdonald Iadh Ounis Shangsong Liang Siwei Liu Guangtao Zeng Liang Junha Yucheng Liang Qiang Zhang Xi Wang and Wu Yaxiong. 2020. BETA-Rec: Build Evaluate and Tune Automated Recommender Systems. (2020).","DOI":"10.1145\/3383313.3411524"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190616"},{"key":"e_1_3_2_1_15_1","volume-title":"BPR: Bayesian personalized ranking from implicit feedback. In UAI. 452\u2013461.","author":"Rendle Steffen","year":"2009"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Steffen Rendle Christoph Freudenthaler and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. 811\u2013820. Steffen Rendle Christoph Freudenthaler and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In WWW. 811\u2013820.","DOI":"10.1145\/1772690.1772773"},{"key":"e_1_3_2_1_17_1","volume-title":"Neural Collaborative Filtering vs. Matrix Factorization Revisited. arXiv:2005.09683","author":"Rendle Steffen","year":"2020"},{"key":"e_1_3_2_1_18_1","unstructured":"Steffen Rendle Li Zhang and Yehuda Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. arXiv preprint arXiv:1905.01395(2019). Steffen Rendle Li Zhang and Yehuda Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. arXiv preprint arXiv:1905.01395(2019)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"crossref","unstructured":"Noveen Sachdeva Giuseppe Manco Ettore Ritacco and Vikram Pudi. 2019. Sequential Variational Autoencoders for Collaborative Filtering. In WSDM. 600\u2013608. Noveen Sachdeva Giuseppe Manco Ettore Ritacco and Vikram Pudi. 2019. Sequential Variational Autoencoders for Collaborative Filtering. In WSDM. 600\u2013608.","DOI":"10.1145\/3289600.3291007"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Alan Said and Alejandro Bellog\u00edn. 2014. Comparative Recommender System Evaluation: Benchmarking Recommendation Frameworks. In RecSys. 129\u2013136. Alan Said and Alejandro Bellog\u00edn. 2014. Comparative Recommender System Evaluation: Benchmarking Recommendation Frameworks. In RecSys. 129\u2013136.","DOI":"10.1145\/2645710.2645746"},{"key":"e_1_3_2_1_21_1","volume-title":"Recommender systems handbook","author":"Shani Guy"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Fei Sun Jun Liu Jian Wu Changhua Pei Xiao Lin Wenwu Ou and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM. 1441\u20131450. Fei Sun Jun Liu Jian Wu Changhua Pei Xiao Lin Wenwu Ou and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In CIKM. 1441\u20131450.","DOI":"10.1145\/3357384.3357895"},{"key":"e_1_3_2_1_23_1","unstructured":"Qiaoyu Tan Ninghao Liu Xing Zhao Hongxia Yang Jingren Zhou and Xia Hu. 2020. Learning to Hash with Graph Neural Networks for Recommender Systems. In WWW. 1988\u20131998. Qiaoyu Tan Ninghao Liu Xing Zhao Hongxia Yang Jingren Zhou and Xia Hu. 2020. Learning to Hash with Graph Neural Networks for Recommender Systems. In WWW. 1988\u20131998."},{"key":"e_1_3_2_1_24_1","unstructured":"Mengting Wan Di Wang Jie Liu Paul Bennett and Julian McAuley. 2018. Representing and Recommending Shopping Baskets with Complementarity Compatibility and Loyalty. In CIKM. 1133\u20131142. Mengting Wan Di Wang Jie Liu Paul Bennett and Julian McAuley. 2018. Representing and Recommending Shopping Baskets with Complementarity Compatibility and Loyalty. In CIKM. 1133\u20131142."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330989"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Xiang Wang Xiangnan He Meng Wang Fuli Feng and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR. 165\u2013174. Xiang Wang Xiangnan He Meng Wang Fuli Feng and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR. 165\u2013174.","DOI":"10.1145\/3331184.3331267"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Teng Xiao Shangsong Liang and Zaiqiao Meng. 2019. Dynamic Collaborative Recurrent Learning. In CIKM. 1151\u20131160. Teng Xiao Shangsong Liang and Zaiqiao Meng. 2019. Dynamic Collaborative Recurrent Learning. In CIKM. 1151\u20131160.","DOI":"10.1145\/3357384.3357901"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"crossref","unstructured":"Hamed Zamani and W\u00a0Bruce Croft. 2020. Learning a Joint Search and Recommendation Model from User-Item Interactions. In WSDM. 717\u2013725. Hamed Zamani and W\u00a0Bruce Croft. 2020. Learning a Joint Search and Recommendation Model from User-Item Interactions. In WSDM. 717\u2013725.","DOI":"10.1145\/3336191.3371818"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3285029","article-title":"Deep learning based recommender system: A survey and new perspectives","volume":"52","author":"Zhang Shuai","year":"2019","journal-title":"Comput. Surveys"}],"event":{"name":"RecSys '20: Fourteenth ACM Conference on Recommender Systems","location":"Virtual Event Brazil","acronym":"RecSys '20","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGAI ACM Special Interest Group on Artificial Intelligence","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGIR ACM Special Interest Group on Information Retrieval","SIGCHI ACM Special Interest Group on Computer-Human Interaction","SIGecom Special Interest Group on Economics and Computation"]},"container-title":["Fourteenth ACM Conference on Recommender Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3383313.3418479","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3383313.3418479","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:33:21Z","timestamp":1750185201000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3383313.3418479"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,22]]},"references-count":29,"alternative-id":["10.1145\/3383313.3418479","10.1145\/3383313"],"URL":"https:\/\/doi.org\/10.1145\/3383313.3418479","relation":{},"subject":[],"published":{"date-parts":[[2020,9,22]]}}}