{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T04:42:56Z","timestamp":1780634576853,"version":"3.54.1"},"reference-count":140,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,12,21]],"date-time":"2022-12-21T00:00:00Z","timestamp":1671580800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61872369 and 61802029"],"award-info":[{"award-number":["61872369 and 61802029"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Natural Science Foundation","award":["4222027"],"award-info":[{"award-number":["4222027"]}]},{"name":"Beijing Outstanding Young Scientist Program","award":["BJJWZYJH012019100020098"],"award-info":[{"award-number":["BJJWZYJH012019100020098"]}]},{"name":"Beijing Academy of Artificial Intelligence"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,4,30]]},"abstract":"<jats:p>\n            In recommender systems, top-\n            <jats:italic>N<\/jats:italic>\n            recommendation is an important task with implicit feedback data. Although the recent success of deep learning largely pushes forward the research on top-\n            <jats:italic>N<\/jats:italic>\n            recommendation, there are increasing concerns on appropriate evaluation of recommendation algorithms. It therefore is important to study how recommendation algorithms can be reliably evaluated and thoroughly verified. This work presents a large-scale, systematic study on six important factors from three aspects for evaluating recommender systems. We carefully select 12 top-\n            <jats:italic>N<\/jats:italic>\n            recommendation algorithms and eight recommendation datasets. Our experiments are carefully designed and extensively conducted with these algorithms and datasets. In particular, all the experiments in our work are implemented based on an open sourced recommendation library, Recbole\u00a0[\n            <jats:xref ref-type=\"bibr\">139<\/jats:xref>\n            ], which ensures the reproducibility and reliability of our results. Based on the large-scale experiments and detailed analysis, we derive several key findings on the experimental settings for evaluating recommender systems. Our findings show that some settings can lead to substantial or significant differences in performance ranking of the compared algorithms. In response to recent evaluation concerns, we also provide several suggested settings that are specially important for performance comparison.\n          <\/jats:p>","DOI":"10.1145\/3545796","type":"journal-article","created":{"date-parts":[[2022,6,27]],"date-time":"2022-06-27T12:53:55Z","timestamp":1656334435000},"page":"1-41","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["A Revisiting Study of Appropriate Offline Evaluation for Top-\n            <i>N<\/i>\n            Recommendation Algorithms"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8333-6196","authenticated-orcid":false,"given":"Wayne Xin","family":"Zhao","sequence":"first","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6877-4470","authenticated-orcid":false,"given":"Zihan","family":"Lin","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6913-9260","authenticated-orcid":false,"given":"Zhichao","family":"Feng","sequence":"additional","affiliation":[{"name":"Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8658-7102","authenticated-orcid":false,"given":"Pengfei","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing University of Posts and Telecommunications, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9777-9676","authenticated-orcid":false,"given":"Ji-Rong","family":"Wen","sequence":"additional","affiliation":[{"name":"Renmin University of China, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,12,21]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2507189"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11192-021-03909-y"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463245"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/HiPC.2018.00014"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412226"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2005.1554761"},{"key":"e_1_3_3_8_2","first-page":"35","volume-title":"Proceedings of the KDD Cup and Workshop","volume":"2007","author":"Bennett James","year":"2007","unstructured":"James Bennett, Stan Lanning, et\u00a0al. 2007. The netflix prize. In Proceedings of the KDD Cup and Workshop, Vol. 2007. Citeseer, 35."},{"key":"e_1_3_3_9_2","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. J. Mach. Learn. Res. 13, null (Feb.2012), 281\u2013305.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412259"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043990"},{"key":"e_1_3_3_12_2","first-page":"1","article-title":"Offline evaluation options for recommender systems","author":"Ca\u00f1amares Roc\u00edo","year":"2020","unstructured":"Roc\u00edo Ca\u00f1amares, Pablo Castells, and Alistair Moffat. 2020. Offline evaluation options for recommender systems. Inf. Retriev. J. (2020), 1\u201324.","journal-title":"Inf. Retriev. J."},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2044016"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44801-2_15"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290982"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373807"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i01.5329"},{"key":"e_1_3_3_18_2","article-title":"Bias and debias in recommender system: A survey and future directions","author":"Chen Jiawei","year":"2020","unstructured":"Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and debias in recommender system: A survey and future directions. arXiv:2010.03240. Retrieved from https:\/\/arxiv.org\/abs\/2010.03240.","journal-title":"arXiv:2010.03240"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/293"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330880"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3270323.3270326"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498519"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220112"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434185"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347058"},{"key":"e_1_3_3_26_2","first-page":"4706","volume-title":"Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI\u201920)","author":"Dacrema Maurizio Ferrari","year":"2020","unstructured":"Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2020. Methodological issues in recommender systems research. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (IJCAI\u201920). 4706\u20134710."},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401046"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.330161"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963776"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783346"},{"key":"e_1_3_3_31_2","unstructured":"Jingtao Ding Yuhan Quan Quanming Yao Yong Li and Depeng Jin. 2020. Simplify and robustify negative sampling for implicit collaborative filtering. arXiv:2009.03376. Retrieved from https:\/\/arxiv.org\/abs\/2009.03376."},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372338"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3209991"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347036"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347036"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741667"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645766"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347055"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043932.2043989"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159687"},{"key":"e_1_3_3_41_2","volume-title":"UMAP Workshops","author":"Guo Guibing","year":"2015","unstructured":"Guibing Guo, Jie Zhang, Zhu Sun, and Neil Yorke-Smith. 2015. LibRec: A java library for recommender systems. In UMAP Workshops, Vol. 4. Citeseer."},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA45697.2020.00084"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/2827872"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380155"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401063"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3209981"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2831682"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911489"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911489"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271761"},{"key":"e_1_3_3_52_2","article-title":"Session-based recommendations with recurrent neural networks","author":"Hidasi Bal\u00e1zs","year":"2015","unstructured":"Bal\u00e1zs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv:1511.06939. Retrieved from https:\/\/arxiv.org\/abs\/1511.06939.","journal-title":"arXiv:1511.06939"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219965"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347069"},{"key":"e_1_3_3_56_2","doi-asserted-by":"crossref","unstructured":"Olivier Jeunen Koen Verstrepen and Bart Goethals. 2018. Fair offline evaluation methodologies for implicit-feedback recommender systems with MNAR data. In Proceedings of the REVEAL 18 Workshop on Offine Evaluation October 2018 Vancouver Canada.","DOI":"10.1145\/3298689.3347069"},{"key":"e_1_3_3_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403253"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401233"},{"key":"e_1_3_3_59_2","unstructured":"Yitong Ji Aixin Sun Jie Zhang and Chenliang Li. 2021. A critical study on data leakage in recommender system offline evaluation. arxiv:2010.11060 [cs.IR]. Retrieved from https:\/\/arxiv.org\/abs\/2010.11060."},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959134"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2487589"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-7687-1_891"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/1401890.1401944"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403226"},{"key":"e_1_3_3_66_2","article-title":"Metadata embeddings for user and item cold-start recommendations","author":"Kula Maciej","year":"2015","unstructured":"Maciej Kula. 2015. Metadata embeddings for user and item cold-start recommendations. arXiv:1507.08439. Retrieved from https:\/\/arxiv.org\/abs\/1507.08439.","journal-title":"arXiv:1507.08439"},{"key":"e_1_3_3_67_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-31178-9_15"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132913"},{"key":"e_1_3_3_69_2","unstructured":"DongSheng Li Chao Chen Qin Lv Li Shang Stephen Chu and Hongyuan Zha. 2017. ERMMA: Expected risk minimization for matrix approximation-based recommender systems. In Proceedings of the Thirty-first AAAI Conference on Artificial Intelligence ."},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403262"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098077"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883090"},{"key":"e_1_3_3_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186150"},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290977"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357892"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767755"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/1125451.1125659"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271813"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401096"},{"key":"e_1_3_3_80_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3418479"},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3418479"},{"key":"e_1_3_3_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3346985"},{"key":"e_1_3_3_83_2","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3291016"},{"key":"e_1_3_3_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109903"},{"key":"e_1_3_3_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.16"},{"key":"e_1_3_3_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109916"},{"key":"e_1_3_3_87_2","article-title":"Value-aware recommendation based on reinforced profit maximization in e-commerce systems","author":"Pei Changhua","year":"2019","unstructured":"Changhua Pei, Xinru Yang, Qing Cui, Xiao Lin, Fei Sun, Peng Jiang, Wenwu Ou, and Yongfeng Zhang. 2019. Value-aware recommendation based on reinforced profit maximization in e-commerce systems. arXiv:1902.00851. Retrieved from https:\/\/arxiv.org\/abs\/1902.00851.","journal-title":"arXiv:1902.00851"},{"key":"e_1_3_3_88_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-92007-8_34"},{"key":"e_1_3_3_89_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(98)00010-0"},{"key":"e_1_3_3_90_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2010.127"},{"key":"e_1_3_3_91_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 452\u2013461."},{"key":"e_1_3_3_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412488"},{"key":"e_1_3_3_93_2","article-title":"On the difficulty of evaluating baselines: A study on recommender systems","author":"Rendle Steffen","year":"2019","unstructured":"Steffen Rendle, Li Zhang, and Yehuda Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. arXiv:1905.01395. Retrieved from https:\/\/arxiv.org\/abs\/1905.01395.","journal-title":"arXiv:1905.01395"},{"key":"e_1_3_3_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102441"},{"key":"e_1_3_3_95_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-85820-3_1"},{"key":"e_1_3_3_96_2","doi-asserted-by":"publisher","DOI":"10.5555\/1941884"},{"key":"e_1_3_3_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959176"},{"key":"e_1_3_3_98_2","doi-asserted-by":"publisher","DOI":"10.1145\/2645710.2645746"},{"key":"e_1_3_3_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/2792838.2792841"},{"key":"e_1_3_3_100_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-85820-3_8"},{"issue":"5","key":"e_1_3_3_101_2","first-page":"1989","article-title":"A survey of recommendation system: Research challenges","volume":"4","author":"Sharma Lalita","year":"2013","unstructured":"Lalita Sharma and Anju Gera. 2013. A survey of recommendation system: Research challenges. Int. J. Eng. Trends Technol. 4, 5 (2013), 1989\u20131992.","journal-title":"Int. J. Eng. Trends Technol."},{"key":"e_1_3_3_102_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-017-0762-9"},{"key":"e_1_3_3_103_2","first-page":"2951","volume-title":"Advances in Neural Information Processing Systems","author":"Snoek Jasper","year":"2012","unstructured":"Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 25. Curran Associates, Inc., 2951\u20132959."},{"key":"e_1_3_3_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/2507157.2507160"},{"key":"e_1_3_3_105_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403254"},{"key":"e_1_3_3_106_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411947"},{"key":"e_1_3_3_107_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/391"},{"key":"e_1_3_3_108_2","doi-asserted-by":"publisher","DOI":"10.1145\/3383313.3412489"},{"key":"e_1_3_3_109_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313460"},{"key":"e_1_3_3_110_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240347"},{"key":"e_1_3_3_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/2600428.2610382"},{"key":"e_1_3_3_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313411"},{"key":"e_1_3_3_113_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330989"},{"key":"e_1_3_3_114_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331267"},{"key":"e_1_3_3_115_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380098"},{"key":"e_1_3_3_116_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/498"},{"key":"e_1_3_3_117_2","unstructured":"Bin Wu Zhongchuan Sun Xiangnan He Xiang Wang and Jonathan Staniforth. 2017. NeuRec. Retrieved from https:\/\/github.com\/wubinzzu\/NeuRec."},{"key":"e_1_3_3_118_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331201"},{"key":"e_1_3_3_119_2","doi-asserted-by":"publisher","DOI":"10.1145\/2835776.2835837"},{"key":"e_1_3_3_120_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/545"},{"key":"e_1_3_3_121_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331188"},{"key":"e_1_3_3_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357924"},{"key":"e_1_3_3_123_2","doi-asserted-by":"publisher","DOI":"10.1145\/3314578"},{"key":"e_1_3_3_124_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/447"},{"key":"e_1_3_3_125_2","doi-asserted-by":"publisher","DOI":"10.1145\/3240323.3240355"},{"key":"e_1_3_3_126_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462928"},{"key":"e_1_3_3_127_2","first-page":"847","volume-title":"Proceedings of the Conference and Workshop on Neural Information Processing Systems (NIPS\u201916)","author":"Yu Hsiang-Fu","year":"2016","unstructured":"Hsiang-Fu Yu, Nikhil Rao, and Inderjit S. Dhillon. 2016. Temporal regularized matrix factorization for high-dimensional time series prediction. In Proceedings of the Conference and Workshop on Neural Information Processing Systems (NIPS\u201916). 847\u2013855."},{"key":"e_1_3_3_128_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271725"},{"key":"e_1_3_3_129_2","doi-asserted-by":"crossref","unstructured":"Lu Yu Chuxu Zhang Shichao Pei Guolei Sun and Xiangliang Zhang. 2018. WalkRanker: A unified pairwise ranking model with multiple relations for item recommendation.","DOI":"10.1609\/aaai.v32i1.11866"},{"key":"e_1_3_3_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401155"},{"key":"e_1_3_3_131_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3186146"},{"issue":"1","key":"e_1_3_3_132_2","first-page":"5:1\u20135:38","article-title":"Deep learning based recommender system: A survey and new perspectives","volume":"52","author":"Zhang Shuai","year":"2019","unstructured":"Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep learning based recommender system: A survey and new perspectives. ACM Comput. Surv. 52, 1 (2019), 5:1\u20135:38.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_3_133_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/599"},{"key":"e_1_3_3_134_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080689"},{"key":"e_1_3_3_135_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132892"},{"key":"e_1_3_3_136_2","doi-asserted-by":"publisher","DOI":"10.1145\/3336191.3371790"},{"key":"e_1_3_3_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159688"},{"key":"e_1_3_3_138_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2017\/485"},{"key":"e_1_3_3_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3412095"},{"key":"e_1_3_3_140_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482016"},{"key":"e_1_3_3_141_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219826"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3545796","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3545796","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:46Z","timestamp":1750186966000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3545796"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,21]]},"references-count":140,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,4,30]]}},"alternative-id":["10.1145\/3545796"],"URL":"https:\/\/doi.org\/10.1145\/3545796","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,21]]},"assertion":[{"value":"2021-05-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-05-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}