{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T10:18:28Z","timestamp":1794651508826,"version":"4.9.1"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,1,31]],"date-time":"2023-01-31T00:00:00Z","timestamp":1675123200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["U21B2026, 62002191"],"award-info":[{"award-number":["U21B2026, 62002191"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Tsinghua University Guoqiang Research Institute"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,1,31]]},"abstract":"<jats:p>Recommendation systems play an important role in alleviating the information overload issue. Generally, a recommendation model is trained to discern between positive (liked) and negative (disliked) instances for each user. However, under the open-world assumption, there are only positive instances but no negative instances from users\u2019 implicit feedback, which poses the imbalanced learning challenge of lacking negative samples. To address this, two types of learning strategies have been proposed before, the negative sampling strategy and non-sampling strategy. The first strategy samples negative instances from missing data (i.e., unlabeled data), while the non-sampling strategy regards all the missing data as negative. Although learning strategies are known to be essential for algorithm performance, the in-depth comparison of negative sampling and non-sampling has not been sufficiently explored by far. To bridge this gap, we systematically analyze the role of negative sampling and non-sampling for implicit recommendation in this work. Specifically, we first theoretically revisit the objection of negative sampling and non-sampling. Then, with a careful setup of various representative recommendation methods, we explore the performance of negative sampling and non-sampling in different scenarios. Our results empirically show that although negative sampling has been widely applied to recent recommendation models, it is non-trivial for uniform sampling methods to show comparable performance to non-sampling learning methods. Finally, we discuss the scalability and complexity of negative sampling and non-sampling and present some open problems and future research topics that are worth being further explored.<\/jats:p>","DOI":"10.1145\/3522672","type":"journal-article","created":{"date-parts":[[2022,3,25]],"date-time":"2022-03-25T06:20:02Z","timestamp":1648189202000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":102,"title":["Revisiting Negative Sampling vs. Non-sampling in Implicit Recommendation"],"prefix":"10.1145","volume":"41","author":[{"given":"Chong","family":"Chen","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weizhi","family":"Ma","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenyang","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiqun","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shaoping","family":"Ma","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,2,25]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"1341","DOI":"10.1145\/3038912.3052694","volume-title":"Proceedings of the 26th International Conference on World Wide Web","author":"Bayer Immanuel","year":"2017","unstructured":"Immanuel Bayer, Xiangnan He, Bhargav Kanagal, and Steffen Rendle. 2017. A generic coordinate descent framework for learning from implicit feedback. In Proceedings of the 26th International Conference on World Wide Web. 1341\u20131350."},{"key":"e_1_3_2_3_2","article-title":"Adversarial contrastive estimation","author":"Bose Avishek Joey","year":"2018","unstructured":"Avishek Joey Bose, Huan Ling, and Yanshuai Cao. 2018. Adversarial contrastive estimation. arXiv preprint arXiv:1805.03642 (2018).","journal-title":"arXiv preprint arXiv:1805.03642"},{"key":"e_1_3_2_4_2","article-title":"KBGAN: Adversarial learning for knowledge graph embeddings","author":"Cai Liwei","year":"2017","unstructured":"Liwei Cai and William Yang Wang. 2017. KBGAN: Adversarial learning for knowledge graph embeddings. arXiv preprint arXiv:1711.04071 (2017).","journal-title":"arXiv preprint arXiv:1711.04071"},{"key":"e_1_3_2_5_2","volume-title":"Proceedings of the Web Conference 2022","author":"Chen Chong","year":"2022","unstructured":"Chong Chen, Fei Sun, Min Zhang, and Bolin Ding. 2022. Recommendation unlearning. In Proceedings of the Web Conference 2022."},{"key":"e_1_3_2_6_2","volume-title":"Proceedings of the Web Conference","author":"Chen Chong","year":"2018","unstructured":"Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2018. Neural attentional rating regression with review-level explanations. In Proceedings of the Web Conference."},{"key":"e_1_3_2_7_2","first-page":"177","volume-title":"Proceedings of the 12th ACM International Conference on Web Search and Data Mining","author":"Chen Chong","year":"2019","unstructured":"Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. 2019. Social attentional memory network: Modeling aspect-and friend-level differences in recommendation. In Proceedings of the 12th ACM International Conference on Web Search and Data Mining. 177\u2013185."},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"2400","DOI":"10.1145\/3366423.3380303","volume-title":"Proceedings of the Web Conference 2020","author":"Chen Chong","year":"2020","unstructured":"Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020. Efficient non-sampling factorization machines for optimal context-aware recommendation. In Proceedings of the Web Conference 2020. 2400\u20132410."},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1145\/3397271.3401040","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Chen Chong","year":"2020","unstructured":"Chong Chen, Min Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020. Jointly non-sampling learning for knowledge graph enhanced recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 189\u2013198."},{"key":"e_1_3_2_10_2","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1145\/3331184.3331192","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Chen Chong","year":"2019","unstructured":"Chong Chen, Min Zhang, Chenyang Wang, Weizhi Ma, Minming Li, Yiqun Liu, and Shaoping Ma. 2019. An efficient adaptive transfer neural network for social-aware recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 225\u2013234."},{"issue":"2","key":"e_1_3_2_11_2","article-title":"Efficient neural matrix factorization without sampling for recommendation","volume":"38","author":"Chen Chong","year":"2020","unstructured":"Chong Chen, Min Zhang, Yongfeng Zhang, Yiqun Liu, and Shaoping Ma. 2020. Efficient neural matrix factorization without sampling for recommendation. ACM Trans. Inf. Syst. 38, 2, (Jan.2020), Article 14.","journal-title":"ACM Trans. Inf. Syst."},{"key":"e_1_3_2_12_2","first-page":"19","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"34","author":"Chen Chong","year":"2020","unstructured":"Chong Chen, Min Zhang, Yongfeng Zhang, Weizhi Ma, Yiqun Liu, and Shaoping Ma. 2020. Efficient heterogeneous collaborative filtering without negative sampling for recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 19\u201326."},{"key":"e_1_3_2_13_2","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1145\/3077136.3080797","volume-title":"Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval","author":"Chen Jingyuan","year":"2017","unstructured":"Jingyuan Chen, Hanwang Zhang, Xiangnan He, Liqiang Nie, Wei Liu, and Tat-Seng Chua. 2017. Attentive collaborative filtering: Multimedia recommendation with item-and component-level attention. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 335\u2013344."},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1145\/2959100.2959190","volume-title":"Proceedings of the 10th ACM Conference on Recommender Systems","author":"Covington Paul","year":"2016","unstructured":"Paul Covington, Jay Adams, and Emre Sargin. 2016. Deep neural networks for Youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems. 191\u2013198."},{"key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1145\/3298689.3347058","volume-title":"Proceedings of the 13th ACM Conference on Recommender Systems","author":"Dacrema Maurizio Ferrari","year":"2019","unstructured":"Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. 2019. Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the 13th ACM Conference on Recommender Systems. 101\u2013109."},{"key":"e_1_3_2_16_2","volume-title":"Proceedings of the 29th ACM International Conference on Information & Knowledge Management","author":"Dacrema Maurizio Ferrari","year":"2020","unstructured":"Maurizio Ferrari Dacrema, Federico Parroni, Paolo Cremonesi, and Dietmar Jannach. 2020. Critically examining the claimed value of convolutions over user-item embedding maps for recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management."},{"key":"e_1_3_2_17_2","first-page":"189","volume-title":"Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Devooght Robin","year":"2015","unstructured":"Robin Devooght, Nicolas Kourtellis, and Amin Mantrach. 2015. Dynamic matrix factorization with priors on unknown values. In Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 189\u2013198."},{"key":"e_1_3_2_18_2","first-page":"1094","article-title":"Simplify and robustify negative sampling for implicit collaborative filtering","volume":"33","author":"Ding Jingtao","year":"2020","unstructured":"Jingtao Ding, Yuhan Quan, Quanming Yao, Yong Li, and Depeng Jin. 2020. Simplify and robustify negative sampling for implicit collaborative filtering. Advances in Neural Information Processing Systems 33 (2020), 1094\u20131105.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_19_2","first-page":"3343","volume-title":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","author":"Ding Jingtao","year":"2018","unstructured":"Jingtao Ding, Guanghui Yu, Xiangnan He, Yuhan Quan, Yong Li, Tat-Seng Chua, Depeng Jin, and Jiajie Yu. 2018. Improving implicit recommender systems with view data. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 3343\u20133349."},{"key":"e_1_3_2_20_2","first-page":"1775","volume-title":"Proceedings of the 2018 World Wide Web Conference","author":"Eksombatchai Chantat","year":"2018","unstructured":"Chantat Eksombatchai, Pranav Jindal, Jerry Zitao Liu, Yuchen Liu, Rahul Sharma, Charles Sugnet, Mark Ulrich, and Jure Leskovec. 2018. Pixie: A system for recommending 3+ billion items to 200+ million users in real-time. In Proceedings of the 2018 World Wide Web Conference. International World Wide Web Conferences Steering Committee, 1775\u20131784."},{"key":"e_1_3_2_21_2","article-title":"Graph neural networks for social recommendation","author":"Fan Wenqi","year":"2019","unstructured":"Wenqi Fan, Yao Ma, Qing Li, Yuan He, Eric Zhao, Jiliang Tang, and Dawei Yin. 2019. Graph neural networks for social recommendation. arXiv preprint arXiv:1902.07243 (2019).","journal-title":"arXiv preprint arXiv:1902.07243"},{"key":"e_1_3_2_22_2","volume-title":"International Conference on Data Engineering (ICDE\u201919)","author":"Gao Chen","year":"2019","unstructured":"Chen Gao, Xiangnan He, Dahua Gan, Xiangning Chen, Fuli Feng, Yong Li, Tat-Seng Chua, and Depeng Jin. 2019. Neural multi-task recommendation from multi-behavior data. In International Conference on Data Engineering (ICDE\u201919)."},{"key":"e_1_3_2_23_2","first-page":"1406","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Gao Hongchang","year":"2018","unstructured":"Hongchang Gao and Heng Huang. 2018. Self-paced network embedding. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1406\u20131415."},{"key":"e_1_3_2_24_2","first-page":"4274","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Geng Xue","year":"2015","unstructured":"Xue Geng, Hanwang Zhang, Jingwen Bian, and Tat-Seng Chua. 2015. Learning image and user features for recommendation in social networks. In Proceedings of the IEEE International Conference on Computer Vision. 4274\u20134282."},{"key":"e_1_3_2_25_2","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (NeurIPS\u201920)","author":"Grill Jean-Bastien","year":"2020","unstructured":"Jean-Bastien Grill, Florian Strub, Florent Altch\u00e9, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo \u00c1vila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R\u00e9mi Munos, and Michal Valko. 2020. Bootstrap your own latent - A new approach to self-supervised learning. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 (NeurIPS\u201920)."},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"855","DOI":"10.1145\/2939672.2939754","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Grover Aditya","year":"2016","unstructured":"Aditya Grover and Jure Leskovec. 2016. Node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855\u2013864."},{"key":"e_1_3_2_27_2","first-page":"355","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"He Xiangnan","year":"2017","unstructured":"Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 355\u2013364."},{"key":"e_1_3_2_28_2","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"He Xiangnan","year":"2020","unstructured":"Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. 2020. LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval."},{"key":"e_1_3_2_29_2","first-page":"2227","volume-title":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","author":"He Xiangnan","year":"2018","unstructured":"Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat-Seng Chua. 2018. Outer product-based neural collaborative filtering. In Proceedings of the 27th International Joint Conference on Artificial Intelligence. 2227\u20132233."},{"key":"e_1_3_2_30_2","first-page":"355","volume-title":"The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval","author":"He Xiangnan","year":"2018","unstructured":"Xiangnan He, Zhankui He, Xiaoyu Du, and Tat-Seng Chua. 2018. Adversarial personalized ranking for recommendation. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 355\u2013364."},{"key":"e_1_3_2_31_2","first-page":"173","volume-title":"Proceedings of the Web Conference 2017","author":"He Xiangnan","year":"2017","unstructured":"Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the Web Conference 2017. 173\u2013182."},{"issue":"8","key":"e_1_3_2_32_2","first-page":"2791","article-title":"Fast matrix factorization with nonuniform weights on missing data","volume":"31","author":"He Xiangnan","year":"2019","unstructured":"Xiangnan He, Jinhui Tang, Xiaoyu Du, Richang Hong, Tongwei Ren, and Tat-Seng Chua. 2019. Fast matrix factorization with nonuniform weights on missing data. IEEE Transactions on Neural Networks and Learning Systems 31, 8 (2019), 2791\u20132804.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_2_33_2","first-page":"549","volume-title":"Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"He Xiangnan","year":"2016","unstructured":"Xiangnan He, Hanwang Zhang, Min-Yen Kan, and Tat-Seng Chua. 2016. Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval. 549\u2013558."},{"key":"e_1_3_2_34_2","first-page":"263","volume-title":"2008 8th IEEE International Conference on Data Mining","author":"Hu Yifan","year":"2008","unstructured":"Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative filtering for implicit feedback datasets. In 2008 8th IEEE International Conference on Data Mining. IEEE, 263\u2013272."},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1145\/3447548.3467408","volume-title":"Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining","author":"Huang Tinglin","year":"2021","unstructured":"Tinglin Huang, Yuxiao Dong, Ming Ding, Zhen Yang, Wenzheng Feng, Xinyu Wang, and Jie Tang. 2021. MixGCF: An improved training method for graph neural network-based recommender systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 665\u2013674."},{"key":"e_1_3_2_36_2","first-page":"22629","article-title":"Sampling-decomposable generative adversarial recommender","volume":"33","author":"Jin Binbin","year":"2020","unstructured":"Binbin Jin, Defu Lian, Zheng Liu, Qi Liu, Jianhui Ma, Xing Xie, and Enhong Chen. 2020. Sampling-decomposable generative adversarial recommender. Advances in Neural Information Processing Systems 33 (2020), 22629\u201322639.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1145\/1401890.1401944","volume-title":"Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Koren Yehuda","year":"2008","unstructured":"Yehuda Koren. 2008. Factorization meets the neighborhood: A multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 426\u2013434."},{"issue":"8","key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/MC.2009.263","article-title":"Matrix factorization techniques for recommender systems","author":"Koren Yehuda","year":"2009","unstructured":"Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer8 (2009), 30\u201337.","journal-title":"Computer"},{"key":"e_1_3_2_39_2","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1145\/3394486.3403226","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Krichene Walid","year":"2020","unstructured":"Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1748\u20131757."},{"key":"e_1_3_2_40_2","article-title":"Bootstrapping user and item representations for one-class collaborative filtering","author":"Lee Dongha","year":"2021","unstructured":"Dongha Lee, SeongKu Kang, Hyunjun Ju, Chanyoung Park, and Hwanjo Yu. 2021. Bootstrapping user and item representations for one-class collaborative filtering. arXiv preprint arXiv:2105.06323 (2021).","journal-title":"arXiv preprint arXiv:2105.06323"},{"key":"e_1_3_2_41_2","first-page":"1727","volume-title":"Proceedings of the Web Conference 2021","author":"Li Zelong","year":"2021","unstructured":"Zelong Li, Jianchao Ji, Zuohui Fu, Yingqiang Ge, Shuyuan Xu, Chong Chen, and Yongfeng Zhang. 2021. Efficient non-sampling knowledge graph embedding. In Proceedings of the Web Conference 2021. 1727\u20131736."},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"1093","DOI":"10.1145\/3366423.3380187","volume-title":"Proceedings of the Web Conference 2020","author":"Lian Defu","year":"2020","unstructured":"Defu Lian, Qi Liu, and Enhong Chen. 2020. Personalized ranking with importance sampling. In Proceedings of the Web Conference 2020. 1093\u20131103."},{"key":"e_1_3_2_43_2","first-page":"951","volume-title":"Proceedings of the Web Conference 2016","author":"Liang Dawen","year":"2016","unstructured":"Dawen Liang, Laurent Charlin, James McInerney, and David M. Blei. 2016. Modeling user exposure in recommendation. In Proceedings of the Web Conference 2016. 951\u2013961."},{"key":"e_1_3_2_44_2","first-page":"583","article-title":"Related pins at Pinterest: The evolution of a real-world recommender system","author":"Liu David C.","year":"2017","unstructured":"David C. Liu, Stephanie Rogers, Raymond Shiau, Dmitry Kislyuk, Kevin C. Ma, Zhigang Zhong, Jenny Liu, and Yushi Jing. 2017. Related pins at Pinterest: The evolution of a real-world recommender system. Proceedings of the Web Conference 2017, 583\u2013592.","journal-title":"Proceedings of the Web Conference 2017"},{"issue":"4","key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s11257-018-9209-6","article-title":"Evaluation of session-based recommendation algorithms","volume":"28","author":"Ludewig Malte","year":"2018","unstructured":"Malte Ludewig and Dietmar Jannach. 2018. Evaluation of session-based recommendation algorithms. User Modeling and User-adapted Interaction 28, 4\u20135 (2018), 331\u2013390.","journal-title":"User Modeling and User-adapted Interaction"},{"key":"e_1_3_2_46_2","first-page":"3111","volume-title":"Advances in Neural Information Processing Systems","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems. 3111\u20133119."},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"1933","DOI":"10.1145\/3097983.3098108","volume-title":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Okura Shumpei","year":"2017","unstructured":"Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017. Embedding-based news recommendation for millions of users. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1933\u20131942."},{"key":"e_1_3_2_48_2","first-page":"502","volume-title":"2008 8th IEEE International Conference on Data Mining","author":"Pan Rong","year":"2008","unstructured":"Rong Pan, Yunhong Zhou, Bin Cao, Nathan N. Liu, Rajan Lukose, Martin Scholz, and Qiang Yang. 2008. One-class collaborative filtering. In 2008 8th IEEE International Conference on Data Mining. IEEE, 502\u2013511."},{"key":"e_1_3_2_49_2","article-title":"Bag of tricks for adversarial training","author":"Pang Tianyu","year":"2020","unstructured":"Tianyu Pang, Xiao Yang, Yinpeng Dong, Hang Su, and Jun Zhu. 2020. Bag of tricks for adversarial training. arXiv preprint arXiv:2010.00467 (2020).","journal-title":"arXiv preprint arXiv:2010.00467"},{"key":"e_1_3_2_50_2","first-page":"1443","volume-title":"Proceedings of the Web Conference 2019","author":"Park Dae Hoon","year":"2019","unstructured":"Dae Hoon Park and Yi Chang. 2019. Adversarial sampling and training for semi-supervised information retrieval. In Proceedings of the Web Conference 2019. 1443\u20131453."},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1145\/2623330.2623732","volume-title":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Perozzi Bryan","year":"2014","unstructured":"Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. 2014. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 701\u2013710."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1145\/2556195.2556248","volume-title":"Proceedings of the 7th ACM International Conference on Web Search and Data Mining","author":"Rendle Steffen","year":"2014","unstructured":"Steffen Rendle and Christoph Freudenthaler. 2014. Improving pairwise learning for item recommendation from implicit feedback. In Proceedings of the 7th ACM International Conference on Web Search and Data Mining. 273\u2013282."},{"key":"e_1_3_2_53_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 452\u2013461."},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1145\/3383313.3412488","volume-title":"14th ACM Conference on Recommender Systems","author":"Rendle Steffen","year":"2020","unstructured":"Steffen Rendle, Walid Krichene, Li Zhang, and John Anderson. 2020. Neural collaborative filtering vs. matrix factorization revisited. In 14th ACM Conference on Recommender Systems. 240\u2013248."},{"key":"e_1_3_2_55_2","article-title":"On the difficulty of evaluating baselines: A study on recommender systems","author":"Rendle Steffen","year":"2019","unstructured":"Steffen Rendle, Li Zhang, and Yehuda Koren. 2019. On the difficulty of evaluating baselines: A study on recommender systems. arXiv preprint arXiv:1905.01395 (2019).","journal-title":"arXiv preprint arXiv:1905.01395"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-85820-3","volume-title":"Recommender Systems Handbook","author":"Ricci Francesco","year":"2011","unstructured":"Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender Systems Handbook. Springer."},{"key":"e_1_3_2_57_2","first-page":"285","volume-title":"Proceedings of the Web Conference 2001","author":"Sarwar Badrul","year":"2001","unstructured":"Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the Web Conference 2001. 285\u2013295."},{"key":"e_1_3_2_58_2","article-title":"Green AI","author":"Schwartz Roy","year":"2019","unstructured":"Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. 2019. Green AI. arXiv preprint arXiv:1907.10597 (2019).","journal-title":"arXiv preprint arXiv:1907.10597"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1145\/2736277.2741093","volume-title":"Proceedings of the 24th International Conference on World Wide Web","author":"Tang Jian","year":"2015","unstructured":"Jian Tang, Meng Qu, Mingzhe Wang, Ming Zhang, Jun Yan, and Qiaozhu Mei. 2015. Line: Large-scale information network embedding. In Proceedings of the 24th International Conference on World Wide Web. 1067\u20131077."},{"key":"e_1_3_2_60_2","first-page":"729","volume-title":"Proceedings of the Web Conference 2018","author":"Tay Yi","year":"2018","unstructured":"Yi Tay, Luu Anh Tuan, and Siu Cheung Hui. 2018. Latent relational metric learning via memory-based attention for collaborative ranking. In Proceedings of the Web Conference 2018. 729\u2013739."},{"issue":"3","key":"e_1_3_2_61_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.4018\/jdwm.2007070101","article-title":"Multi-label classification: An overview","volume":"3","author":"Tsoumakas Grigorios","year":"2007","unstructured":"Grigorios Tsoumakas and Ioannis Katakis. 2007. Multi-label classification: An overview. International Journal of Data Warehousing and Mining 3, 3 (2007), 1\u201313.","journal-title":"International Journal of Data Warehousing and Mining"},{"key":"e_1_3_2_62_2","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1145\/3077136.3080786","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang Jun","year":"2017","unstructured":"Jun Wang, Lantao Yu, Weinan Zhang, Yu Gong, Yinghui Xu, Benyou Wang, Peng Zhang, and Dell Zhang. 2017. Irgan: A minimax game for unifying generative and discriminative information retrieval models. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 515\u2013524."},{"key":"e_1_3_2_63_2","unstructured":"Menghan Wang Mingming Gong Xiaolin Zheng and Kun Zhang. 2018. Modeling dynamic missingness of implicit feedback for recommendation. In Proceedings of the 32nd International Conference on Neural Information Processing Systems 6670\u20136679."},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"950","DOI":"10.1145\/3292500.3330989","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Wang Xiang","year":"2019","unstructured":"Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, and Tat-Seng Chua. 2019. KGAT: Knowledge graph attention network for recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 950\u2013958."},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1145\/3077136.3080771","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang Xiang","year":"2017","unstructured":"Xiang Wang, Xiangnan He, Liqiang Nie, and Tat-Seng Chua. 2017. Item silk road: Recommending items from information domains to social users. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval. 185\u2013194."},{"key":"e_1_3_2_66_2","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1145\/3331184.3331267","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang Xiang","year":"2019","unstructured":"Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. 2019. Neural graph collaborative filtering. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 165\u2013174."},{"key":"e_1_3_2_67_2","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1145\/3366423.3380098","volume-title":"Proceedings of the Web Conference 2020","author":"Wang Xiang","year":"2020","unstructured":"Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, and Tat-Seng Chua. 2020. Reinforced negative sampling over knowledge graph for recommendation. In Proceedings of the Web Conference 2020. 99\u2013109."},{"key":"e_1_3_2_68_2","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1145\/3132847.3132911","volume-title":"Proceedings of the 2017 ACM Conference on Information and Knowledge Management","author":"Xiao Lin","year":"2017","unstructured":"Lin Xiao, Zhang Min, Zhang Yongfeng, Liu Yiqun, and Ma Shaoping. 2017. Learning and transferring social and item visibilities for personalized recommendation. In Proceedings of the 2017 ACM Conference on Information and Knowledge Management. 337\u2013346."},{"key":"e_1_3_2_69_2","volume-title":"Proceedings of the 28th International Joint Conference on Artificial Intelligence","author":"Xin Xin","year":"2019","unstructured":"Xin Xin, Bo Chen, Xiangnan He, Dong Wang, Yue Ding, and Joemon Jose. 2019. CFM: Convolutional factorization machines for context-aware recommendation. In Proceedings of the 28th International Joint Conference on Artificial Intelligence."},{"key":"e_1_3_2_70_2","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1145\/3331184.3331188","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Xin Xin","year":"2019","unstructured":"Xin Xin, Xiangnan He, Yongfeng Zhang, Yongdong Zhang, and Joemon Jose. 2019. Relational collaborative filtering: Modeling multiple item relations for recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 125\u2013134."},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"1853","DOI":"10.18653\/v1\/P18-1172","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Xin Xin","year":"2018","unstructured":"Xin Xin, Fajie Yuan, Xiangnan He, and Joemon M. Jose. 2018. Batch IS NOT heavy: Learning word representations from all samples. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1853\u20131862."},{"issue":"3","key":"e_1_3_2_72_2","first-page":"33","article-title":"Deep item-based collaborative filtering for top-N recommendation","volume":"37","author":"Xue Feng","year":"2019","unstructured":"Feng Xue, Xiangnan He, Xiang Wang, Jiandong Xu, Kai Liu, and Richang Hong. 2019. Deep item-based collaborative filtering for top-N recommendation. ACM Transactions on Information Systems (TOIS) 37, 3 (2019), 33.","journal-title":"ACM Transactions on Information Systems (TOIS)"},{"key":"e_1_3_2_73_2","first-page":"441","volume-title":"Companion Proceedings of the Web Conference 2020","author":"Yang Ji","year":"2020","unstructured":"Ji Yang, Xinyang Yi, Derek Zhiyuan Cheng, Lichan Hong, Yang Li, Simon Xiaoming Wang, Taibai Xu, and Ed H. Chi. 2020. Mixed negative sampling for learning two-tower neural networks in recommendations. In Companion Proceedings of the Web Conference 2020. 441\u2013447."},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","first-page":"1666","DOI":"10.1145\/3394486.3403218","volume-title":"Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Yang Zhen","year":"2020","unstructured":"Zhen Yang, Ming Ding, Chang Zhou, Hongxia Yang, Jingren Zhou, and Jie Tang. 2020. Understanding negative sampling in graph representation learning. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1666\u20131676."},{"key":"e_1_3_2_75_2","first-page":"269","volume-title":"Proceedings of the 13th ACM Conference on Recommender Systems","author":"Yi Xinyang","year":"2019","unstructured":"Xinyang Yi, Ji Yang, Lichan Hong, Derek Zhiyuan Cheng, Lukasz Heldt, Aditee Kumthekar, Zhe Zhao, Li Wei, and Ed Chi. 2019. Sampling-bias-corrected neural modeling for large corpus item recommendations. In Proceedings of the 13th ACM Conference on Recommender Systems. 269\u2013277."},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","first-page":"974","DOI":"10.1145\/3219819.3219890","volume-title":"Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Ying Rex","year":"2018","unstructured":"Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, and Jure Leskovec. 2018. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 974\u2013983."},{"key":"e_1_3_2_77_2","first-page":"363","volume-title":"Proceedings of the 2017 SIAM International Conference on Data Mining","author":"Yu Hsiang-Fu","year":"2017","unstructured":"Hsiang-Fu Yu, Mikhail Bilenko, and Chih-Jen Lin. 2017. Selection of negative samples for one-class matrix factorization. In Proceedings of the 2017 SIAM International Conference on Data Mining. SIAM, 363\u2013371."},{"key":"e_1_3_2_78_2","first-page":"198","volume-title":"Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence","author":"Yuan Fajie","year":"2018","unstructured":"Fajie Yuan, Xin Xin, Xiangnan He, Guibing Guo, Weinan Zhang, Tat-Seng Chua, and Jose M. Joemon. 2018. fBGD: Learning embeddings from positive unlabeled data with BGD. In Proceedings of the 34th Conference on Uncertainty in Artificial Intelligence. 198\u2013207."},{"key":"e_1_3_2_79_2","doi-asserted-by":"crossref","first-page":"785","DOI":"10.1145\/2484028.2484126","volume-title":"Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhang Weinan","year":"2013","unstructured":"Weinan Zhang, Tianqi Chen, Jun Wang, and Yong Yu. 2013. Optimizing top-n collaborative filtering via dynamic negative item sampling. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval. 785\u2013788."},{"key":"e_1_3_2_80_2","doi-asserted-by":"crossref","first-page":"614","DOI":"10.1109\/ICDE.2019.00061","volume-title":"2019 IEEE 35th International Conference on Data Engineering (ICDE\u201919)","author":"Zhang Yongqi","year":"2019","unstructured":"Yongqi Zhang, Quanming Yao, Yingxia Shao, and Lei Chen. 2019. NSCaching: Simple and efficient negative sampling for knowledge graph embedding. In 2019 IEEE 35th International Conference on Data Engineering (ICDE\u201919). IEEE, 614\u2013625."},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1145\/3298689.3346997","volume-title":"Proceedings of the 13th ACM Conference on Recommender Systems","author":"Zhao Zhe","year":"2019","unstructured":"Zhe Zhao, Lichan Hong, Li Wei, Jilin Chen, Aniruddh Nath, Shawn Andrews, Aditee Kumthekar, Maheswaran Sathiamoorthy, Xinyang Yi, and Ed Chi. 2019. Recommending what video to watch next: A multitask ranking system. In Proceedings of the 13th ACM Conference on Recommender Systems. ACM, 43\u201351."},{"key":"e_1_3_2_82_2","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1145\/3018661.3018665","volume-title":"Proceedings of the 10th ACM International Conference on Web Search and Data Mining","author":"Zheng Lei","year":"2017","unstructured":"Lei Zheng, Vahid Noroozi, and Philip S. Yu. 2017. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining. ACM, 425\u2013434."},{"key":"e_1_3_2_83_2","article-title":"SelfCF: A simple framework for self-supervised collaborative filtering","author":"Zhou Xin","year":"2021","unstructured":"Xin Zhou, Aixin Sun, Yong Liu, Jie Zhang, and Chunyan Miao. 2021. SelfCF: A simple framework for self-supervised collaborative filtering. arXiv preprint arXiv:2107.03019 (2021).","journal-title":"arXiv preprint arXiv:2107.03019"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3522672","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3522672","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:30:15Z","timestamp":1750188615000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3522672"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,31]]},"references-count":82,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,31]]}},"alternative-id":["10.1145\/3522672"],"URL":"https:\/\/doi.org\/10.1145\/3522672","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,31]]},"assertion":[{"value":"2021-05-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-02-24","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-02-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}