{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,15]],"date-time":"2026-03-15T16:57:35Z","timestamp":1773593855779,"version":"3.50.1"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,4,10]],"date-time":"2025-04-10T00:00:00Z","timestamp":1744243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"\u201cResearch and Development Project of the Enhanced Infrastructures for Post-5G Information and Communication Systems\u201d","award":["JPNP20017"],"award-info":[{"award-number":["JPNP20017"]}]},{"name":"New Energy and Industrial Technology Development Organization"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Recomm. Syst."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>Modern recommender systems heavily rely on high-quality representations learned from high-dimensional sparse data. While significant efforts have been invested in designing powerful algorithms for extracting user preferences, the factors contributing to good representations have remained relatively unexplored. In this work, we shed light on an issue in the existing pairwise learning paradigm (i.e., embedding collapse), that the representations tend to span a subspace of the whole embedding space, leading to a suboptimal solution and reducing the model capacity. Specifically, we show that alignment of positive pairs is equivalent to a low-pass filter causing users and items to collapse to a constant vector. While negative sampling can partially mitigate this issue by acting as a high-pass filter to balance the spectrum, leading to an incomplete collapse.<\/jats:p>\n          <jats:p>\n            To tackle this issue, we present a novel learning paradigm DirectSpec, which directly optimizes the spectrum distribution to ensure that users and items effectively span the entire embedding space. We demonstrate that many self-supervised learning algorithms without explicit negative sampling can be considered as special cases of DirectSpec. Furthermore, we show that optimizing the spectrum inappropriately could also be detrimental to data representation, where the key lies in a dynamic balance between alignment of positive pairs and spectrum balancing. Finally, we propose an enhanced and practical implementation DirectSpec\n            <jats:sup>+<\/jats:sup>\n            to balance the embedding spectrum more adaptively and effectively. We implement DirectSpec\n            <jats:sup>+<\/jats:sup>\n            on two popular recommender models: matrix factorization and LightGCN. Our experimental results demonstrate its effectiveness and efficiency over competitive baselines.\n          <\/jats:p>","DOI":"10.1145\/3718488","type":"journal-article","created":{"date-parts":[[2025,2,17]],"date-time":"2025-02-17T11:23:58Z","timestamp":1739791438000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Balancing Embedding Spectrum for Recommendation"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4020-9100","authenticated-orcid":false,"given":"Shaowen","family":"Peng","sequence":"first","affiliation":[{"name":"Nara Institute of Science and Technology, Ikoma, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3962-821X","authenticated-orcid":false,"given":"Kazunari","family":"Sugiyama","sequence":"additional","affiliation":[{"name":"Osaka Seikei University, Osaka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2336-7409","authenticated-orcid":false,"given":"Xin","family":"Liu","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Research Center, Koto-ku, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7462-8074","authenticated-orcid":false,"given":"Tsunenori","family":"Mine","sequence":"additional","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,4,10]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i1.8715"},{"key":"e_1_3_2_3_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR\u201923)","author":"Cai Xuheng","year":"2023","unstructured":"Xuheng Cai, Chao Huang, Lianghao Xia, and Xubin Ren. 2023. LightGCL: Simple yet effective graph contrastive learning for recommendation. In Proceedings of the 11th International Conference on Learning Representations (ICLR\u201923)."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3373807"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Huiyuan Chen Vivian Lai Hongye Jin Zhimeng Jiang Mahashweta Das and Xia Hu. 2024. Towards mitigating dimensional collapse of representations in collaborative filtering. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM\u201924). 106\u2013115.","DOI":"10.1145\/3616855.3635832"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080797"},{"key":"e_1_3_2_7_2","first-page":"1597","volume-title":"Proceedings of the International Conference on Machine Learning (ICML\u201920)","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning (ICML\u201920). 1597\u20131607."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01549"},{"key":"e_1_3_2_9_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)","author":"Gasteiger Johannes","year":"2019","unstructured":"Johannes Gasteiger, Aleksandar Bojchevski, and Stephan G\u00fcnnemann. 2019. Predict then propagate: Graph neural networks meet personalized PageRank. In Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)."},{"key":"e_1_3_2_10_2","first-page":"249","volume-title":"Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS\u201910)","author":"Glorot Xavier","year":"2010","unstructured":"Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS\u201910). 249\u2013256."},{"key":"e_1_3_2_11_2","first-page":"21271","article-title":"Bootstrap your own latent a new approach to self-supervised learning","author":"Grill Jean-Bastien","year":"2020","unstructured":"Jean-Bastien Grill, Florian Strub, Florent Altch\u00e9, Tallec, et\u00a0al. 2020. Bootstrap your own latent a new approach to self-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS\u201920). ACM, New York, NY, 21271\u201321284.","journal-title":"Advances in Neural Information Processing Systems (NeurIPS\u201920)"},{"key":"e_1_3_2_12_2","volume-title":"Proceedings of the International Conference on Machine Learning (ICML\u201924)","author":"Guo Xingzhuo","year":"2024","unstructured":"Xingzhuo Guo, Junwei Pan, Ximei Wang, Baixu Chen, Jie Jiang, and Mingsheng Long. 2024. On the embedding collapse when scaling up recommendation models. In Proceedings of the International Conference on Machine Learning (ICML\u201924)."},{"key":"e_1_3_2_13_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR\u201923)","author":"Guo Xiaojun","year":"2023","unstructured":"Xiaojun Guo, Yifei Wang, Tianqi Du, and Yisen Wang. 2023. ContraNorm: A contrastive learning perspective on oversmoothing and beyond. In Proceedings of the 11th International Conference on Learning Representations (ICLR\u201923)."},{"key":"e_1_3_2_14_2","first-page":"5000","article-title":"Provable guarantees for self-supervised deep learning with spectral contrastive loss","author":"HaoChen Jeff Z.","year":"2021","unstructured":"Jeff Z. HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. 2021. Provable guarantees for self-supervised deep learning with spectral contrastive loss. In Advances in Neural Information Processing Systems (NeuriPS\u201921). ACM, New York, NY, 5000\u20135011.","journal-title":"Advances in Neural Information Processing Systems (NeuriPS\u201921)"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401063"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052569"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911489"},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)","author":"Hjelm R. Devon","year":"2019","unstructured":"R. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In Proceedings of the 7th International Conference on Learning Representations (ICLR\u201919)."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00946"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2396761.2396771"},{"key":"e_1_3_2_22_2","volume-title":"Proceedings of the 10th International Conference on Learning Representations (ICLR\u201922)","author":"Jing Li","year":"2022","unstructured":"Li Jing, Pascal Vincent, Yann LeCun, and Yuandong Tian. 2022. Understanding dimensional collapse in contrastive self-supervised learning. In Proceedings of the 10th International Conference on Learning Representations (ICLR\u201922)."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00035"},{"key":"e_1_3_2_24_2","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917)","author":"Kipf Thomas N.","year":"2017","unstructured":"Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In Proceedings of the 5th International Conference on Learning Representations (ICLR\u201917)."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2009.263"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11604"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623638"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313513"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482297"},{"key":"e_1_3_2_30_2","volume-title":"Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913)","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913)."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2011.134"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671789"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3532014"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557462"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-03098-8_5"},{"key":"e_1_3_2_36_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI\u201909)","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI\u201909). 452\u2013461."},{"key":"e_1_3_2_37_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920)","author":"Rong Yu","year":"2020","unstructured":"Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. 2020. DropEdge: Towards deep graph convolutional networks on node classification. In Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920)."},{"key":"e_1_3_2_38_2","first-page":"606","volume-title":"Proceedings of the 15th European Signal Processing Conference","author":"Roy Olivier","year":"2007","unstructured":"Olivier Roy and Martin Vetterli. 2007. The effective rank: A measure of effective dimensionality. In Proceedings of the 15th European Signal Processing Conference. 606\u2013610."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2014.2321121"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/371920.372071"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2740908.2742726"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357895"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539253"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219869"},{"key":"e_1_3_2_45_2","first-page":"9929","volume-title":"Proceedings of the International Conference on Machine Learning (ICML\u201920)","author":"Wang Tongzhou","year":"2020","unstructured":"Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the International Conference on Machine Learning (ICML\u201920). 9929\u20139939."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591663"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331267"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462862"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657724"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219890"},{"key":"e_1_3_2_51_2","first-page":"12310","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML\u201921)","author":"Zbontar Jure","year":"2021","unstructured":"Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and St\u00e9phane Deny. 2021. Barlow twins: Self-supervised learning via redundancy reduction. In Proceedings of the 38th International Conference on Machine Learning (ICML\u201921). 12310\u201312320."},{"key":"e_1_3_2_52_2","unstructured":"Yifei Zhang Hao Zhu Zixing Song Piotr Koniusz Irwin King et\u00a0al. 2023. Mitigating the popularity bias of graph collaborative filtering: A dimensional collapse perspective. In Proceedings of the 37th International Conference on Neural Information Processing Systems. ACM New York NY 67533\u201367550."},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3178876.3185994"}],"container-title":["ACM Transactions on Recommender Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3718488","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3718488","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:57:35Z","timestamp":1750298255000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3718488"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,10]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3718488"],"URL":"https:\/\/doi.org\/10.1145\/3718488","relation":{},"ISSN":["2770-6699"],"issn-type":[{"value":"2770-6699","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,10]]},"assertion":[{"value":"2024-10-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}