{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T02:40:41Z","timestamp":1778812841595,"version":"3.51.4"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,1,28]],"date-time":"2025-01-28T00:00:00Z","timestamp":1738022400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2025,3,31]]},"abstract":"<jats:p>\n            With the thriving of the pre-trained language model (PLM) widely verified in various NLP tasks, pioneer efforts attempt to explore the possible cooperation of the general textual information in PLM with the personalized behavioral information in user historical behavior sequences to enhance sequential recommendation (SR). However, despite the commonalities of input format and task goal, there are huge gaps between the behavioral and textual information, which obstruct thoroughly modeling SR as language modeling via PLM. To bridge the gap, we propose a novel unified pre-trained language model enhanced sequential recommendation (UPSR) that thoroughly transfers the next item prediction task to a text generation task, aiming to build a unified pre-trained recommendation model for multi-domain recommendation tasks. We formally design five key indicators, namely naturalness, domain consistency, informativeness, noise and ambiguity, and text length, to guide the\n            <jats:italic>\n              text\n              <jats:inline-formula content-type=\"math\/tex\">\n                <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\rightarrow\\)<\/jats:tex-math>\n              <\/jats:inline-formula>\n              item\n            <\/jats:italic>\n            adaptation (selecting appropriate text to form the item textual representation) and\n            <jats:italic>\n              behavior sequence\n              <jats:inline-formula content-type=\"math\/tex\">\n                <jats:tex-math notation=\"LaTeX\" version=\"MathJax\">\\(\\rightarrow\\)<\/jats:tex-math>\n              <\/jats:inline-formula>\n              text sequence\n            <\/jats:italic>\n            adaptation (transferring the sequence of item textual representations into a text sequence) differently for pre-training and fine-tuning stages, which are essential but under-explored by previous works. In experiments, we conduct extensive evaluations on seven datasets with both supervised and zero-shot settings and achieve the overall best performance. Comprehensive model analyses also provide valuable insights for behavior modeling via PLM, shedding light on large pre-trained recommendation models. The source codes will be released in the future.\n          <\/jats:p>","DOI":"10.1145\/3708883","type":"journal-article","created":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T11:50:36Z","timestamp":1734609036000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Thoroughly Modeling Multi-domain Pre-trained Recommendation as Language"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-6918-074X","authenticated-orcid":false,"given":"Zekai","family":"Qu","sequence":"first","affiliation":[{"name":"Institute of Information Engineering, China University of Geosciences Beijing, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3170-5647","authenticated-orcid":false,"given":"Ruobing","family":"Xie","sequence":"additional","affiliation":[{"name":"Tencent, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6039-0942","authenticated-orcid":false,"given":"Chaojun","family":"Xiao","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Technology, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8276-3620","authenticated-orcid":false,"given":"Yuan","family":"Yao","sequence":"additional","affiliation":[{"name":"School of Computing, National University of Singapore, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7709-2543","authenticated-orcid":false,"given":"Zhiyuan","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9871-3038","authenticated-orcid":false,"given":"Fengzong","family":"Lian","sequence":"additional","affiliation":[{"name":"Tencent, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-5151-4222","authenticated-orcid":false,"given":"Zhanhui","family":"Kang","sequence":"additional","affiliation":[{"name":"Tencent, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5899-5165","authenticated-orcid":false,"given":"Jie","family":"Zhou","sequence":"additional","affiliation":[{"name":"Tencent, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,1,28]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Keqin Bao Jizhi Zhang Yang Zhang Wenjie Wang Fuli Feng and Xiangnan He. 2023. TALLRec: An effective and efficient tuning framework to align large language model with recommendation. arXiv:2305.00447. Retrieved from https:\/\/arxiv.org\/abs\/2305.00447","DOI":"10.1145\/3604915.3608857"},{"key":"e_1_3_1_3_2","unstructured":"Iz Beltagy Matthew E. Peters and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. Retrieved from https:\/\/arxiv.org\/abs\/2004.05150"},{"key":"e_1_3_1_4_2","first-page":"1877","volume-title":"Advances in Neural Information Processing Systems","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems, Vol. 33, 1877\u20131901."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557262"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512090"},{"key":"e_1_3_1_7_2","unstructured":"Kyunghyun Cho Bart Van Merri\u00ebnboer Caglar Gulcehre Dzmitry Bahdanau Fethi Bougares Holger Schwenk and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078. Retrieved from https:\/\/arxiv.org\/abs\/1406.1078"},{"key":"e_1_3_1_8_2","unstructured":"Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay William Fedus Yunxuan Li Xuezhi Wang Mostafa Dehghani Siddhartha Brahma et al. 2022. Scaling instruction-finetuned language models. arXiv:2210.11416. Retrieved from https:\/\/arxiv.org\/abs\/2210.11416"},{"key":"e_1_3_1_9_2","unstructured":"Zeyu Cui Jianxin Ma Chang Zhou Jingren Zhou and Hongxia Yang. 2022. M6-rec: Generative pretrained language models are open-ended recommender systems. arXiv:2205.08084. Retrieved from https:\/\/arxiv.org\/abs\/2205.08084"},{"key":"e_1_3_1_10_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_1_11_2","unstructured":"Jiayu Ding Shuming Ma Li Dong Xingxing Zhang Shaohan Huang Wenhui Wang and Furu Wei. 2023. Longnet: Scaling transformers to 1 000 000 000 tokens. arXiv:2307.02486. Retrieved from https:\/\/arxiv.org\/abs\/2307.02486"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109877"},{"key":"e_1_3_1_13_2","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929. Retrieved from https:\/\/arxiv.org\/abs\/2010.11929"},{"key":"e_1_3_1_14_2","unstructured":"Yunfan Gao Tao Sheng Youlin Xiang Yun Xiong Haofen Wang and Jiawei Zhang. 2023. Chat-rec: Towards interactive and explainable llms-augmented recommender system. arXiv:2303.14524. Retrieved from https:\/\/arxiv.org\/abs\/arXiv:2303.14524"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3523227.3546767"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Shijie Geng Juntao Tan Shuchang Liu Zuohui Fu and Yongfeng Zhang. 2023. VIP5: Towards multimodal foundation models for recommendation. arXiv:2305.14302. Retrieved from https:\/\/arxiv.org\/abs\/2305.14302","DOI":"10.18653\/v1\/2023.findings-emnlp.644"},{"key":"e_1_3_1_17_2","first-page":"2782","volume-title":"Proceedings of the 25th International Joint Conference on Artificial Intelligence","author":"Gong Yuyun","year":"2016","unstructured":"Yuyun Gong and Qi Zhang. 2016. Hashtag recommendation using attention-based convolutional neural network. In Proceedings of the 25th International Joint Conference on Artificial Intelligence, 2782\u20132788."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467176"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3109859.3109882"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2016.0030"},{"key":"e_1_3_1_21_2","unstructured":"Bal\u00e1zs Hidasi Alexandros Karatzoglou Linas Baltrunas and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv:1511.06939. Retrieved from https:\/\/arxiv.org\/abs\/1511.06939"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539381"},{"key":"e_1_3_1_23_2","unstructured":"Yupeng Hou Junjie Zhang Zihan Lin Hongyu Lu Ruobing Xie Julian McAuley and Wayne Xin Zhao. 2023. Large language models are zero-shot rankers for recommender systems. arXiv:2305.08845. Retrieved from https:\/\/arxiv.org\/abs\/2305.08845"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210017"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2018.00035"},{"key":"e_1_3_1_26_2","unstructured":"Wang-Cheng Kang Jianmo Ni Nikhil Mehta Maheswaran Sathiamoorthy Lichan Hong Ed Chi and Derek Zhiyuan Cheng. 2023. Do LLMs understand user preferences? Evaluating LLMs on user rating prediction. arXiv:2305.06474. Retrieved from https:\/\/arxiv.org\/abs\/2305.06474"},{"key":"e_1_3_1_27_2","unstructured":"Jiacheng Li Ming Wang Jin Li Jinmiao Fu Xin Shen Jingbo Shang and Julian McAuley. 2023. Text is all you need: Learning language representations for sequential recommendation. arXiv:2305.13731. Retrieved from https:\/\/arxiv.org\/abs\/2305.13731"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i5.16549"},{"key":"e_1_3_1_29_2","unstructured":"Junling Liu Chao Liu Renjie Lv Kang Zhou and Yan Zhang. 2023. Is ChatGPT a good recommender? A preliminary study. arXiv:2304.10149. Retrieved from https:\/\/arxiv.org\/abs\/2304.10149"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331200"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2010-343"},{"key":"e_1_3_1_33_2","first-page":"16","volume-title":"Proceedings of the 28th China Conference on Information Retrieval (CCIR \u201922)","author":"Mu Shanlei","year":"2023","unstructured":"Shanlei Mu, Yupeng Hou, Wayne Xin Zhao, Yaliang Li, and Bolin Ding. 2023. ID-agnostic user behavior pre-training for sequential recommendation. In Proceedings of the 28th China Conference on Information Retrieval (CCIR \u201922). Springer, 16\u201327."},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1018"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3488560.3498433"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772773"},{"key":"e_1_3_1_38_2","doi-asserted-by":"crossref","unstructured":"Rico Sennrich Barry Haddow and Alexandra Birch. 2015. Neural machine translation of rare words with subword units. arXiv:1508.07909. Retrieved from https:\/\/arxiv.org\/abs\/1508.07909","DOI":"10.18653\/v1\/P16-1162"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357895"},{"key":"e_1_3_1_40_2","unstructured":"Juntao Tan Shuyuan Xu Wenyue Hua Yingqiang Ge Zelong Li and Yongfeng Zhang. 2024. Towards LLM-recsys alignment with textual ID learning. arXiv:2403.19021. Retrieved from https:\/\/arxiv.org\/abs\/2403.19021"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159656"},{"key":"e_1_3_1_42_2","first-page":"6000","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, Vol. 30, 6000\u20136010."},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-97-7235-3_13"},{"key":"e_1_3_1_44_2","unstructured":"Lei Wang and Ee-Peng Lim. 2023. Zero-shot next-item recommendation using large pretrained language models. arXiv:2304.03153. Retrieved from https:\/\/arxiv.org\/abs\/2304.03153"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330989"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645671"},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463069"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.3301346"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591949"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531913"},{"key":"e_1_3_1_52_2","unstructured":"Yiqing Wu Ruobing Xie Yongchun Zhu Fuzhen Zhuang Xu Zhang Leyu Lin and Qing He. 2022. Personalized prompts for sequential recommendation. arXiv:2205.09666. Retrieved from https:\/\/arxiv.org\/abs\/2205.09666"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.aiopen.2023.08.008"},{"key":"e_1_3_1_54_2","first-page":"1259","volume-title":"Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE \u201922).","author":"Xie Xu","year":"2022","unstructured":"Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Jiandong Zhang, Bolin Ding, and Bin Cui. 2022. Contrastive learning for sequential recommendation. In Proceedings of the 2022 IEEE 38th International Conference on Data Engineering (ICDE \u201922). IEEE, 1259\u20131273."},{"key":"e_1_3_1_55_2","first-page":"580","volume-title":"Neurocomputing","author":"Xu Chengfeng","year":"2021","unstructured":"Chengfeng Xu, Jian Feng, Pengpeng Zhao, Fuzhen Zhuang, Deqing Wang, Yanchi Liu, and Victor S. Sheng. 2021. Long-and short-term self-attention network for sequential recommendation. Neurocomputing 423 (2021), 580\u2013589."},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401156"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462884"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463060"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdata.2021.602071"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939673"},{"key":"e_1_3_1_61_2","unstructured":"Junjie Zhang Yupeng Hou Ruobing Xie Wenqi Sun Julian McAuley Wayne Xin Zhao Leyu Lin and Ji-Rong Wen. 2023. AgentCF: Collaborative learning with autonomous language agents for recommender systems. arXiv:2310.09233. Retrieved from https:\/\/arxiv.org\/abs\/2310.09233"},{"key":"e_1_3_1_62_2","doi-asserted-by":"crossref","unstructured":"Junjie Zhang Ruobing Xie Yupeng Hou Wayne Xin Zhao Leyu Lin and Ji-Rong Wen. 2023. Recommendation as instruction following: A large language model empowered recommendation approach. arXiv:2305.07001. Retrieved from https:\/\/arxiv.org\/abs\/2305.07001","DOI":"10.1145\/3708882"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.5555\/3367471.3367642"},{"key":"e_1_3_1_64_2","unstructured":"Yuhui Zhang Hao Ding Zeren Shui Yifei Ma James Zou Anoop Deoras and Hao Wang. 2021. Language models as recommender systems: Evaluations and limitations."},{"key":"e_1_3_1_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3018661.3018665"},{"key":"e_1_3_1_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411954"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3708883","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3708883","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:54Z","timestamp":1750295874000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3708883"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,28]]},"references-count":65,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,3,31]]}},"alternative-id":["10.1145\/3708883"],"URL":"https:\/\/doi.org\/10.1145\/3708883","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,28]]},"assertion":[{"value":"2023-11-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-01-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}