{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T09:37:47Z","timestamp":1774690667037,"version":"3.50.1"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T00:00:00Z","timestamp":1713139200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022ZD0116300"],"award-info":[{"award-number":["2022ZD0116300"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Science Foundation of China","doi-asserted-by":"crossref","award":["62106249"],"award-info":[{"award-number":["62106249"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Asian Low-Resour. Lang. Inf. Process."],"published-print":{"date-parts":[[2024,4,30]]},"abstract":"<jats:p>\n            Recent years have witnessed a surge of academic interest in knowledge-enhanced pre-trained language models (PLMs) that incorporate factual knowledge to enhance knowledge-driven applications. Nevertheless, existing studies primarily focus on shallow, static, and separately pre-trained entity embeddings, with few delving into the potential of deep contextualized knowledge representation for knowledge incorporation. Consequently, the performance gains of such models remain limited. In this article, we introduce a simple yet effective knowledge-enhanced model,\n            <jats:sc>College<\/jats:sc>\n            (\n            <jats:bold>Co<\/jats:bold>\n            ntrastive\n            <jats:bold>L<\/jats:bold>\n            anguage-Know\n            <jats:bold>le<\/jats:bold>\n            dge\n            <jats:bold>G<\/jats:bold>\n            raph Pr\n            <jats:bold>e<\/jats:bold>\n            -training), which leverages contrastive learning to incorporate factual knowledge into PLMs. This approach maintains the knowledge in its original graph structure to provide the most available information and circumvents the issue of heterogeneous embedding fusion. Experimental results demonstrate that our approach achieves more effective results on several knowledge-intensive tasks compared to previous state-of-the-art methods. Our code and trained models are available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/github.com\/Stacy027\/COLLEGE\">https:\/\/github.com\/Stacy027\/COLLEGE<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3644820","type":"journal-article","created":{"date-parts":[[2024,2,9]],"date-time":"2024-02-09T11:54:22Z","timestamp":1707479662000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Contrastive Language-knowledge Graph Pre-training"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0841-266X","authenticated-orcid":false,"given":"Xiaowei","family":"Yuan","sequence":"first","affiliation":[{"name":"The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, CAS, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing Academy of Artificial Intelligence, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6083-8433","authenticated-orcid":false,"given":"Kang","family":"Liu","sequence":"additional","affiliation":[{"name":"The Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, CAS, School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing Academy of Artificial Intelligence, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7530-6125","authenticated-orcid":false,"given":"Yequan","family":"Wang","sequence":"additional","affiliation":[{"name":"Beijing Academy of Artificial Intelligence, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,4,15]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"2787","volume-title":"27th Annual Conference on Neural Information Processing Systems","author":"Bordes Antoine","year":"2013","unstructured":"Antoine Bordes, Nicolas Usunier, Alberto Garc\u00eda-Dur\u00e1n, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In 27th Annual Conference on Neural Information Processing Systems. 2787\u20132795."},{"key":"e_1_3_3_3_2","volume-title":"Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners. In Annual Conference on Neural Information Processing Systems (NeurIPS\u201920)."},{"key":"e_1_3_3_4_2","first-page":"3576","volume-title":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201921)","author":"Chi Zewen","year":"2021","unstructured":"Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, and Ming Zhou. 2021. InfoXLM: An information-theoretic framework for cross-lingual language model pre-training. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201921). 3576\u20133588."},{"key":"e_1_3_3_5_2","first-page":"87","volume-title":"56th Annual Meeting of the Association for Computational Linguistics (ACL\u201918)","author":"Choi Eunsol","year":"2018","unstructured":"Eunsol Choi, Omer Levy, Yejin Choi, and Luke Zettlemoyer. 2018. Ultra-fine entity typing. In 56th Annual Meeting of the Association for Computational Linguistics (ACL\u201918). 87\u201396."},{"key":"e_1_3_3_6_2","first-page":"3079","volume-title":"Annual Conference on Neural Information Processing Systems","author":"Dai Andrew M.","year":"2015","unstructured":"Andrew M. Dai and Quoc V. Le. 2015. Semi-supervised sequence learning. In Annual Conference on Neural Information Processing Systems. 3079\u20133087."},{"key":"e_1_3_3_7_2","first-page":"4171","volume-title":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201919)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201919). 4171\u20134186."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF01386390"},{"key":"e_1_3_3_9_2","article-title":"CERT: Contrastive self-supervised learning for language understanding","volume":"2005","author":"Fang Hongchao","year":"2020","unstructured":"Hongchao Fang and Pengtao Xie. 2020. CERT: Contrastive self-supervised learning for language understanding. CoRR abs\/2005.12766 (2020).","journal-title":"CoRR"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871689"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1749-6632.1976.tb25467.x"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1514"},{"key":"e_1_3_3_13_2","first-page":"2281","volume-title":"Findings of the Association for Computational Linguistics (EMNLP\u201920)","author":"He Bin","year":"2020","unstructured":"Bin He, Di Zhou, Jinghui Xiao, Xin Jiang, Qun Liu, Nicholas Jing Yuan, and Tong Xu. 2020. Integrating graph contextualized knowledge into pre-trained language models. In Findings of the Association for Computational Linguistics (EMNLP\u201920), Vol. EMNLP 2020. 2281\u20132290."},{"key":"e_1_3_3_14_2","first-page":"328","volume-title":"56th Annual Meeting of the Association for Computational Linguistics (ACL\u201918)","author":"Howard Jeremy","year":"2018","unstructured":"Jeremy Howard and Sebastian Ruder. 2018. Universal language model fine-tuning for text classification. In 56th Annual Meeting of the Association for Computational Linguistics (ACL\u201918). 328\u2013339."},{"key":"e_1_3_3_15_2","doi-asserted-by":"crossref","first-page":"4859","DOI":"10.18653\/v1\/2020.acl-main.439","volume-title":"58th Annual Meeting of the Association for Computational Linguistics (ACL\u201920)","author":"Iter Dan","year":"2020","unstructured":"Dan Iter, Kelvin Guu, Larry Lansing, and Dan Jurafsky. 2020. Pretraining with contrastive sentence objectives improves discourse performance of language models. In 58th Annual Meeting of the Association for Computational Linguistics (ACL\u201920). 4859\u20134870."},{"key":"e_1_3_3_16_2","first-page":"5962","volume-title":"57th Conference of the Association for Computational Linguistics (ACL\u201919)","author":"IV Robert L. Logan","year":"2019","unstructured":"Robert L. Logan IV, Nelson F. Liu, Matthew E. Peters, Matt Gardner, and Sameer Singh. 2019. Barack\u2019s wife hillary: Using knowledge graphs for fact-aware language modeling. In 57th Conference of the Association for Computational Linguistics (ACL\u201919). 5962\u20135971."},{"key":"e_1_3_3_17_2","first-page":"251:1\u2013251:43","article-title":"Atlas: Few-shot learning with retrieval augmented language models","volume":"24","author":"Izacard Gautier","year":"2023","unstructured":"Gautier Izacard, Patrick S. H. Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2023. Atlas: Few-shot learning with retrieval augmented language models. J. Mach. Learn. Res. 24 (2023), 251:1\u2013251:43. Retrieved from http:\/\/jmlr.org\/papers\/v24\/23-0037.html","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.418"},{"key":"e_1_3_3_19_2","article-title":"RealTime QA: What\u2019s the answer right now?","volume":"2207","author":"Kasai Jungo","year":"2022","unstructured":"Jungo Kasai, Keisuke Sakaguchi, Yoichi Takahashi, Ronan Le Bras, Akari Asai, Xinyan Yu, Dragomir R. Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. 2022. RealTime QA: What\u2019s the answer right now? CoRR abs\/2207.13332 (2022).","journal-title":"CoRR"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/D19-1282"},{"key":"e_1_3_3_21_2","first-page":"2901","volume-title":"34th AAAI Conference on Artificial Intelligence (AAAI\u201d20), 32nd Innovative Applications of Artificial Intelligence Conference (IAAI\u201920), 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201920)","author":"Liu Weijie","year":"2020","unstructured":"Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Qi Ju, Haotang Deng, and Ping Wang. 2020. K-BERT: Enabling language representation with knowledge graph. In 34th AAAI Conference on Artificial Intelligence (AAAI\u201d20), 32nd Innovative Applications of Artificial Intelligence Conference (IAAI\u201920), 10th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI\u201920). 2901\u20132908."},{"key":"e_1_3_3_22_2","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","volume":"1907","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR abs\/1907.11692 (2019).","journal-title":"CoRR"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.ACL-LONG.546"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/D18-1260"},{"key":"e_1_3_3_25_2","first-page":"3111","volume-title":"27th Annual Conference on Neural Information Processing Systems","author":"Mikolov Tom\u00e1s","year":"2013","unstructured":"Tom\u00e1s Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013. Distributed representations of words and phrases and their compositionality. In 27th Annual Conference on Neural Information Processing Systems. 3111\u20133119."},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.FINDINGS-ACL.132"},{"key":"e_1_3_3_27_2","article-title":"Society of Mind","author":"Minsky Marvin","year":"1988","unstructured":"Marvin Minsky. 1988. Society of Mind. Simon and Schuster.","journal-title":"Simon and Schuster"},{"key":"e_1_3_3_28_2","volume-title":"Annual Conference on Neural Information Processing Systems (NeurIPS\u201922)","author":"Pan Xuran","year":"2022","unstructured":"Xuran Pan, Tianzhu Ye, Dongchen Han, Shiji Song, and Gao Huang. 2022. Contrastive language-image pre-training with knowledge graphs. In Annual Conference on Neural Information Processing Systems (NeurIPS\u201922)."},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_3_30_2","doi-asserted-by":"crossref","first-page":"43","DOI":"10.18653\/v1\/D19-1005","volume-title":"Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919)","author":"Peters Matthew E.","year":"2019","unstructured":"Matthew E. Peters, Mark Neumann, Robert L. Logan IV, Roy Schwartz, Vidur Joshi, Sameer Singh, and Noah A. Smith. 2019. Knowledge enhanced contextual word representations. In Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919). 43\u201354."},{"key":"e_1_3_3_31_2","first-page":"2227","volume-title":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201918)","author":"Peters Matthew E.","year":"2018","unstructured":"Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201918). 2227\u20132237."},{"key":"e_1_3_3_32_2","doi-asserted-by":"crossref","first-page":"2463","DOI":"10.18653\/v1\/D19-1250","volume-title":"Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919)","author":"Petroni Fabio","year":"2019","unstructured":"Fabio Petroni, Tim Rockt\u00e4schel, Sebastian Riedel, Patrick S. H. Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander H. Miller. 2019. Language models as knowledge bases? In Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919). 2463\u20132473."},{"key":"e_1_3_3_33_2","first-page":"803","volume-title":"Findings of the Association for Computational Linguistics (EMNLP\u201920)","author":"P\u00f6rner Nina","year":"2020","unstructured":"Nina P\u00f6rner, Ulli Waltinger, and Hinrich Sch\u00fctze. 2020. E-BERT: Efficient-yet-effective entity embeddings for BERT. In Findings of the Association for Computational Linguistics (EMNLP\u201920), Vol. EMNLP 2020. 803\u2013818."},{"key":"e_1_3_3_34_2","first-page":"8748","volume-title":"38th International Conference on Machine Learning (ICML\u201921)","volume":"139","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In 38th International Conference on Machine Learning (ICML\u201921), Vol. 139. 8748\u20138763."},{"key":"e_1_3_3_35_2","article-title":"Improving Language Understanding by Generative Pre-training","author":"Radford Alec","year":"2018","unstructured":"Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-training. Technical Report. OpenAI..","journal-title":"Technical Report. OpenAI."},{"issue":"8","key":"e_1_3_3_36_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_3_37_2","first-page":"5628","volume-title":"36th International Conference on Machine Learning (ICML\u201919)","author":"Saunshi Nikunj","year":"2019","unstructured":"Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. 2019. A theoretical analysis of contrastive unsupervised representation learning. In 36th International Conference on Machine Learning (ICML\u201919). 5628\u20135637."},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_3_39_2","first-page":"5149","volume-title":"IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201912)","author":"Schuster Mike","year":"2012","unstructured":"Mike Schuster and Kaisuke Nakajima. 2012. Japanese and Korean voice search. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201912). 5149\u20135152."},{"key":"e_1_3_3_40_2","volume-title":"54th Annual Meeting of the Association for Computational Linguistics (ACL\u201916)","author":"Sennrich Rico","year":"2016","unstructured":"Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In 54th Annual Meeting of the Association for Computational Linguistics (ACL\u201916)."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1609\/AAAI.V31I1.11164"},{"key":"e_1_3_3_42_2","first-page":"3660","volume-title":"28th International Conference on Computational Linguistics (COLING\u201920)","author":"Sun Tianxiang","year":"2020","unstructured":"Tianxiang Sun, Yunfan Shao, Xipeng Qiu, Qipeng Guo, Yaru Hu, Xuanjing Huang, and Zheng Zhang. 2020. CoLAKE: Contextualized language and knowledge embedding. In 28th International Conference on Computational Linguistics (COLING\u201920). 3660\u20133670."},{"key":"e_1_3_3_43_2","volume-title":"7th International Conference on Learning Representations (ICLR\u201919)","author":"Sun Zhiqing","year":"2019","unstructured":"Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations (ICLR\u201919)."},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/N19-1421"},{"key":"e_1_3_3_45_2","first-page":"2071","volume-title":"33rd International Conference on Machine Learning (ICML16)","volume":"48","author":"Trouillon Th\u00e9o","year":"2016","unstructured":"Th\u00e9o Trouillon, Johannes Welbl, Sebastian Riedel, \u00c9ric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In 33rd International Conference on Machine Learning (ICML16), Vol. 48. 2071\u20132080."},{"key":"e_1_3_3_46_2","first-page":"384","volume-title":"48th Annual Meeting of the Association for Computational Linguistics (ACL\u201910)","author":"Turian Joseph P.","year":"2010","unstructured":"Joseph P. Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In 48th Annual Meeting of the Association for Computational Linguistics (ACL\u201910). 384\u2013394."},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295349"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629489"},{"key":"e_1_3_3_49_2","volume-title":"7th International Conference on Learning Representations (ICLR\u201919)","author":"Wang Alex","year":"2019","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In 7th International Conference on Learning Representations (ICLR\u201919)."},{"key":"e_1_3_3_50_2","first-page":"1405","volume-title":"Findings of the Association for Computational Linguistics (ACL\/IJCNLP\u201921)","author":"Wang Ruize","year":"2021","unstructured":"Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, and Ming Zhou. 2021. K-Adapter: Infusing knowledge into pre-trained models with adapters. In Findings of the Association for Computational Linguistics (ACL\/IJCNLP\u201921). 1405\u20131418."},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00360"},{"key":"e_1_3_3_52_2","first-page":"2659","volume-title":"30th AAAI Conference on Artificial Intelligence","author":"Xie Ruobing","year":"2016","unstructured":"Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation learning of knowledge graphs with entity descriptions. In 30th AAAI Conference on Artificial Intelligence. 2659\u20132665."},{"key":"e_1_3_3_53_2","volume-title":"8th International Conference on Learning Representations (ICLR\u201920)","author":"Xiong Wenhan","year":"2020","unstructured":"Wenhan Xiong, Jingfei Du, William Yang Wang, and Veselin Stoyanov. 2020. Pretrained encyclopedia: Weakly supervised knowledge-pretrained language model. In 8th International Conference on Learning Representations (ICLR\u201920). Retrieved from https:\/\/openreview.net\/forum?id=BJlzm64tDH"},{"key":"e_1_3_3_54_2","doi-asserted-by":"crossref","first-page":"250","DOI":"10.18653\/v1\/K16-1025","volume-title":"20th SIGNLL Conference on Computational Natural Language Learning (CoNLL\u201916)","author":"Yamada Ikuya","year":"2016","unstructured":"Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, and Yoshiyasu Takefuji. 2016. Joint learning of the embedding of words and entities for named entity disambiguation. In 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL\u201916). 250\u2013259."},{"key":"e_1_3_3_55_2","volume-title":"3rd International Conference on Learning Representations (ICLR\u201915)","author":"Yang Bishan","year":"2015","unstructured":"Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Representations (ICLR\u201915)."},{"key":"e_1_3_3_56_2","first-page":"5754","volume-title":"Annual Conference on Neural Information Processing Systems (NeurIPS\u201919)","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized autoregressive pretraining for language understanding. In Annual Conference on Neural Information Processing Systems (NeurIPS\u201919). 5754\u20135764."},{"key":"e_1_3_3_57_2","article-title":"KG-BERT: BERT for knowledge graph completion","volume":"1909","author":"Yao Liang","year":"2019","unstructured":"Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. KG-BERT: BERT for knowledge graph completion. CoRR abs\/1909.03193 (2019).","journal-title":"CoRR"},{"key":"e_1_3_3_58_2","unstructured":"Michihiro Yasunaga Antoine Bosselut Hongyu Ren Xikun Zhang Christopher D. Manning Percy S. Liang and Jure Leskovec. 2022. Deep bidirectional language-knowledge graph pretraining. In Advances in Neural Information Processing Systems S. Koyejo S. Mohamed A. Agarwal D. Belgrave K. Cho and A. Oh (Eds.). Vol. 35. Curran Associates Inc. 37309\u201337323."},{"key":"e_1_3_3_59_2","first-page":"535","volume-title":"Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201921)","author":"Yasunaga Michihiro","year":"2021","unstructured":"Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT\u201921). 535\u2013546."},{"key":"e_1_3_3_60_2","volume-title":"10th International Conference on Learning Representations (ICLR\u201922)","author":"Zhang Xikun","year":"2022","unstructured":"Xikun Zhang, Antoine Bosselut, Michihiro Yasunaga, Hongyu Ren, Percy Liang, Christopher D. Manning, and Jure Leskovec. 2022. GreaseLM: Graph REASoning enhanced language models. In 10th International Conference on Learning Representations (ICLR\u201922). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=41e9o6cQPj"},{"key":"e_1_3_3_61_2","first-page":"1441","volume-title":"57th Conference of the Association for Computational Linguistics (ACL\u201919)","author":"Zhang Zhengyan","year":"2019","unstructured":"Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019. ERNIE: Enhanced language representation with informative entities. In 57th Conference of the Association for Computational Linguistics (ACL\u201919). 1441\u20131451."},{"key":"e_1_3_3_62_2","article-title":"A survey of large language models","volume":"2303","author":"Zhao Wayne Xin","year":"2023","unstructured":"Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A survey of large language models. CoRR abs\/2303.18223 (2023).","journal-title":"CoRR"}],"container-title":["ACM Transactions on Asian and Low-Resource Language Information Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3644820","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3644820","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:27Z","timestamp":1750291407000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3644820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,15]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,4,30]]}},"alternative-id":["10.1145\/3644820"],"URL":"https:\/\/doi.org\/10.1145\/3644820","relation":{},"ISSN":["2375-4699","2375-4702"],"issn-type":[{"value":"2375-4699","type":"print"},{"value":"2375-4702","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,15]]},"assertion":[{"value":"2023-05-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-25","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-15","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}