{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T15:31:39Z","timestamp":1772119899254,"version":"3.50.1"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,10,16]],"date-time":"2023-10-16T00:00:00Z","timestamp":1697414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62106243, U19B2026"],"award-info":[{"award-number":["62106243, U19B2026"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2024,1,31]]},"abstract":"<jats:p>\n            Nowadays, a large number of Knowledge Graph Completion (KGC) methods have been proposed by using embedding based manners, to overcome the incompleteness problem faced with knowledge graph (KG). One important recent innovation in Natural Language Processing (NLP) domain is the employ of deep neural models that make the most of pre-training, culminating in BERT, the most popular example of this line of approaches today. Recently, a series of new KGC methods introducing a pre-trained language model, such as KG-BERT, have been developed and released compelling performance. However, previous pre-training based KGC methods usually train the model by using simple training task and\n            <jats:italic>only<\/jats:italic>\n            utilize one-hop relational signals in KG, which leads that they\n            <jats:italic>cannot<\/jats:italic>\n            model high-order semantic contexts and multi-hop complex relatedness. To overcome this problem, this article presents a novel pre-training framework for KGC task, which especially consists of both one-hop relation level task (low-order) and multi-hop meta-graph level task (high-order). Hence, the proposed method can capture not only the elaborate sub-graph structure but also the subtle semantic information on the given KG. The empirical results show the efficiency of the proposed method on the widely used real-world datasets.\n          <\/jats:p>","DOI":"10.1145\/3617379","type":"journal-article","created":{"date-parts":[[2023,8,25]],"date-time":"2023-08-25T11:56:23Z","timestamp":1692964583000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["MEGA: Meta-Graph Augmented Pre-Training Model for Knowledge Graph Completion"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9414-4985","authenticated-orcid":false,"given":"Yashen","family":"Wang","sequence":"first","affiliation":[{"name":"National Engineering Laboratory for Risk Perception and Prevention (RPP), China Academy of Electronics and Information Technology of CETC; Key Laboratory of Cognition and Intelligence Technology (CIT), Information Science Academy of CETC"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0283-7790","authenticated-orcid":false,"given":"Xiaoye","family":"Ouyang","sequence":"additional","affiliation":[{"name":"National Engineering Laboratory for Risk Perception and Prevention (RPP), China Academy of Electronics and Information Technology of CETC"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3668-5748","authenticated-orcid":false,"given":"Dayu","family":"Guo","sequence":"additional","affiliation":[{"name":"CETC Academy of Electronics and Information Technology Group Co., Ltd."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7493-3167","authenticated-orcid":false,"given":"Xiaoling","family":"Zhu","sequence":"additional","affiliation":[{"name":"CETC Academy of Electronics and Information Technology Group Co., Ltd."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,10,16]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Ivana Balazevic Carl Allen and Timothy M. Hospedales. 2019. TuckER: Tensor factorization for knowledge graph completion. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . 5185\u20135194.","DOI":"10.18653\/v1\/D19-1522"},{"key":"e_1_3_2_3_2","first-page":"2787","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems","author":"Bordes Antoine","year":"2013","unstructured":"Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 2787\u20132795."},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","unstructured":"Chandrahas and Partha P. Talukdar. 2021. OKGIT: Open knowledge graph link prediction with implicit types. In Findings of the Association for Computational Linguistics: (ACL-IJCNLP\u201921) . 2546\u20132559.","DOI":"10.18653\/v1\/2021.findings-acl.225"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3113329"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11573"},{"key":"e_1_3_2_7_2","volume-title":"Proceedings of the NAACL-HLT","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the NAACL-HLT."},{"key":"e_1_3_2_8_2","first-page":"277","article-title":"Semantic proximity search on graphs with metagraph-based learning","author":"Fang Yuan","year":"2016","unstructured":"Yuan Fang, Wenqing Lin, Vincent Wenchen Zheng, Min Wu, Kevin Chen-Chuan Chang, and Xiaoli Li. 2016. Semantic proximity search on graphs with metagraph-based learning. In Proceedings of the 2016 IEEE 32nd International Conference on Data Engineering. 277\u2013288.","journal-title":"Proceedings of the 2016 IEEE 32nd International Conference on Data Engineering"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Jianlong Fu Jinqiao Wang Yong Rui Xin-Jing Wang Tao Mei and Hanqing Lu. 2015. Image tag refinement with view-dependent concept representations. IEEE Transactions on Circuits and Systems for Video Technology 25 8 (2015) 1409\u20131422.","DOI":"10.1109\/TCSVT.2014.2380211"},{"key":"e_1_3_2_10_2","unstructured":"Haipeng Gao Kun Yang Yuxue Yang Rufai Yusuf Zakari Jim Wilson Owusu and Ke Qin. 2021. QuatDE: Dynamic quaternion embedding for knowledge graph completion. arXiv:2105.09002. Retrieved from https:\/\/arxiv.org\/abs\/2105.09002"},{"key":"e_1_3_2_11_2","unstructured":"Lingbing Guo Weiqing Wang Zequn Sun Chenghao Liu and Wei Hu. 2020. Decentralized knowledge graph representation learning. arXiv:2010.08114. Retrieved from https:\/\/arxiv.org\/abs\/2010.08114"},{"key":"e_1_3_2_12_2","first-page":"9726","article-title":"Momentum contrast for unsupervised visual representation learning","author":"He Kaiming","year":"2020","unstructured":"Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross B. Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 9726\u20139735.","journal-title":"Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380027"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2787709"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939815"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1067"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Shaoxiong Ji Shirui Pan E. Cambria P. Marttinen and Philip S. Yu. 2021. A survey on knowledge graphs: Representation acquisition and applications. IEEE Transactions on Neural Networks and Learning Systems 33 2 (2021) 494\u2013514.","DOI":"10.1109\/TNNLS.2021.3070843"},{"key":"e_1_3_2_18_2","article-title":"Contrastive pre-training of GNNs on heterogeneous graphs","author":"Jiang Xunqiang","year":"2021","unstructured":"Xunqiang Jiang, Yuanfu Lu, Yuan Fang, and Chuan Shi. 2021. Contrastive pre-training of GNNs on heterogeneous graphs. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management.","journal-title":"Proceedings of the 30th ACM International Conference on Information & Knowledge Management"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","unstructured":"Sanjay Kamath Brigitte Grau and Yue Ma. 2019. How to pre-train your model? Comparison of different pre-training models for biomedical question answering. In Proceedings of Machine Learning and Knowledge Discovery in Databases International Workshops of (ECML-PKDD\u201919) . 646\u2013660.","DOI":"10.1007\/978-3-030-43887-6_58"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.coling-main.153"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9491"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"Xin Lv Yankai Lin Yixin Cao Lei Hou Juan-Zi Li Zhiyuan Liu Peng Li and Jie Zhou. 2022. Do pre-trained models benefit knowledge graph completion? A reliable evaluation and a reasonable approach. In Findings of the Association for Computational Linguistics: (ACL\u201922) . 3570\u20133581.","DOI":"10.18653\/v1\/2022.findings-acl.282"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-71249-9_43"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.205"},{"key":"e_1_3_2_25_2","unstructured":"Guanglin Niu Bo Li Yongfei Zhang Shiliang Pu and Jingyan Li. 2020. AutoETER: Automated entity type representation for knowledge graph embedding. In Findings of the Association for Computational Linguistics: (EMNLP\u201920) . 1172\u20131181."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10332"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-demo.28"},{"key":"e_1_3_2_28_2","unstructured":"Alec Radford and Karthik Narasimhan. 2018. Improving language understanding by generative pre-training. https:\/\/www.mikecaptain.com\/resources\/pdf\/GPT-1.pdf"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2016.2598561"},{"key":"e_1_3_2_30_2","first-page":"926","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems","author":"Socher Richard","year":"2013","unstructured":"Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the 26th International Conference on Neural Information Processing Systems. 926\u2013934."},{"key":"e_1_3_2_31_2","first-page":"2330","volume-title":"Proceedings of the 22nd International Joint Conference on Artificial Intelligence","author":"Song Yangqiu","year":"2011","unstructured":"Yangqiu Song, Haixun Wang, Zhongyuan Wang, Hongsong Li, and Weizhu Chen. 2011. Short text conceptualization using a probabilistic knowledgebase. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Vol. 3, 2330\u20132336."},{"key":"e_1_3_2_32_2","first-page":"2330","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence","author":"Song Yangqiu","year":"2011","unstructured":"Yangqiu Song, Haixun Wang, Zhongyuan Wang, Hongsong Li, and Weizhu Chen. 2011. Short text conceptualization using a probabilistic knowledgebase. In Proceedings of the International Joint Conference on Artificial Intelligence. 2330\u20132336."},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the 24th International Conference on Artificial Intelligence","author":"Song Yangqiu","year":"2015","unstructured":"Yangqiu Song, Shusen Wang, and Haixun Wang. 2015. Open domain short text conceptualization: A generative + descriptive modeling approach. In Proceedings of the 24th International Conference on Artificial Intelligence."},{"key":"e_1_3_2_34_2","first-page":"3820","volume-title":"Proceedings of the International Conference on Artificial Intelligence","author":"Song Yangqiu","year":"2015","unstructured":"Yangqiu Song, Shusen Wang, and Haixun Wang. 2015. Open domain short text conceptualization: A generative + descriptive modeling approach. In Proceedings of the International Conference on Artificial Intelligence. 3820\u20133826."},{"key":"e_1_3_2_35_2","unstructured":"Zhiqing Sun Zhihong Deng Jian-Yun Nie and Jian Tang. 2019. RotatE: Knowledge graph embedding by relational rotation in complex space. arXiv:1902.10197. Retrieved from https:\/\/arxiv.org\/abs\/1902.10197"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","unstructured":"Jinhui Tang Xiangbo Shu Zechao Li Yu-Gang Jiang and Qi Tian. 2019. Social anchor-unit graph regularized tensor completion for large-scale image retagging. IEEE Transactions on Pattern Analysis and Machine Intelligence 41 8 (2019) 2027\u20132034.","DOI":"10.1109\/TPAMI.2019.2906603"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Jinhui Tang Xiangbo Shu Guo-Jun Qi Zechao Li Meng Wang Shuicheng Yan and Ramesh C. Jain. 2017. Tri-clustered tensor completion for social-aware image tag refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence 39 8 (2017) 1662\u20131674.","DOI":"10.1109\/TPAMI.2016.2608882"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-4007"},{"key":"e_1_3_2_39_2","unstructured":"A\u00e4ron van den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv:1807.03748. Retrieved from https:\/\/arxiv.org\/abs\/1807.03748"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3442381.3450043"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2662067"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17217"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.295"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Q. Wang Zhendong Mao B. Wang and Li Guo. 2017. Knowledge graph embedding: A survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering 29 12 (2017) 2724\u20132743.","DOI":"10.1109\/TKDE.2017.2754499"},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","unstructured":"Xiao Wang Yuanfu Lu Chuan Shi Ruijia Wang Peng Cui and Shuai Mou. 2022. Dynamic heterogeneous information network embedding with meta-path based proximity. IEEE Transactions on Knowledge and Data Engineering 34 3 (2022) 1117\u20131132.","DOI":"10.1109\/TKDE.2020.2993870"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052710"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1048"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-26072-9_28"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-10983-6_32"},{"issue":"3","key":"e_1_3_2_50_2","first-page":"41:1\u201341:32","article-title":"Microblog retrieval based on concept-enhanced pre-training model","volume":"17","author":"Wang Yashen","year":"2022","unstructured":"Yashen Wang, Zhaoyu Wang, Huanhuan Zhang, and Zhirun Liu. 2022. Microblog retrieval based on concept-enhanced pre-training model. ACM Transactions on Knowledge Discovery from Data 17, 3 (2022), 41:1\u201341:32.","journal-title":"ACM Transactions on Knowledge Discovery from Data"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Yashen Wang He yan Huang and Chong Feng. 2021. Query expansion with local conceptual word embeddings in microblog retrieval. IEEE Transactions on Knowledge and Data Engineering 33 4 (2021) 1737\u20131749.","DOI":"10.1109\/TKDE.2019.2945764"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-31624-2_6"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","unstructured":"Yashen Wang Huanhuan Zhang Zhirun Liu and Qiang Zhou. 2021. Hierarchical concept-driven language model. ACM Transactions on Knowledge Discovery from Data 15 6 (2021) 1\u201322.","DOI":"10.1145\/3451167"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i1.8870"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832581.2832704"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832581.2832704"},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","unstructured":"Lei Wu Xiansheng Hua Nenghai Yu Wei-Ying Ma and Shipeng Li. 2012. Flickr distance: A relationship measure for visual concepts. IEEE Transactions on Pattern Analysis and Machine Intelligence 34 5 (2012) 863\u2013875.","DOI":"10.1109\/TPAMI.2011.195"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213891"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10952"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1219"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.106524"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v30i1.10329"},{"key":"e_1_3_2_63_2","unstructured":"Bishan Yang Wen tau Yih Xiaodong He Jianfeng Gao and Li Deng. 2014. Embedding entities and relations for learning and inference in knowledge bases. arXiv:1412.6575. Retrieved from https:\/\/arxiv.org\/abs\/1412.6575"},{"key":"e_1_3_2_64_2","unstructured":"Liang Yao Chengsheng Mao and Yuan Luo. 2019. KG-BERT: BERT for knowledge graph completion. arXiv:1909.03193. Retrieved from https:\/\/arxiv.org\/abs\/1909.03193"},{"key":"e_1_3_2_65_2","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Zhang Zhanqiu","year":"2019","unstructured":"Zhanqiu Zhang, Jianyu Cai, Yongdong Zhang, and Jie Wang. 2019. Learning hierarchy-aware knowledge graph embeddings for link prediction. In Proceedings of the AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098063"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617379","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3617379","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:45:54Z","timestamp":1750178754000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617379"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,16]]},"references-count":65,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,1,31]]}},"alternative-id":["10.1145\/3617379"],"URL":"https:\/\/doi.org\/10.1145\/3617379","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,16]]},"assertion":[{"value":"2022-12-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-28","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}