{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:13:06Z","timestamp":1784178786585,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":41,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,5,31]],"date-time":"2020-05-31T00:00:00Z","timestamp":1590883200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100007515","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1719054, IIS-1937143"],"award-info":[{"award-number":["IIS-1719054, IIS-1937143"]}],"id":[{"id":"10.13039\/100007515","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,6,11]]},"DOI":"10.1145\/3318464.3380599","type":"proceedings-article","created":{"date-parts":[[2020,5,29]],"date-time":"2020-05-29T17:12:33Z","timestamp":1590772353000},"page":"1995-2010","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["Realistic Re-evaluation of Knowledge Graph Completion Methods: An Experimental Study"],"prefix":"10.1145","author":[{"given":"Farahnaz","family":"Akrami","sequence":"first","affiliation":[{"name":"University of Texas at Arlington, Arlington, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohammed Samiul","family":"Saeef","sequence":"additional","affiliation":[{"name":"University of Texas at Arlington, Arlington, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qingheng","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Hu","sequence":"additional","affiliation":[{"name":"Nanjing University, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengkai","family":"Li","sequence":"additional","affiliation":[{"name":"University of Texas at Arlington, Arlington, TX, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,5,31]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3269266"},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1522"},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376746"},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 2787--2795","author":"Bordes Antoine","year":"2013","unstructured":"Antoine Bordes , Nicolas Usunier , Alberto Garcia-Dur\u00e1n , Jason Weston , and Oksana Yakhnenko . 2013 . Translating embeddings for modeling multi-relational data . In Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 2787--2795 . Antoine Bordes, Nicolas Usunier, Alberto Garcia-Dur\u00e1n, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 2787--2795."},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2807452"},{"key":"e_1_3_2_2_7_1","volume-title":"Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI). 1306--1313","author":"Carlson Andrew","year":"2010","unstructured":"Andrew Carlson , Justin Betteridge , Bryan Kisiel , Burr Settles , Estevam R Hruschka Jr , and Tom M Mitchell . 2010 . Toward an architecture for never-ending language learning . In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI). 1306--1313 . Andrew Carlson, Justin Betteridge, Bryan Kisiel, Burr Settles, Estevam R Hruschka Jr, and Tom M Mitchell. 2010. Toward an architecture for never-ending language learning. In Proceedings of the 24th AAAI Conference on Artificial Intelligence (AAAI). 1306--1313."},{"key":"e_1_3_2_2_8_1","volume-title":"Proceedings of the 6th International Conference on Learning Representations (ICLR) .","author":"Das Rajarshi","year":"2018","unstructured":"Rajarshi Das , Shehzaad Dhuliawala , Manzil Zaheer , Luke Vilnis , Ishan Durugkar , Akshay Krishnamurthy , Alex Smola , and Andrew McCallum . 2018 . Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning . In Proceedings of the 6th International Conference on Learning Representations (ICLR) . Rajarshi Das, Shehzaad Dhuliawala, Manzil Zaheer, Luke Vilnis, Ishan Durugkar, Akshay Krishnamurthy, Alex Smola, and Andrew McCallum. 2018. Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning. In Proceedings of the 6th International Conference on Learning Representations (ICLR) ."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11573"},{"key":"e_1_3_2_2_10_1","first-page":"109","article-title":"Knowledge graph based search system","volume":"13","author":"Eder Jeffrey Scott","year":"2012","unstructured":"Jeffrey Scott Eder . 2012 . Knowledge graph based search system . US Patent App. 13\/404 , 109 . Jeffrey Scott Eder. 2012. Knowledge graph based search system. US Patent App. 13\/404,109.","journal-title":"US Patent App."},{"key":"e_1_3_2_2_11_1","volume-title":"Semantic Search for Novel Information","author":"Farber Michael","unstructured":"Michael Farber . 2017. Semantic Search for Novel Information . IOS Press , Amsterdam, The Netherlands, The Netherlands. Michael Farber. 2017. Semantic Search for Novel Information .IOS Press, Amsterdam, The Netherlands, The Netherlands."},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-015-0394-1"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2488388.2488425"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-2024"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1067"},{"key":"e_1_3_2_2_16_1","volume-title":"Relational retrieval using a combination of path-constrained random walks. Machine learning","author":"Lao Ni","year":"2010","unstructured":"Ni Lao and William W Cohen . 2010. Relational retrieval using a combination of path-constrained random walks. Machine learning , Vol. 81 , 1 ( 2010 ), 53--67. https:\/\/doi.org\/10.1007\/s10994-010--5205--8 Ni Lao and William W Cohen. 2010. Relational retrieval using a combination of path-constrained random walks. Machine learning, Vol. 81, 1 (2010), 53--67. https:\/\/doi.org\/10.1007\/s10994-010--5205--8"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1362"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1082"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9491"},{"key":"e_1_3_2_2_20_1","volume-title":"Conference on Innovative Data Systems Research (CIDR) .","author":"Mahdisoltani Farzaneh","year":"2015","unstructured":"Farzaneh Mahdisoltani , Joanna Biega , and Fabian M Suchanek . 2015 . Yago3: A knowledge base from multilingual wikipedias . In Conference on Innovative Data Systems Research (CIDR) . Farzaneh Mahdisoltani, Joanna Biega, and Fabian M Suchanek. 2015. Yago3: A knowledge base from multilingual wikipedias. In Conference on Innovative Data Systems Research (CIDR) ."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-00671-6_1"},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2483592"},{"key":"e_1_3_2_2_24_1","volume-title":"Proceedings of the 28th International Conference on Machine Learning (ICML). 809--816","author":"Nickel Maximilian","year":"2011","unstructured":"Maximilian Nickel , Volker Tresp , and Hans-Peter Kriegel . 2011 . A Three-Way Model for Collective Learning on Multi-Relational Data . In Proceedings of the 28th International Conference on Machine Learning (ICML). 809--816 . Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A Three-Way Model for Collective Learning on Multi-Relational Data. In Proceedings of the 28th International Conference on Machine Learning (ICML). 809--816."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2874809"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-017-05778-z"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10677"},{"key":"e_1_3_2_2_28_1","volume-title":"Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 926--934","author":"Socher Richard","year":"2013","unstructured":"Richard Socher , Danqi Chen , Christopher D Manning , and Andrew Ng . 2013 . Reasoning with neural tensor networks for knowledge base completion . In Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 926--934 . Richard Socher, Danqi Chen, Christopher D Manning, and Andrew Ng. 2013. Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the 26th International Conference on Neural Information Processing Systems (NeurIPS). 926--934."},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2008.06.001"},{"key":"e_1_3_2_2_30_1","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR). 926--934","author":"Sun Zhiqing","year":"2019","unstructured":"Zhiqing Sun , Zhi-Hong Deng , Jian-Yun Nie , and Jian Tang . 2019 . RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space . In Proceedings of the International Conference on Learning Representations (ICLR). 926--934 . Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. In Proceedings of the International Conference on Learning Representations (ICLR). 926--934."},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W15-4007"},{"key":"e_1_3_2_2_32_1","volume-title":"Proceedings of the 33rd International Conference on Machine Learning (ICML). 2071--2080","author":"Trouillon Th\u00e9o","year":"2016","unstructured":"Th\u00e9o Trouillon , Johannes Welbl , Sebastian Riedel , \u00c9ric Gaussier , and Guillaume Bouchard . 2016 . Complex embeddings for simple link prediction . In Proceedings of the 33rd International Conference on Machine Learning (ICML). 2071--2080 . Th\u00e9o Trouillon, Johannes Welbl, Sebastian Riedel, \u00c9ric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33rd International Conference on Machine Learning (ICML). 2071--2080."},{"key":"e_1_3_2_2_33_1","unstructured":"Ledyard R Tucker etal 1964. The extension of factor analysis to three-dimensional matrices. Contributions to mathematical psychology Vol. 110119 (1964).  Ledyard R Tucker et al. 1964. The extension of factor analysis to three-dimensional matrices. Contributions to mathematical psychology Vol. 110119 (1964)."},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629489"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W19-4313"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i1.8870"},{"key":"e_1_3_2_2_37_1","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1366--1371","author":"Weston Jason","year":"2013","unstructured":"Jason Weston , Antoine Bordes , Oksana Yakhnenko , and Nicolas Usunier . 2013 . Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction . In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1366--1371 . Jason Weston, Antoine Bordes, Oksana Yakhnenko, and Nicolas Usunier. 2013. Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1366--1371."},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1220"},{"key":"e_1_3_2_2_39_1","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR) .","author":"Yang Bishan","year":"2015","unstructured":"Bishan Yang , Wen-tau Yih, Xiaodong He , Jianfeng Gao , and Li Deng . 2015 . Embedding entities and relations for learning and inference in knowledge bases . In Proceedings of the International Conference on Learning Representations (ICLR) . Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the International Conference on Learning Representations (ICLR) ."},{"key":"e_1_3_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P14-1090"},{"key":"e_1_3_2_2_41_1","volume-title":"Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL+IJCNLP). 1321--1331","author":"Chang Ming-Wei","year":"2015","unstructured":"Wen-tau Yih, Ming-Wei Chang , Xiaodong He , and Jianfeng Gao . 2015 . Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base . In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL+IJCNLP). 1321--1331 . https:\/\/doi.org\/10.3115\/v1\/P15--1128 Wen-tau Yih, Ming-Wei Chang, Xiaodong He, and Jianfeng Gao. 2015. Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL+IJCNLP). 1321--1331. https:\/\/doi.org\/10.3115\/v1\/P15--1128"}],"event":{"name":"SIGMOD\/PODS '20: International Conference on Management of Data","location":"Portland OR USA","acronym":"SIGMOD\/PODS '20","sponsor":["SIGMOD ACM Special Interest Group on Management of Data"]},"container-title":["Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3318464.3380599","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3318464.3380599","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:38:23Z","timestamp":1750199903000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3318464.3380599"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,31]]},"references-count":41,"alternative-id":["10.1145\/3318464.3380599","10.1145\/3318464"],"URL":"https:\/\/doi.org\/10.1145\/3318464.3380599","relation":{},"subject":[],"published":{"date-parts":[[2020,5,31]]},"assertion":[{"value":"2020-05-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}