{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T15:11:27Z","timestamp":1777043487845,"version":"3.51.4"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100013209","name":"Hellenic Foundation for Research and Innovation","doi-asserted-by":"publisher","award":["969"],"award-info":[{"award-number":["969"]}],"id":[{"id":"10.13039\/501100013209","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Foundation for Research and Technology"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Min Knowl Disc"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In recent years, we have witnessed the proliferation of knowledge graphs (KG) in various domains, aiming to support applications like question answering, recommendations, etc. A frequent task when integrating knowledge from different KGs is to find which subgraphs refer to the same real-world entity, a task largely known as the Entity Alignment. Recently, embedding methods have been used for entity alignment tasks, that learn a vector-space representation of entities which preserves their similarity in the original KGs. A wide variety of supervised, unsupervised, and semi-supervised methods have been proposed that exploit both factual (attribute based) and structural information (relation based) of entities in the KGs. Still, a quantitative assessment of their strengths and weaknesses in real-world KGs according to different performance metrics and KG characteristics is missing from the literature. In this work, we conduct the first meta-level analysis of popular embedding methods for entity alignment, based on a statistically sound methodology. Our analysis reveals statistically significant correlations of different embedding methods with various meta-features extracted by KGs and rank them in a statistically significant way according to their effectiveness across all real-world KGs of our testbed. Finally, we study interesting trade-offs in terms of methods\u2019 effectiveness and efficiency.<\/jats:p>","DOI":"10.1007\/s10618-023-00941-9","type":"journal-article","created":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T08:02:45Z","timestamp":1688025765000},"page":"2070-2137","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Knowledge graph embedding methods for entity alignment: experimental review"],"prefix":"10.1007","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2162-5822","authenticated-orcid":false,"given":"Nikolaos","family":"Fanourakis","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0683-030X","authenticated-orcid":false,"given":"Vasilis","family":"Efthymiou","sequence":"additional","affiliation":[]},{"given":"Dimitris","family":"Kotzinos","sequence":"additional","affiliation":[]},{"given":"Vassilis","family":"Christophides","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,6,29]]},"reference":[{"key":"941_CR1","doi-asserted-by":"crossref","unstructured":"Ahmetaj S, Efthymiou V, Fagin R, et\u00a0al. (2021) Ontology-enriched query answering on relational databases. In: AAAI, pp 15247\u201315254","DOI":"10.1609\/aaai.v35i17.17789"},{"key":"941_CR2","unstructured":"Berrendorf M, Faerman E, Vermue L, et\u00a0al. (2020) On the ambiguity of rank-based evaluation of entity alignment or link prediction methods. CoRR arXiv:abs\/2002.06914"},{"key":"941_CR3","unstructured":"Bordes A, Usunier N, Garc\u00eda-Dur\u00e1n A, et\u00a0al. (2013) Translating embeddings for modeling multi-relational data. In: NeurIPS, pp 2787\u20132795"},{"key":"941_CR4","doi-asserted-by":"crossref","unstructured":"Cai W, Ma W, Zhan J, et\u00a0al. (2022) Entity alignment with reliable path reasoning and relation-aware heterogeneous graph transformer. In: IJCAI, pp 1930\u20131937","DOI":"10.24963\/ijcai.2022\/268"},{"key":"941_CR5","doi-asserted-by":"crossref","unstructured":"Cao Y, Liu Z, Li C, et\u00a0al. (2019) Multi-channel graph neural network for entity alignment. In: ACL, pp 1452\u20131461","DOI":"10.18653\/v1\/P19-1140"},{"key":"941_CR6","unstructured":"Chaurasiya D, Surisetty A, Kumar N, et\u00a0al. (2022) Entity alignment for knowledge graphs: Progress, challenges, and empirical studies. CoRR arXiv:abs\/2205.08777"},{"key":"941_CR7","doi-asserted-by":"crossref","unstructured":"Chen M, Tian Y, Yang M, et\u00a0al. (2017) Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: IJCAI, pp 1511\u20131517","DOI":"10.24963\/ijcai.2017\/209"},{"key":"941_CR8","doi-asserted-by":"crossref","unstructured":"Chen M, Tian Y, Chang K, et\u00a0al. (2018) Co-training embeddings of knowledge graphs and entity descriptions for cross-lingual entity alignment. In: IJCAI, pp 3998\u20134004","DOI":"10.24963\/ijcai.2018\/556"},{"key":"941_CR9","unstructured":"Choudhary S, Luthra T, Mittal A, et\u00a0al. (2021) A survey of knowledge graph embedding and their applications. CoRR arXiv:abs\/2107.07842"},{"key":"941_CR10","doi-asserted-by":"crossref","unstructured":"Christophides V, Efthymiou V, Stefanidis K (2015) Entity Resolution in the Web of Data. Theory and Technology, Morgan & Claypool Publishers, San Rafael, California, Synthesis Lectures on the Semantic Web","DOI":"10.1007\/978-3-031-79468-1"},{"key":"941_CR11","doi-asserted-by":"crossref","unstructured":"Christophides V, Efthymiou V, Palpanas T, et\u00a0al. (2021) An overview of end-to-end entity resolution for big data. ACM Comput Surv 53(6):127:1\u2013127:42","DOI":"10.1145\/3418896"},{"issue":"101","key":"941_CR12","first-page":"102","volume":"100","author":"C Coleman","year":"2017","unstructured":"Coleman C, Narayanan D, Kang D et al (2017) Dawnbench: An end-to-end deep learning benchmark and competition. Training 100(101):102","journal-title":"Training"},{"issue":"1","key":"941_CR13","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1145\/3352020.3352024","volume":"53","author":"C Coleman","year":"2019","unstructured":"Coleman C, Kang D, Narayanan D et al (2019) Analysis of dawnbench, a time-to-accuracy machine learning performance benchmark. ACM SIGOPS Oper Syst Rev 53(1):14\u201325","journal-title":"ACM SIGOPS Oper Syst Rev"},{"key":"941_CR14","first-page":"1","volume":"7","author":"J Demsar","year":"2006","unstructured":"Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1\u201330","journal-title":"J Mach Learn Res"},{"key":"941_CR15","unstructured":"Devlin J, Chang M, Lee K, et\u00a0al. (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp 4171\u20134186"},{"key":"941_CR16","doi-asserted-by":"crossref","unstructured":"Dong X, Gabrilovich E, Heitz G, et\u00a0al. (2014) Knowledge vault: a web-scale approach to probabilistic knowledge fusion. In: SIGKDD, pp 601\u2013610","DOI":"10.1145\/2623330.2623623"},{"key":"941_CR17","doi-asserted-by":"crossref","unstructured":"Efthymiou V, Stefanidis K, Christophides V (2015) Big data entity resolution: From highly to somehow similar entity descriptions in the web. In: IEEE Big Data, pp 401\u2013410","DOI":"10.1109\/BigData.2015.7363781"},{"key":"941_CR18","unstructured":"Efthymiou V, Stefanidis K, Pitoura E, et\u00a0al. (2022) FairER: Entity resolution with fairness constraints. In: CIKM, pp 3004\u20133008"},{"key":"941_CR19","doi-asserted-by":"crossref","unstructured":"Fanourakis N, Efthymiou V, Christophides V, et\u00a0al. (2023) Structural bias in knowledge graphs for the entity alignment task. In: ESWC","DOI":"10.1007\/978-3-031-33455-9_5"},{"key":"941_CR20","doi-asserted-by":"crossref","unstructured":"Fisher J, Mittal A, Palfrey D, et\u00a0al. (2020) Debiasing knowledge graph embeddings. In: EMNLP, pp 7332\u20137345","DOI":"10.18653\/v1\/2020.emnlp-main.595"},{"key":"941_CR21","doi-asserted-by":"crossref","unstructured":"Jiang J, Li M, Gu Z (2021) A survey on translating embedding based entity alignment in knowledge graphs. In: DSC, pp 187\u2013194","DOI":"10.1109\/DSC53577.2021.00033"},{"key":"941_CR22","unstructured":"Kamigaito H, Hayashi K (2022) Comprehensive analysis of negative sampling in knowledge graph representation learning. In: ICML, pp 10661\u201310675"},{"key":"941_CR23","unstructured":"Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: ICLR"},{"key":"941_CR24","unstructured":"Kocmi T, Bojar O (2017) An exploration of word embedding initialization in deep-learning tasks. In: ICON, pp 56\u201364"},{"issue":"2","key":"941_CR25","doi-asserted-by":"publisher","first-page":"167","DOI":"10.3233\/SW-140134","volume":"6","author":"J Lehmann","year":"2015","unstructured":"Lehmann J, Isele R, Jakob M et al (2015) Dbpedia - A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web 6(2):167\u2013195","journal-title":"Semantic Web"},{"issue":"8","key":"941_CR26","first-page":"1712","volume":"15","author":"M Leone","year":"2022","unstructured":"Leone M, Huber S, Arora A et al (2022) A critical re-evaluation of neural methods for entity alignment. PVLDB 15(8):1712\u20131725","journal-title":"PVLDB"},{"issue":"1","key":"941_CR27","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1109\/18.61115","volume":"37","author":"J Lin","year":"1991","unstructured":"Lin J (1991) Divergence measures based on the shannon entropy. IEEE Trans Inf Theory 37(1):145\u2013151","journal-title":"IEEE Trans Inf Theory"},{"key":"941_CR28","doi-asserted-by":"crossref","unstructured":"Mao X, Wang W, Xu H, et\u00a0al. (2020a) MRAEA: an efficient and robust entity alignment approach for cross-lingual knowledge graph. In: WSDM, pp 420\u2013428","DOI":"10.1145\/3336191.3371804"},{"key":"941_CR29","doi-asserted-by":"crossref","unstructured":"Mao X, Wang W, Xu H, et\u00a0al. (2020b) Relational reflection entity alignment. In: CIKM, pp 1095\u20131104","DOI":"10.1145\/3340531.3412001"},{"key":"941_CR30","unstructured":"Million E (2007) The Hadamard product. Course Notes 3(6)"},{"key":"941_CR31","volume-title":"Distribution-free Multiple Comparisons","author":"P Nemenyi","year":"1963","unstructured":"Nemenyi P (1963) Distribution-free Multiple Comparisons. Princeton University, Princeton"},{"key":"941_CR32","unstructured":"Nickel M, Kiela D (2017) Poincar\u00e9 embeddings for learning hierarchical representations. In: NeurIPS, pp 6338\u20136347"},{"key":"941_CR33","unstructured":"Obraczka D, Schuchart J, Rahm E (2021) EAGER: embedding-assisted entity resolution for knowledge graphs. CoRR arXiv:abs\/2101.06126"},{"key":"941_CR34","doi-asserted-by":"publisher","first-page":"115892","DOI":"10.1016\/j.eswa.2021.115892","volume":"187","author":"L Parisi","year":"2022","unstructured":"Parisi L, Neagu D, Ma R et al (2022) Quantum relu activation for convolutional neural networks to improve diagnosis of parkinson\u2019s disease and COVID-19. Expert Syst Appl 187:115892","journal-title":"Expert Syst Appl"},{"key":"941_CR35","doi-asserted-by":"crossref","unstructured":"Rebele T, Suchanek FM, Hoffart J, et\u00a0al. (2016) YAGO: A multilingual knowledge base from wikipedia, wordnet, and geonames. In: ISWC, pp 177\u2013185","DOI":"10.1007\/978-3-319-46547-0_19"},{"issue":"3","key":"941_CR36","first-page":"157","volume":"5","author":"FM Suchanek","year":"2011","unstructured":"Suchanek FM, Abiteboul S, Senellart P (2011) PARIS: probabilistic alignment of relations, instances, and schema. PVLDB 5(3):157\u2013168","journal-title":"PVLDB"},{"key":"941_CR37","doi-asserted-by":"crossref","unstructured":"Sun Z, Hu W, Zhang Q, et\u00a0al. (2018) Bootstrapping entity alignment with knowledge graph embedding. In: IJCAI, pp 4396\u20134402","DOI":"10.24963\/ijcai.2018\/611"},{"key":"941_CR38","unstructured":"Sun Z, Deng Z, Nie J, et\u00a0al. (2019) Rotate: Knowledge graph embedding by relational rotation in complex space. In: ICLR"},{"issue":"11","key":"941_CR39","first-page":"2326","volume":"13","author":"Z Sun","year":"2020","unstructured":"Sun Z, Zhang Q, Hu W et al (2020) A benchmarking study of embedding-based entity alignment for knowledge graphs. PVLDB 13(11):2326\u20132340","journal-title":"PVLDB"},{"key":"941_CR40","doi-asserted-by":"crossref","unstructured":"Tang X, Zhang J, Chen B, et\u00a0al. (2020) BERT-INT: A bert-based interaction model for knowledge graph alignment. In: IJCAI, pp 3174\u20133180","DOI":"10.24963\/ijcai.2020\/439"},{"issue":"1","key":"941_CR41","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1007\/s10462-017-9539-5","volume":"50","author":"JK Tarus","year":"2018","unstructured":"Tarus JK, Niu Z, Mustafa G (2018) Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning. Artif Intell Rev 50(1):21\u201348","journal-title":"Artif Intell Rev"},{"key":"941_CR42","doi-asserted-by":"crossref","unstructured":"Trisedya BD, Qi J, Zhang R (2019) Entity alignment between knowledge graphs using attribute embeddings. In: AAAI, pp 297\u2013304","DOI":"10.1609\/aaai.v33i01.3301297"},{"key":"941_CR43","unstructured":"Trouillon T, Welbl J, Riedel S, et\u00a0al. (2016) Complex embeddings for simple link prediction. In: ICML, pp 2071\u20132080"},{"key":"941_CR44","unstructured":"Velickovic P, Cucurull G, Casanova A, et\u00a0al. (2018) Graph attention networks. In: ICLR"},{"key":"941_CR45","doi-asserted-by":"publisher","unstructured":"Vrandecic D, Kr\u00f6tzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57:78\u201385. https:\/\/doi.org\/10.1145\/2629489","DOI":"10.1145\/2629489"},{"key":"941_CR46","doi-asserted-by":"crossref","unstructured":"Vretinaris A, Lei C, Efthymiou V, et\u00a0al. (2021) Medical entity disambiguation using graph neural networks. In: SIGMOD, pp 2310\u20132318","DOI":"10.1145\/3448016.3457328"},{"issue":"12","key":"941_CR47","doi-asserted-by":"publisher","first-page":"2724","DOI":"10.1109\/TKDE.2017.2754499","volume":"29","author":"Q Wang","year":"2017","unstructured":"Wang Q, Mao Z, Wang B et al (2017) Knowledge graph embedding: a survey of approaches and applications. IEEE Trans Knowl Data Eng 29(12):2724\u20132743","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"941_CR48","doi-asserted-by":"crossref","unstructured":"Wang Z, Yang J, Ye X (2020) Knowledge graph alignment with entity-pair embedding. In: EMNLP, pp 1672\u20131680","DOI":"10.18653\/v1\/2020.emnlp-main.130"},{"key":"941_CR49","doi-asserted-by":"crossref","unstructured":"Wang Z, Li M, Gu Z (2021) A review of entity alignment based on graph convolutional neural network. In: DSC, pp 144\u2013151","DOI":"10.1109\/DSC53577.2021.00027"},{"key":"941_CR50","doi-asserted-by":"crossref","unstructured":"Wu Y, Liu X, Feng Y, et\u00a0al. (2019) Relation-aware entity alignment for heterogeneous knowledge graphs. In: IJCAI, pp 5278\u20135284","DOI":"10.24963\/ijcai.2019\/733"},{"key":"941_CR51","doi-asserted-by":"crossref","unstructured":"Xiong C, Dai Z, Callan J, et\u00a0al. (2017) End-to-end neural ad-hoc ranking with kernel pooling. In: SIGIR, pp 55\u201364","DOI":"10.1145\/3077136.3080809"},{"key":"941_CR52","unstructured":"Yang B, Yih W, He X, et\u00a0al. (2015) Embedding entities and relations for learning and inference in knowledge bases. In: ICLR"},{"key":"941_CR53","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.aiopen.2021.02.002","volume":"2","author":"K Zeng","year":"2021","unstructured":"Zeng K, Li C, Hou L et al (2021) A comprehensive survey of entity alignment for knowledge graphs. AI Open 2:1\u201313","journal-title":"AI Open"},{"issue":"2","key":"941_CR54","doi-asserted-by":"publisher","first-page":"185","DOI":"10.23919\/TST.2017.7889640","volume":"22","author":"C Zhang","year":"2017","unstructured":"Zhang C, Zhou M, Han X et al (2017) Knowledge graph embedding for hyper-relational data. Tsinghua Sci Technol 22(2):185\u2013197","journal-title":"Tsinghua Sci Technol"},{"key":"941_CR55","doi-asserted-by":"crossref","unstructured":"Zhang Q, Sun Z, Hu W, et\u00a0al. (2019) Multi-view knowledge graph embedding for entity alignment. In: IJCAI, pp 5429\u20135435","DOI":"10.24963\/ijcai.2019\/754"},{"issue":"5","key":"941_CR56","doi-asserted-by":"publisher","first-page":"1143","DOI":"10.1007\/s00778-022-00747-z","volume":"31","author":"R Zhang","year":"2022","unstructured":"Zhang R, Trisedya BD, Li M et al (2022) A benchmark and comprehensive survey on knowledge graph entity alignment via representation learning. VLDBJ 31(5):1143\u20131168","journal-title":"VLDBJ"},{"key":"941_CR57","doi-asserted-by":"crossref","unstructured":"Zhang Z, Liu H, Chen J, et\u00a0al. (2020) An industry evaluation of embedding-based entity alignment. In: COLING, pp 179\u2013189","DOI":"10.18653\/v1\/2020.coling-industry.17"},{"issue":"6","key":"941_CR58","first-page":"2610","volume":"34","author":"X Zhao","year":"2022","unstructured":"Zhao X, Zeng W, Tang J et al (2022) An experimental study of state-of-the-art entity alignment approaches. IEEE Trans Knowl Data Eng 34(6):2610\u20132625","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"941_CR59","doi-asserted-by":"crossref","unstructured":"Zhu Q, Zhou X, Wu J, et\u00a0al. (2019) Neighborhood-aware attentional representation for multilingual knowledge graphs. In: IJCAI, pp 1943\u20131949","DOI":"10.24963\/ijcai.2019\/269"}],"container-title":["Data Mining and Knowledge Discovery"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-023-00941-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10618-023-00941-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10618-023-00941-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,19]],"date-time":"2023-08-19T13:11:13Z","timestamp":1692450673000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10618-023-00941-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,29]]},"references-count":59,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["941"],"URL":"https:\/\/doi.org\/10.1007\/s10618-023-00941-9","relation":{},"ISSN":["1384-5810","1573-756X"],"issn-type":[{"value":"1384-5810","type":"print"},{"value":"1573-756X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,29]]},"assertion":[{"value":"28 January 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no conflicts of interest to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}