{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T00:42:08Z","timestamp":1783471328571,"version":"3.55.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T00:00:00Z","timestamp":1680566400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T00:00:00Z","timestamp":1680566400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012165","name":"Key Technologies Research and Development Program","doi-asserted-by":"publisher","award":["2022YFF0712400"],"award-info":[{"award-number":["2022YFF0712400"]}],"id":[{"id":"10.13039\/501100012165","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62276063"],"award-info":[{"award-number":["62276063"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"publisher","award":["BK20221457"],"award-info":[{"award-number":["BK20221457"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Data Sci. Eng."],"published-print":{"date-parts":[[2023,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We study the problem of multimodal embedding-based entity alignment (EA) between different knowledge graphs. Recent works have attempted to incorporate images (visual context) to address EA in a multimodal view. While the benefits of multimodal information have been observed, its negative impacts are non-negligible as injecting images without constraints brings much noise. It also remains unknown under what circumstances or to what extent visual context is truly helpful to the task. In this work, we propose to learn entity representations from graph structures and visual context, and combine feature similarities to find alignments at the output level. On top of this, we explore a mechanism which utilizes classification techniques and entity types to remove potentially un-helpful images (visual noises) during alignment learning and inference. We conduct extensive experiments to examine this mechanism and provide thorough analysis about impacts of the visual modality on EA.<\/jats:p>","DOI":"10.1007\/s41019-023-00208-9","type":"journal-article","created":{"date-parts":[[2023,4,4]],"date-time":"2023-04-04T11:10:40Z","timestamp":1680606640000},"page":"124-134","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Probing the Impacts of Visual Context in Multimodal Entity Alignment"],"prefix":"10.1007","volume":"8","author":[{"given":"Meng","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8525-2300","authenticated-orcid":false,"given":"Yinghui","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Han","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ziheng","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhenxi","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yefeng","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,4,4]]},"reference":[{"key":"208_CR1","doi-asserted-by":"publisher","first-page":"2326","DOI":"10.14778\/3407790.3407828","volume":"13","author":"Z Sun","year":"2020","unstructured":"Sun Z, Zhang Q, Hu W, Wang C, Chen M, Akrami F, Li C (2020) A benchmarking study of embedding-based entity alignment for knowledge graphs. Proc VLDB Endow 13:2326\u20132340","journal-title":"Proc VLDB Endow"},{"key":"208_CR2","doi-asserted-by":"crossref","unstructured":"Zhang Z, Chen J, Chen X, Liu H, Xiang Y, Liu B, Zheng Y (2020) An industry evaluation of embedding-based entity alignment. In: Proceedings of the 28th international conference on computational linguistics, Barcelona, Spain, pp 179\u2013189","DOI":"10.18653\/v1\/2020.coling-industry.17"},{"issue":"2","key":"208_CR3","doi-asserted-by":"publisher","first-page":"167","DOI":"10.3233\/SW-140134","volume":"6","author":"J Lehmann","year":"2015","unstructured":"Lehmann J, Isele R, Jakob M, Jentzsch A, Kontokostas D, Mendes PN, Hellmann S, Morsey M, Van Kleef P, Auer S (2015) DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia. Semant Web 6(2):167\u2013195","journal-title":"Semant Web"},{"issue":"10","key":"208_CR4","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2629489","volume":"57","author":"D Vrande\u010di\u0107","year":"2014","unstructured":"Vrande\u010di\u0107 D, Kr\u00f6tzsch M (2014) Wikidata: a free collaborative knowledgebase. Commun ACM 57(10):78\u201385","journal-title":"Commun ACM"},{"key":"208_CR5","doi-asserted-by":"crossref","unstructured":"Chen L, Li Z, Wang Y, Xu T, Wang Z, Chen E (2020) MMEA: entity alignment for multi-modal knowledge graph. In: International conference on knowledge science, engineering and management, pp 134\u2013147","DOI":"10.1007\/978-3-030-55130-8_12"},{"key":"208_CR6","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1016\/j.neucom.2021.03.132","volume":"461","author":"H Guo","year":"2021","unstructured":"Guo H, Tang J, Zeng W, Zhao X, Liu L (2021) Multi-modal entity alignment in hyperbolic space. Neurocomputing 461:598\u2013607","journal-title":"Neurocomputing"},{"key":"208_CR7","doi-asserted-by":"crossref","unstructured":"Liu F, Chen M, Roth D, Collier N (2021) Visual pivoting for (unsupervised) entity alignment. In: Proceedings of the AAAI conference on artificial intelligence, pp 4257\u20134266","DOI":"10.1609\/aaai.v35i5.16550"},{"key":"208_CR8","doi-asserted-by":"crossref","unstructured":"Sun Z, Hu W, Li C (2017) Cross-lingual entity alignment via joint attribute-preserving embedding. In: International semantic web conference, pp 628\u2013644","DOI":"10.1007\/978-3-319-68288-4_37"},{"key":"208_CR9","doi-asserted-by":"crossref","unstructured":"Chen M, Tian Y, Yang M, Zaniolo C (2017) Multilingual knowledge graph embeddings for cross-lingual knowledge alignment. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 1511\u20131517","DOI":"10.24963\/ijcai.2017\/209"},{"key":"208_CR10","unstructured":"Bordes A, Usunier N, Garc\u00eda-Dur\u00e1n A, Weston J, Yakhnenko O (2013) Translating embeddings for modeling multi-relational data. In: Advances in neural information processing systems, pp 2787\u20132795"},{"key":"208_CR11","doi-asserted-by":"crossref","unstructured":"Zhu H, Xie R, Liu Z, Sun M (2017) Iterative entity alignment via joint knowledge embeddings. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 4258\u20134264","DOI":"10.24963\/ijcai.2017\/595"},{"key":"208_CR12","doi-asserted-by":"crossref","unstructured":"Sun Z, Hu W, Zhang Q, Qu Y (2018) Bootstrapping entity alignment with knowledge graph embedding. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, pp 4396\u20134402","DOI":"10.24963\/ijcai.2018\/611"},{"key":"208_CR13","doi-asserted-by":"crossref","unstructured":"Wang Z, Lv Q, Lan X, Zhang Y (2018) Cross-lingual knowledge graph alignment via graph convolutional networks. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 349\u2013357","DOI":"10.18653\/v1\/D18-1032"},{"key":"208_CR14","doi-asserted-by":"crossref","unstructured":"Trisedya BD, Qi J, Zhang R (2019) Entity alignment between knowledge graphs using attribute embeddings. In: Proceedings of the AAAI conference on artificial intelligence, pp 297\u2013304","DOI":"10.1609\/aaai.v33i01.3301297"},{"key":"208_CR15","doi-asserted-by":"crossref","unstructured":"Zhang Q, Sun Z, Hu W, Chen M, Guo L, Qu Y (2019) Multi-view knowledge graph embedding for entity alignment. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, pp 5429\u20135435","DOI":"10.24963\/ijcai.2019\/754"},{"key":"208_CR16","doi-asserted-by":"crossref","unstructured":"Yang H-W, Zou Y, Shi P, Lu W, Lin JJ, Sun X (2019) Aligning cross-lingual entities with multi-aspect information. In: Proceedings of the 2019 conference on empirical methods in natural language processing, pp. 4430\u20134440","DOI":"10.18653\/v1\/D19-1451"},{"key":"208_CR17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s41019-022-00175-7","volume":"7","author":"X Zhao","year":"2022","unstructured":"Zhao X, Zeng W, Tang J, Li X, Luo M, Zheng Q (2022) Toward entity alignment in the open world: an unsupervised approach with confidence modeling. Data Scie Eng 7:1\u201314","journal-title":"Data Scie Eng"},{"key":"208_CR18","unstructured":"Kipf T, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: International conference on learning representations"},{"key":"208_CR19","unstructured":"Lin Z, Zhang Z, Wang M, Shi Y, Wu X, Zheng Y (2022) Multi-modal contrastive representation learning for entity alignment. In: Proceedings of the 29th international conference on computational linguistics, pp 2572\u20132584. https:\/\/aclanthology.org\/2022.coling-1.227"},{"key":"208_CR20","doi-asserted-by":"crossref","unstructured":"Hao J, Chen M, Yu W, Sun Y, Wang W (2019) Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1709\u20131719","DOI":"10.1145\/3292500.3330838"},{"key":"208_CR21","doi-asserted-by":"crossref","unstructured":"Xiang Y, Zhang Z, Chen J, Chen X, Lin Z, Zheng Y (2021) OntoEA: ontology-guided entity alignment via joint knowledge graph embedding. In: Findings of the association for computational linguistics: ACL-IJCNLP 2021, pp 1117\u20131128","DOI":"10.18653\/v1\/2021.findings-acl.96"},{"key":"208_CR22","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"key":"208_CR23","doi-asserted-by":"crossref","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 248\u2013255","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"208_CR24","unstructured":"Conneau A, Lample G, Ranzato M, Denoyer L, J\u2019egou H (2018) Word translation without parallel data. In: International conference on learning representation"},{"key":"208_CR25","doi-asserted-by":"crossref","unstructured":"Liu F, Ye R, Wang X, Li S (2020) HAL: improved text-image matching by mitigating visual semantic hubs. In: Proceedings of the AAAI conference on artificial intelligence, pp 11563\u201311571","DOI":"10.1609\/aaai.v34i07.6823"},{"key":"208_CR26","doi-asserted-by":"crossref","unstructured":"Cao Y, Liu Z, Li C, Liu Z, Li J-Z, Chua T-S (2019) Multi-channel graph neural network for entity alignment. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 1452\u20131461","DOI":"10.18653\/v1\/P19-1140"},{"key":"208_CR27","doi-asserted-by":"crossref","unstructured":"Pei S, Yu L, Hoehndorf R, Zhang X (2019) Semi-supervised entity alignment via knowledge graph embedding with awareness of degree difference. In: The world wide web conference, pp 3130\u20133136","DOI":"10.1145\/3308558.3313646"},{"key":"208_CR28","doi-asserted-by":"crossref","unstructured":"Chen L, Li Z, Xu T, Wu H, Wang Z, Yuan NJ, Chen E (2022) Multi-modal siamese network for entity alignment. In: Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp 118\u2013126","DOI":"10.1145\/3534678.3539244"},{"key":"208_CR29","doi-asserted-by":"crossref","unstructured":"Sun Z, Wang C, Hu W, Chen M, Dai J, Zhang W, Qu Y (2020) Knowledge graph alignment network with gated multi-hop neighborhood aggregation. In: Proceedings of the AAAI conference on artificial intelligence, pp 222\u2013229","DOI":"10.1609\/aaai.v34i01.5354"}],"container-title":["Data Science and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41019-023-00208-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s41019-023-00208-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s41019-023-00208-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,15]],"date-time":"2023-05-15T14:44:19Z","timestamp":1684161859000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s41019-023-00208-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,4]]},"references-count":29,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6]]}},"alternative-id":["208"],"URL":"https:\/\/doi.org\/10.1007\/s41019-023-00208-9","relation":{},"ISSN":["2364-1185","2364-1541"],"issn-type":[{"value":"2364-1185","type":"print"},{"value":"2364-1541","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,4]]},"assertion":[{"value":"7 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 March 2023","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 March 2023","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 April 2023","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All authors disclosed no relevant relationships.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}