{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T18:46:37Z","timestamp":1781117197911,"version":"3.54.1"},"reference-count":43,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2025,6,23]],"date-time":"2025-06-23T00:00:00Z","timestamp":1750636800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62471493"],"award-info":[{"award-number":["62471493"]}]},{"name":"National Natural Science Foundation of China","award":["ZR2023LZH017"],"award-info":[{"award-number":["ZR2023LZH017"]}]},{"name":"National Natural Science Foundation of China","award":["ZR2024MF066"],"award-info":[{"award-number":["ZR2024MF066"]}]},{"name":"National Natural Science Foundation of China","award":["ZR2020MF140"],"award-info":[{"award-number":["ZR2020MF140"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["62471493"],"award-info":[{"award-number":["62471493"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["ZR2023LZH017"],"award-info":[{"award-number":["ZR2023LZH017"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["ZR2024MF066"],"award-info":[{"award-number":["ZR2024MF066"]}]},{"name":"Natural Science Foundation of Shandong Province","award":["ZR2020MF140"],"award-info":[{"award-number":["ZR2020MF140"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Entity alignment is a critical technique for integrating diverse knowledge graphs. Although existing methods have achieved impressive success in traditional entity alignment, they may struggle to handle the complexities arising from interactions and dependencies in multi-modal knowledge. In this paper, a novel multi-modal entity alignment model called ERMF is proposed, which leverages distinct modal characteristics of entities to identify equivalent entities across different multi-modal knowledge graphs. The symmetry in cross-modal interactions and hierarchical feature fusion is a core design principle of our approach. Specifically, we first utilize different feature encoders to independently extract features from different modalities. Concurrently, visual features and nearest neighbor negative sampling methods are incorporated to design a vision-guided negative sample generation strategy based on contrastive learning, ensuring a symmetric balance between positive and negative samples and guiding the model to learn effective relationship embeddings. Subsequently, in the feature fusion stage, we propose a multi-layer feature fusion approach that incorporates cross-attention and cross-modal attention mechanisms with symmetric processing of intra- and inter-modal correlations, thereby obtaining multi-granularity features. Extensive experiments were conducted on two public datasets, namely FB15K-DB15K and FB15K-YAGO15K. With 20% aligned seeds, ERMF improves Hits@1 by 8.4% and 26%, and MRR by 6% and 19.2% compared to the best baseline. The symmetric architecture of our model ensures the robust and balanced utilization of multi-modal information, aligning with the principles of structural and functional symmetry in knowledge integration.<\/jats:p>","DOI":"10.3390\/sym17070990","type":"journal-article","created":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T10:44:41Z","timestamp":1750761881000},"page":"990","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multi-Modal Entity Alignment Based on Enhanced Relationship Learning and Multi-Layer Feature Fusion"],"prefix":"10.3390","volume":"17","author":[{"given":"Huayu","family":"Li","sequence":"first","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yujie","family":"Hou","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Cultural Heritage and Information Management, Shanghai University, Shanghai 200444, China"},{"name":"Library of Shanghai Lixin University of Accounting and Finance, Shanghai 201209, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0990-5581","authenticated-orcid":false,"given":"Peiying","family":"Zhang","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cuicui","family":"Wang","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7331-4727","authenticated-orcid":false,"given":"Kai","family":"Liu","sequence":"additional","affiliation":[{"name":"Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China"},{"name":"State Key Laboratory of Space Network and Communications, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wang, Y., Sun, H., Wang, J., Wang, J., Tang, W., Qi, Q., Sun, S., and Liao, J. (2024). Towards semantic consistency: Dirichlet energy driven robust multi-modal entity alignment. arXiv.","DOI":"10.1109\/ICDE60146.2024.00274"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"167","DOI":"10.3233\/SW-140134","article-title":"Dbpedia\u2013a large-scale, multilingual knowledge base extracted from wikipedia","volume":"6","author":"Lehmann","year":"2015","journal-title":"Semant. Web"},{"key":"ref_3","unstructured":"Mahdisoltani, F., Biega, J., and Suchanek, F.M. (2013, January 6\u20139). Yago3: A knowledge base from multilingual wikipedias. Proceedings of the CIDR, Asilomar, CA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008, January 9\u201312). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada.","DOI":"10.1145\/1376616.1376746"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"21094","DOI":"10.1109\/ACCESS.2021.3055371","article-title":"Research on medical question answering system based on knowledge graph","volume":"9","author":"Jiang","year":"2021","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Huang, X., Zhang, J., Li, D., and Li, P. (2019, January 11\u201315). Knowledge graph embedding based question answering. Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, Melbourne, VIC, Australia.","DOI":"10.1145\/3289600.3290956"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Jin, Q., Bao, T., and Li, W. (2023, January 7\u201314). Multi-modal knowledge hypergraph for diverse image retrieval. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v37i3.25445"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sun, R., Cao, X., Zhao, Y., Wan, J., Zhou, K., Zhang, F., Wang, Z., and Zheng, K. (2020, January 19\u201323). Multi-modal knowledge graphs for recommender systems. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Virtual Event.","DOI":"10.1145\/3340531.3411947"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chen, L., Li, Z., Wang, Y., Xu, T., Wang, Z., and Chen, E. (2020). MMEA: Entity alignment for multi-modal knowledge graph. Knowledge Science, Engineering and Management, Proceedings of the 13th International Conference, KSEM 2020, Hangzhou, China, 28\u201330 August 2020, Springer. Proceedings, Part I 13.","DOI":"10.1007\/978-3-030-55130-8_12"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, F., Chen, M., Roth, D., and Collier, N. (2021, January 19\u201321). Visual pivoting for (unsupervised) entity alignment. Proceedings of the AAAI Conference on Artificial Intelligence, Virtual Event.","DOI":"10.1609\/aaai.v35i5.16550"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Chen, L., Li, Z., Xu, T., Wu, H., Wang, Z., Yuan, N.J., and Chen, E. (2022, January 14\u201318). Multi-modal siamese network for entity alignment. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.","DOI":"10.1145\/3534678.3539244"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liu, Y., Li, H., Garcia-Duran, A., Niepert, M., Onoro-Rubio, D., and Rosenblum, D.S. (2019). MMKG: Multi-modal knowledge graphs. The Semantic Web, Proceedings of the 16th International Conference, ESWC 2019, Portoro\u017e, Slovenia, 2\u20136 June 2019, Springer. Proceedings 16.","DOI":"10.1007\/978-3-030-21348-0_30"},{"key":"ref_13","unstructured":"Chen, Z., Chen, J., Zhang, W., Guo, L., Fang, Y., Huang, Y., Zhang, Y., Geng, Y., Pan, J.Z., and Song, W. (November, January 29). Meaformer: Multi-modal entity alignment transformer for meta modality hybrid. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada."},{"key":"ref_14","unstructured":"Lin, Z., Zhang, Z., Wang, M., Shi, Y., Wu, X., and Zheng, Y. (2022). Multi-modal contrastive representation learning for entity alignment. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"100159","DOI":"10.1016\/j.bdr.2020.100159","article-title":"Richpedia: A large-scale, comprehensive multi-modal knowledge graph","volume":"22","author":"Wang","year":"2020","journal-title":"Big Data Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"897","DOI":"10.1109\/TMM.2019.2937181","article-title":"Enhancing the quality of image tagging using a visio-textual knowledge base","volume":"22","author":"Chaudhary","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103241","DOI":"10.1016\/j.ipm.2022.103241","article-title":"Medical knowledge-based network for patient-oriented visual question answering","volume":"60","author":"Huang","year":"2023","journal-title":"Inf. Process. Manag."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mondal, P., Chakder, D., Raj, S., Saha, S., and Onoe, N. (2023, January 27\u201331). Graph convolutional neural network for multimodal movie recommendation. Proceedings of the 38th ACM\/SIGAPP Symposium on Applied Computing, Tallinn, Estonia.","DOI":"10.1145\/3555776.3577853"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xie, R., Liu, Z., Luan, H., and Sun, M. (2016). Image-embodied knowledge representation learning. arXiv.","DOI":"10.24963\/ijcai.2017\/438"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, Z., Li, L., Li, Q., and Zeng, D. (2019, January 14\u201319). Multimodal data enhanced representation learning for knowledge graphs. Proceedings of the 2019 International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary.","DOI":"10.1109\/IJCNN.2019.8852079"},{"key":"ref_21","unstructured":"Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., and Yakhnenko, O. (2013, January 5\u201310). Translating embeddings for modeling multi-relational data. Proceedings of the NIPS\u201913: Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Moon, C., Jones, P., and Samatova, N.F. (2017, January 6\u201310). Learning entity type embeddings for knowledge graph completion. Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore.","DOI":"10.1145\/3132847.3133095"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7480","DOI":"10.1007\/s10489-021-02693-9","article-title":"MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning","volume":"52","author":"Lu","year":"2022","journal-title":"Appl. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1016\/j.neucom.2022.05.058","article-title":"MultiJAF: Multi-modal joint entity alignment framework for multi-modal knowledge graph","volume":"500","author":"Cheng","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.inffus.2022.09.012","article-title":"DFMKE: A dual fusion multi-modal knowledge graph embedding framework for entity alignment","volume":"90","author":"Zhu","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Wu, Y., Liu, X., Feng, Y., Wang, Z., Yan, R., and Zhao, D. (2019, January 10\u201316). Relation-Aware Entity Alignment for Heterogeneous Knowledge Graphs. Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-2019, Macao, China.","DOI":"10.24963\/ijcai.2019\/733"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, B., Yue, L., Tao, C., Han, X., Calvanese, D., and Amagasa, T. (2023, January 6\u20138). Probing the Impacts of Visual Context in Multimodal Entity Alignment. Proceedings of the Web and Big Data, Wuhan, China.","DOI":"10.1007\/978-3-031-25201-3"},{"key":"ref_28","unstructured":"Xu, B., Xu, C., and Su, B. (November, January 29). Cross-modal graph attention network for entity alignment. Proceedings of the 31st ACM International Conference on Multimedia, Ottawa, ON, Canada."},{"key":"ref_29","unstructured":"Chen, L., Sun, Y., Zhang, S., Ye, Y., Wu, W., and Xiong, H. (2024, January 10\u201315). Tackling Uncertain Correspondences for Multi-Modal Entity Alignment. Proceedings of the Thirty-Eighth Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_30","unstructured":"Hu, Z., Guti\u00e9rrez-Basulto, V., Xiang, Z., Li, R., and Pan, J.Z. (2024). Leveraging Intra-modal and Inter-modal Interaction for Multi-Modal Entity Alignment. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, L., Qi, P., Bao, X., Zhou, C., and Qin, B. (2024, January 26\u201327). Pseudo-Label Calibration Semi-supervised Multi-Modal Entity Alignment. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i8.28762"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1016\/j.neucom.2021.03.132","article-title":"Multi-modal entity alignment in hyperbolic space","volume":"461","author":"Guo","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_33","first-page":"10","article-title":"Graph attention networks","volume":"1050","author":"Velickovic","year":"2017","journal-title":"Stat"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, C., Cao, Y., Hou, L., Shi, J., Li, J., and Chua, T.S. (2019). Semi-Supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-Graph Model, Association for Computational Linguistics.","DOI":"10.18653\/v1\/D19-1274"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Xia, L., Mi, S., Zhang, J., Luo, J., Shen, Z., and Cheng, Y. (2023). Dual-Stream Feature Extraction Network Based on CNN and Transformer for Building Extraction. Remote Sens., 15.","DOI":"10.3390\/rs15102689"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"062032","DOI":"10.1088\/1742-6596\/1087\/6\/062032","article-title":"Feature extraction and image recognition with convolutional neural networks","volume":"1087","author":"Liu","year":"2018","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_37","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_38","first-page":"758","article-title":"Adaptive Feature Fusion for Multi-modal Entity Alignment","volume":"50","author":"Guo","year":"2024","journal-title":"Acta Autom. Sin."},{"key":"ref_39","unstructured":"Guo, L., Chen, Z., Chen, J., and Chen, H. (2023). Revisit and outstrip entity alignment: A perspective of generative models. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Li, Q., Guo, S., Luo, Y., Ji, C., Wang, L., Sheng, J., and Li, J. (May, January 30). Attribute-consistent knowledge graph representation learning for multi-modal entity alignment. Proceedings of the ACM Web Conference 2023, Austin, TX, USA.","DOI":"10.1145\/3543507.3583328"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Li, Q., Ji, C., Guo, S., Liang, Z., Wang, L., and Li, J. (2023). Multi-modal knowledge graph transformer framework for multi-modal entity alignment. arXiv.","DOI":"10.18653\/v1\/2023.findings-emnlp.70"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhang, X., Chen, T., and Wang, H. (2024). A Novel Method for Boosting Knowledge Representation Learning in Entity Alignment through Triple Confidence. Mathematics, 12.","DOI":"10.3390\/math12081214"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"106479","DOI":"10.1016\/j.neunet.2024.106479","article-title":"Triplet-aware graph neural networks for factorized multi-modal knowledge graph entity alignment","volume":"179","author":"Li","year":"2024","journal-title":"Neural Netw."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/7\/990\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:56:56Z","timestamp":1760032616000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/7\/990"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,23]]},"references-count":43,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2025,7]]}},"alternative-id":["sym17070990"],"URL":"https:\/\/doi.org\/10.3390\/sym17070990","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,23]]}}}