{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T06:57:20Z","timestamp":1775199440707,"version":"3.50.1"},"reference-count":103,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,4,21]],"date-time":"2025-04-21T00:00:00Z","timestamp":1745193600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000038","name":"NSERC (Natural Sciences and Engineering Research Council) Discovery Grant","doi-asserted-by":"publisher","award":["RGPIN-2020-05869"],"award-info":[{"award-number":["RGPIN-2020-05869"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>The reciprocal relationship between Large Language Models (LLMs) and Knowledge Graphs (KGs) highlights their synergistic potential in enhancing artificial intelligence (AI) applications. LLMs, with their natural language understanding and generative capabilities, support the automation of KG construction through entity recognition, relation extraction, and schema generation. Conversely, KGs serve as structured and interpretable data sources that improve the transparency, factual consistency and reliability of LLM-based applications, mitigating challenges such as hallucinations and lack of explainability. This study conducts a systematic literature review of 77 studies to examine AI methodologies supporting LLM\u2013KG integration, including symbolic AI, machine learning, and hybrid approaches. The research explores diverse applications spanning healthcare, finance, justice, and industrial automation, revealing the transformative potential of this synergy. Through in-depth analysis, this study identifies key limitations in current approaches, including challenges in scalability with maintaining dynamic and real-time Knowledge Graphs, difficulty in adapting general-purpose LLMs to specialized domains, limited explainability in tracing model outputs to interpretable reasoning, and ethical concerns surrounding bias, fairness, and transparency. In response, the study highlights potential strategies to optimize LLM\u2013KG synergy. The findings from this study provide actionable insights for researchers and practitioners aiming for robust, transparent, and adaptive AI systems to enhance knowledge-driven AI applications through LLM\u2013KG integration, further advancing generative AI and explainable AI (XAI) applications.<\/jats:p>","DOI":"10.3390\/make7020038","type":"journal-article","created":{"date-parts":[[2025,4,21]],"date-time":"2025-04-21T20:42:00Z","timestamp":1745268120000},"page":"38","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Knowledge Graphs and Their Reciprocal Relationship with Large Language Models"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-1515-0914","authenticated-orcid":false,"given":"Ramandeep Singh","family":"Dehal","sequence":"first","affiliation":[{"name":"Management Science Department, Cape Breton University, Sydney, NS B1M 1A2, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2198-5973","authenticated-orcid":false,"given":"Mehak","family":"Sharma","sequence":"additional","affiliation":[{"name":"Management Science Department, Cape Breton University, Sydney, NS B1M 1A2, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9557-0043","authenticated-orcid":false,"given":"Enayat","family":"Rajabi","sequence":"additional","affiliation":[{"name":"Management Science Department, Cape Breton University, Sydney, NS B1M 1A2, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Qin, C., Zhang, A., Zhang, Z., Chen, J., Yasunaga, M., and Yang, D. (2023). Is ChatGPT a General-Purpose Natural Language Processing Task Solver?. arXiv.","DOI":"10.18653\/v1\/2023.emnlp-main.85"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Danilevsky, M., Qian, K., Aharonov, R., Katsis, Y., Kawas, B., and Sen, P. (2020). A Survey of the State of Explainable AI for Natural Language Processing. arXiv.","DOI":"10.18653\/v1\/2020.aacl-main.46"},{"key":"ref_3","unstructured":"Pan, J.Z., Razniewski, S., Kalo, J.C., Singhania, S., Chen, J., Dietze, S., Jabeen, H., Omeliyanenko, J., Zhang, W., and Lissandrini, M. (2023). Large Language Models and Knowledge Graphs: Opportunities and Challenges. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.1177\/01655515221112844","article-title":"Knowledge-graph-based explainable AI: A systematic review","volume":"50","author":"Rajabi","year":"2024","journal-title":"J. Inf. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1007\/s44163-024-00175-8","article-title":"A survey on augmenting Knowledge Graphs (KGs) with large language models (LLMs): Models, evaluation metrics, benchmarks, and challenges","volume":"4","author":"Ibrahim","year":"2024","journal-title":"Discov. Artif. Intell."},{"key":"ref_6","unstructured":"Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., Arx, S.v., Bernstein, M.S., Bohg, J., Bosselut, A., and Brunskill, E. (2022). On the Opportunities and Risks of Foundation Models. arXiv."},{"key":"ref_7","unstructured":"Li, D., and Xu, F. (2024). Synergizing Knowledge Graphs with Large Language Models: A Comprehensive Review and Future Prospects. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1007\/s11280-024-01297-w","article-title":"LLMs for Knowledge Graph construction and reasoning: Recent capabilities and future opportunities","volume":"27","author":"Zhu","year":"2024","journal-title":"World Wide Web"},{"key":"ref_9","first-page":"1877","article-title":"Language Models are Few-Shot Learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","unstructured":"Adhikari, A., Wenink, E., van der Waa, J., Bouter, C., Tolios, I., and Raaijmakers, S. (July, January 29). Towards FAIR Explainable AI: A standardized ontology for mapping XAI solutions to use cases, explanations, and AI systems. Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments, New York, NY, USA."},{"key":"ref_11","unstructured":"Ananya, A., Tiwari, S., Mihindukulasooriya, N., Soru, T., Xu, Z., and Moussallem, D. (2025, March 01). Towards Harnessing Large Language Models as Autonomous Agents for Semantic Triple Extraction from Unstructured Text. Available online: https:\/\/ceur-ws.org\/Vol-3747\/text2kg_paper1.pdf."},{"key":"ref_12","unstructured":"Saleh, A.O.M., Tur, G., and Saygin, Y. (2024, January 19\u201320). SG-RAG: Multi-Hop Question Answering with Large Language Models Through Knowledge Graphs. Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP), Trento, NJ, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Trinh, T., Dao, A., Nhung, H.T.H., and Son, H.T. (2024). VieMedKG: Knowledge Graph and Benchmark for Traditional Vietnamese Medicine. bioRxiv.","DOI":"10.1101\/2024.08.07.606195"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Y., Wang, Z., Liu, Y., Wang, L., Liu, L., and Zhou, L. (2024). KARGEN: Knowledge-enhanced Automated Radiology Report Generation Using Large Language Models. arXiv.","DOI":"10.1007\/978-3-031-72086-4_36"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ren, X., Tang, J., Yin, D., Chawla, N., and Huang, C. (2024, January 25\u201329). A Survey of Large Language Models for Graphs. Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain.","DOI":"10.1145\/3637528.3671460"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Petersen, K., Feldt, R., Mujtaba, S., and Mattsson, M. (2008). Systematic Mapping Studies in Software Engineering, BCS Learning & Development.","DOI":"10.14236\/ewic\/EASE2008.8"},{"key":"ref_17","unstructured":"Dehal, R.S. (2025, March 01). Dataset\u2014Reciprocal_Relationship_Of_KGs_And_LLMs. Available online: https:\/\/figshare.com\/articles\/dataset\/Dataset_-_Reciprocal_Relationship_Of_KGs_And_LLMs\/28468637\/1?file=52560449."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2919","DOI":"10.14778\/3681954.3681973","article-title":"Are Large Language Models a Good Replacement of Taxonomies?","volume":"17","author":"Sun","year":"2024","journal-title":"Proc. VLDB Endow."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Venkatakrishnan, R., Tanyildizi, E., and Canbaz, M.A. (2024, January 13\u201317). Semantic interlinking of Immigration Data using LLMs for Knowledge Graph Construction. Proceedings of the Companion Proceedings of the ACM Web Conference 2024, Singapore.","DOI":"10.1145\/3589335.3651557"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Huang, Y., and Zeng, G. (2024, January 21\u201325). RD-P: A Trustworthy Retrieval-Augmented Prompter with Knowledge Graphs for LLMs. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, New York, NY, USA.","DOI":"10.1145\/3627673.3679659"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Colombo, A. (2024, January 21\u201325). Leveraging Knowledge Graphs and LLMs to Support and Monitor Legislative Systems. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, ID, USA.","DOI":"10.1145\/3627673.3680268"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1145\/3655103.3655110","article-title":"Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs","volume":"25","author":"Chen","year":"2024","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wei, W., Ren, X., Tang, J., Wang, Q., Su, L., Cheng, S., Wang, J., Yin, D., and Huang, C. (2024, January 4\u20138). LLMRec: Large Language Models with Graph Augmentation for Recommendation. Proceedings of the 17th ACM International Conference on Web Search and Data Mining, Merida, Mexico.","DOI":"10.1145\/3616855.3635853"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"4130","DOI":"10.14778\/3611540.3611636","article-title":"Generations of Knowledge Graphs: The Crazy Ideas and the Business Impact","volume":"16","author":"Dong","year":"2023","journal-title":"Proc. VLDB Endow."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wu, Q., and Wang, Y. (2023, January 16\u201317). Research on Intelligent Question-Answering Systems Based on Large Language Models and Knowledge Graphs. Proceedings of the 2023 16th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China.","DOI":"10.1109\/ISCID59865.2023.00045"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Fieblinger, R., Alam, M.T., and Rastogi, N. (2024, January 8\u201312). Actionable Cyber Threat Intelligence Using Knowledge Graphs and Large Language Models. Proceedings of the 2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), Vienna, Austria.","DOI":"10.1109\/EuroSPW61312.2024.00018"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Vizcarra, J., Haruta, S., and Kurokawa, M. (2024, January 5\u20137). Representing the Interaction between Users and Products via LLM-assisted Knowledge Graph Construction. Proceedings of the 2024 IEEE 18th International Conference on Semantic Computing (ICSC), Laguna Hills, CA, USA.","DOI":"10.1109\/ICSC59802.2024.00043"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kosten, C., Cudr\u00e9-Mauroux, P., and Stockinger, K. (2023, January 15\u201318). Spider4SPARQL: A Complex Benchmark for Evaluating Knowledge Graph Question Answering Systems. Proceedings of the 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy.","DOI":"10.1109\/BigData59044.2023.10386182"},{"key":"ref_29","unstructured":"Taffa, T.A., and Usbeck, R. (2023). Leveraging LLMs in Scholarly Knowledge Graph Question Answering. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Ding, Z., Cai, H., Wu, J., Ma, Y., Liao, R., Xiong, B., and Tresp, V. (2024). zrLLM: Zero-Shot Relational Learning on Temporal Knowledge Graphs with Large Language Models. arXiv.","DOI":"10.18653\/v1\/2024.naacl-long.104"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chen, L., Xu, J., Wu, T., and Liu, J. (2024). Information Extraction of Aviation Accident Causation Knowledge Graph: An LLM-Based Approach. Electronics, 13.","DOI":"10.3390\/electronics13193936"},{"key":"ref_32","first-page":"3936","article-title":"Framing Few-Shot Knowledge Graph Completion with Large Language Models","volume":"13","author":"Nixon","year":"2024","journal-title":"Electronics"},{"key":"ref_33","unstructured":"Jiang, L., Yan, X., and Usbeck, R. (2025, March 01). A Structure and Content Prompt-Based Method for Knowledge Graph Question Answering over Scholarly Data. Available online: https:\/\/ceur-ws.org\/Vol-3592\/paper3.pdf."},{"key":"ref_34","unstructured":"Mohanty, A. (2025, March 01). EduEmbedd\u2014A Knowledge Graph Embedding for Education. Available online: https:\/\/ceur-ws.org\/Vol-3532\/paper1.pdf."},{"key":"ref_35","unstructured":"Pliukhin, D., Radyush, D., Kovriguina, L., and Mouromtsev, D. (2025, March 01). Improving Subgraph Extraction Algorithms for One-Shot SPARQL Query Generation with Large Language Models. Available online: https:\/\/ceur-ws.org\/Vol-3592\/paper6.pdf."},{"key":"ref_36","unstructured":"Pawar, J.D., and Lalitha Devi, S. (2023, January 14\u201317). Infusing Knowledge into Large Language Models with Contextual Prompts. Proceedings of the 20th International Conference on Natural Language Processing (ICON), Goa, India."},{"key":"ref_37","unstructured":"Schmidt, W.J., Rincon-Yanez, D., Kharlamov, E., and Paschke, A. (2025, March 01). Scaling Scientific Knowledge Discovery with Neuro-Symbolic AI and Large Language Models. Available online: https:\/\/publica.fraunhofer.de\/entities\/publication\/a752f6fb-4cc0-46cc-9312-b6eff7f64334."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Liu, S., and Fang, Y. (2023). Use Large Language Models for Named Entity Disambiguation in Academic Knowledge Graphs, Atlantis Press.","DOI":"10.2991\/978-94-6463-264-4_79"},{"key":"ref_39","unstructured":"Momii, Y., Takiguchi, T., and Ariki, Y. (2025, March 01). Rule-based Fact Verification Utilizing Knowledge Graphs. Available online: https:\/\/www.jstage.jst.go.jp\/article\/jsaislud\/99\/0\/99_51\/_article\/-char\/en."},{"key":"ref_40","unstructured":"de Paiva, V., Gao, Q., Kovalev, P., and Moss, L.S. (2025, March 01). Extracting Mathematical Concepts with Large Language Models. Available online: https:\/\/cicm-conference.org\/2023\/mathui\/mathuiPubs\/CICM_2023_paper_8826.pdf."},{"key":"ref_41","unstructured":"Thie\u00dfen, F., D\u2019Souza, J., and Stocker, M. (2025, March 01). Probing Large Language Models for Scientific Synonyms. Available online: https:\/\/ceur-ws.org\/Vol-3510\/paper_nlp_2.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Wang, F., Shi, D., Aguilar, J., Cui, X., Jiang, J., Shen, L., and Li, M. (2025, March 01). LLM-KGMQA: Large Language Model-Augmented Multi-Hop Question-Answering System Based on Knowledge Graph in Medical Field. Available online: https:\/\/www.researchsquare.com\/article\/rs-4721418\/v1.","DOI":"10.1007\/s10115-025-02399-1"},{"key":"ref_43","first-page":"56","article-title":"Integrated Application of LLM Model and Knowledge Graph in Medical Text Mining and Knowledge Extraction","volume":"5","author":"Yang","year":"2024","journal-title":"Soc. Med. Health Manag."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Li, Q., Chen, Z., Ji, C., Jiang, S., and Li, J. (2024, January 3\u20139). LLM-based Multi-Level Knowledge Generation for Few-shot Knowledge Graph Completion. Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence, Jeju, Republic of Korea.","DOI":"10.24963\/ijcai.2024\/236"},{"key":"ref_45","first-page":"148","article-title":"An LLM-Aided Enterprise Knowledge Graph (EKG) Engineering Process","volume":"3","author":"Laurenzi","year":"2024","journal-title":"Proc. AAAI Symp. Ser."},{"key":"ref_46","unstructured":"Gillani, K., Novak, E., Kenda, K., and Mladeni\u0107, D. (2025, March 01). Knowledge Graph Extraction from Textual Data Using LLM. Available online: https:\/\/is.ijs.si\/wp-content\/uploads\/2024\/10\/IS2024_-_SIKDD_2024_paper_15-1.pdf."},{"key":"ref_47","unstructured":"Gupta, T.K., Goel, T., Verma, I., Dey, L., and Bhardwaj, S. (2025, March 01). Knowledge Graph Aided LLM Based ESG Question-Answering from News. Available online: https:\/\/ceur-ws.org\/Vol-3753\/paper6.pdf."},{"key":"ref_48","unstructured":"Ghanem, H., and Cruz, C. (2025, March 01). Fine-Tuning vs. Prompting: Evaluating the Knowledge Graph Construction with LLMs. Available online: https:\/\/hal.science\/hal-04862235\/."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Biswas, R., Kaffee, L.A., Agarwal, O., Minervini, P., Singh, S., and de Melo, G. (2024, January 15). Multi-hop Database Reasoning with Virtual Knowledge Graph. Proceedings of the 1st Workshop on Knowledge Graphs and Large Language Models (KaLLM 2024), Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.kallm-1.1"},{"key":"ref_50","unstructured":"Ventura de los Ojos, X. (2024). Application of LLM-Augmented Knowledge Graphs for Wirearchy Management, Universitat Oberta de Catalunya (UOC)."},{"key":"ref_51","first-page":"82","article-title":"GLaM: Fine-Tuning Large Language Models for Domain Knowledge Graph Alignment via Neighborhood Partitioning and Generative Subgraph Encoding","volume":"3","author":"Dernbach","year":"2024","journal-title":"Proc. AAAI Symp. Ser."},{"key":"ref_52","unstructured":"Reitemeyer, B., and Fill, H.G. (2024). Leveraging LLMs in Semantic Mapping for Knowledge Graph-Based Automated Enterprise Model Generation, Gesellschaft f\u00fcr Informatik e.V."},{"key":"ref_53","first-page":"61:1","article-title":"Knowledge Graph Builder\u2014Constructing a Graph from Arbitrary Text Using an LLM","volume":"Volume 320","author":"Felsner","year":"2024","journal-title":"Proceedings of the 32nd International Symposium on Graph Drawing and Network Visualization (GD 2024)"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Li, V.X., and Tan, Y. (2025, March 01). Dynamic Knowledge Graph Asset Pricing. Available online: https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=4841921.","DOI":"10.2139\/ssrn.4841921"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Tufek, N., Saissre, A., Just, V.P., Ekaputra, F.J., Sabou, M., and Hanbury, A. (2024). Validating Semantic Artifacts with Large Language Models. European Semantic Web Conference, Springer Nature.","DOI":"10.1007\/978-3-031-78952-6_9"},{"key":"ref_56","unstructured":"Gan, L., Blum, M., Dess\u00ed, D., Mathiak, B., Schenkel, R., and Dietze, S. (2024). Hidden Entity Detection from GitHub Leveraging Large Language Models. arXiv."},{"key":"ref_57","first-page":"188","article-title":"Semantic Verification in Large Language Model-based Retrieval Augmented Generation","volume":"3","author":"Martin","year":"2024","journal-title":"Proc. AAAI Symp. Ser."},{"key":"ref_58","unstructured":"Li, S., Li, M., Zhang, M.J., Choi, E., Geva, M., Hase, P., and Ji, H. (2024, January 16). Patent Response System Optimised for Faithfulness: Procedural Knowledge Embodiment with Knowledge Graph and Retrieval Augmented Generation. Proceedings of the 1st Workshop on Towards Knowledgeable Language Models (KnowLLM 2024), Bangkok, Thailand."},{"key":"ref_59","unstructured":"Dobriy, D. (2025, March 01). Employing RAG to Create a Conference Knowledge Graph from Text. Available online: https:\/\/ceur-ws.org\/Vol-3747\/text2kg_paper4.pdf."},{"key":"ref_60","unstructured":"Seif, A., Toh, S., and Lee, H.K. (2025, March 01). A Dynamic Jobs-Skills Knowledge Graph. Available online: https:\/\/recsyshr.aau.dk\/wp-content\/uploads\/2024\/10\/RecSysHR2024-paper_1.pdf."},{"key":"ref_61","unstructured":"Hosseini-Kivanani, N., H\u00f6hn, S., Anastasiou, D., Migge, B., Soltan, A., Dippold, D., Kamlovskaya, E., and Philippy, F. (2024, January 22). Socio-cultural adapted chatbots: Harnessing Knowledge Graphs and Large Language Models for enhanced context awarenes. Proceedings of the 1st Worskhop on Towards Ethical and Inclusive Conversational AI: Language Attitudes, Linguistic Diversity, and Language Rights (TEICAI 2024), St. Julian\u2019s, Malta."},{"key":"ref_62","unstructured":"Daga, E., Carvalho, J., and Morales Tirado, A. (2025, March 01). Extracting Licence Information from Web Resources with a Large Language Model, Heraklion, Greece. Available online: https:\/\/oro.open.ac.uk\/97612\/."},{"key":"ref_63","unstructured":"D\u2019Souza, J., and Mihindukulasooriya, N. (2025, March 01). The State of the Art Large Language Models for Knowledge Graph Construction from Text: Techniques, Tools, and Challenges. Available online: https:\/\/research.ibm.com\/publications\/the-state-of-the-art-large-language-models-for-knowledge-graph-construction-from-text-techniques-tools-and-challenges."},{"key":"ref_64","unstructured":"Almeida, J.P.A., Di Ciccio, C., and Kalloniatis, C. (2024, January 3\u20137). LLMs for Knowledge-Graphs Enhanced Task-Oriented Dialogue Systems: Challenges and Opportunities. Proceedings of the Advanced Information Systems Engineering Workshops, Limassol, Cyprus."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Zhao, Q., Qian, H., Liu, Z., Zhang, G.D., and Gu, L. (2024, January 21\u201325). Breaking the Barrier: Utilizing Large Language Models for Industrial Recommendation Systems through an Inferential Knowledge Graph. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, ID, USA.","DOI":"10.1145\/3627673.3680022"},{"key":"ref_66","unstructured":"Zhang, Y., Chen, Z., Guo, L., Xu, Y., Zhang, W., and Chen, H. (November, January 28). Making Large Language Models Perform Better in Knowledge Graph Completion. Proceedings of the 32nd ACM International Conference on Multimedia, Melbourne, VIC, Australia."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Liu, L., Wang, Z., Bai, J., Song, Y., and Tong, H. (2024, January 13\u201317). New Frontiers of Knowledge Graph Reasoning: Recent Advances and Future Trends. Proceedings of the Companion Proceedings of the ACM Web Conference 2024, Singapore.","DOI":"10.1145\/3589335.3641254"},{"key":"ref_68","first-page":"3657305","article-title":"Zero-Shot Construction of Chinese Medical Knowledge Graph with GPT-3.5-turbo and GPT-4","volume":"16","author":"Wu","year":"2024","journal-title":"ACM Trans. Manag. Inf. Syst."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Bui, T., Tran, O., Nguyen, P., Ho, B., Nguyen, L., Bui, T., and Quan, T. (2024, January 14). Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A Case Study at HCMUT. Proceedings of the 1st ACM Workshop on AI-Powered Q&A Systems for Multimedia, Phuket, Thailand.","DOI":"10.1145\/3643479.3662055"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Jiang, Y., Yao, J., Li, F., and Zhang, Y. (2024, January 12\u201314). Research on Engineering Management Question-answering System in the Communication Industry Based on Large Language Models and Knowledge Graphs. Proceedings of the 2024 7th International Conference on Machine Vision and Applications, Singapore.","DOI":"10.1145\/3653946.3653961"},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Le, D., Zhao, K., Wang, M., and Wu, Y. (2024, January 13\u201316). GraphLingo: Domain Knowledge Exploration by Synchronizing Knowledge Graphs and Large Language Models. Proceedings of the 2024 IEEE 40th International Conference on Data Engineering (ICDE), Utrecht, The Netherlands.","DOI":"10.1109\/ICDE60146.2024.00432"},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"3580","DOI":"10.1109\/TKDE.2024.3352100","article-title":"Unifying Large Language Models and Knowledge Graphs: A Roadmap","volume":"36","author":"Pan","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Li, D., and Xu, F. (2024, January 29\u201331). The Deep Integration of Knowledge Graphs and Large Language Models: Advancements, Challenges, and Future Directions. Proceedings of the 2024 IEEE 2nd International Conference on Sensors, Electronics and Computer Engineering (ICSECE), Jinzhou, China.","DOI":"10.1109\/ICSECE61636.2024.10729340"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Abu-Rasheed, H., Weber, C., and Fathi, M. (2024, January 8\u201311). Knowledge Graphs as Context Sources for LLM-Based Explanations of Learning Recommendations. Proceedings of the 2024 IEEE Global Engineering Education Conference (EDUCON), Kos, Greece.","DOI":"10.1109\/EDUCON60312.2024.10578654"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"162638","DOI":"10.1109\/ACCESS.2024.3485877","article-title":"ChatTf: A Knowledge Graph-Enhanced Intelligent Q&A System for Mitigating Factuality Hallucinations in Traditional Folklore","volume":"12","author":"Xu","year":"2024","journal-title":"IEEE Access"},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"6962","DOI":"10.1109\/TKDE.2024.3399746","article-title":"Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI","volume":"36","author":"Song","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_77","first-page":"103","article-title":"Connecting AI: Merging Large Language Models and Knowledge Graph","volume":"56","author":"Campbell","year":"2023","journal-title":"Computer"},{"key":"ref_78","unstructured":"Demner-Fushman, D., Ananiadou, S., Thompson, P., and Ondov, B. Towards Using Automatically Enhanced Knowledge Graphs to Aid Temporal Relation Extraction. Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024."},{"key":"ref_79","doi-asserted-by":"crossref","unstructured":"Cao, X., Xu, W., Zhao, J., Duan, Y., and Yang, X. (2024). Research on Large Language Model for Coal Mine Equipment Maintenance Based on Multi-Source Text. Appl. Sci., 14.","DOI":"10.3390\/app14072946"},{"key":"ref_80","unstructured":"Schneider, P., Klettner, M., Simperl, E., and Matthes, F. (2024). A Comparative Analysis of Conversational Large Language Models in Knowledge-Based Text Generation. arXiv."},{"key":"ref_81","unstructured":"Huang, D.S., Si, Z., and Chen, W. (2024, January 5\u20138). Enhancing Retrieval-Augmented Generation Models with Knowledge Graphs: Innovative Practices Through a Dual-Pathway Approach. Proceedings of the Advanced Intelligent Computing Technology and Applications, Tianjin, China."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Huang, Q., Wan, Z., Xing, Z., Wang, C., Chen, J., Xu, X., and Lu, Q. (2023, January 11\u201315). Let\u2019s Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain. Proceedings of the 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE), Luxembourg.","DOI":"10.1109\/ASE56229.2023.00075"},{"key":"ref_83","doi-asserted-by":"crossref","unstructured":"Sequeda, J., Allemang, D., and Jacob, B. (2024, January 14). A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model\u2019s Accuracy for Question Answering on Enterprise SQL Databases. Proceedings of the 7th Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA), Santiago, Chile.","DOI":"10.1145\/3661304.3661901"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Fu, L., Guan, H., Du, K., Lin, J., Xia, W., Zhang, W., Tang, R., Wang, Y., and Yu, Y. (2024, January 21\u201325). SINKT: A Structure-Aware Inductive Knowledge Tracing Model with Large Language Model. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, Boise, ID, USA.","DOI":"10.1145\/3627673.3679760"},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"843","DOI":"10.26599\/BDMA.2024.9020026","article-title":"Prompting Large Language Models with Knowledge-Injection for Knowledge-Based Visual Question Answering","volume":"7","author":"Hu","year":"2024","journal-title":"Big Data Min. Anal."},{"key":"ref_86","first-page":"23164","article-title":"CyberQ: Generating Questions and Answers for Cybersecurity Education Using Knowledge Graph-Augmented LLMs","volume":"38","author":"Agrawal","year":"2024","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_87","doi-asserted-by":"crossref","unstructured":"Hertling, S., and Paulheim, H. (2023, January 5\u20137). OLaLa: Ontology Matching with Large Language Models. Proceedings of the 12th Knowledge Capture Conference 2023, Pensacola, FL, USA.","DOI":"10.1145\/3587259.3627571"},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Cadeddu, A., Chessa, A., De Leo, V., Fenu, G., Motta, E., Osborne, F., Reforgiato Recupero, D., Salatino, A., and Secchi, L. (2024). Optimizing Tourism Accommodation Offers by Integrating Language Models and Knowledge Graph Technologies. Information, 15.","DOI":"10.3390\/info15070398"},{"key":"ref_89","doi-asserted-by":"crossref","unstructured":"Hello, N., Di Lorenzo, P., and Strinati, E.C. (2024, January 10\u201313). Semantic Communication Enhanced by Knowledge Graph Representation Learning. Proceedings of the 2024 IEEE 25th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Lucca, Italy.","DOI":"10.1109\/SPAWC60668.2024.10694291"},{"key":"ref_90","first-page":"1327","article-title":"HybridGCN: An Integrative Model for Scalable Recommender Systems with Knowledge Graph and Graph Neural Networks","volume":"15","author":"Nguyen","year":"2024","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_91","unstructured":"Huang, D.S., Pan, Y., and Guo, J. (2024, January 5\u20138). Self-consistency, Extract and Rectify: Knowledge Graph Enhance Large Language Model for Electric Power Question Answering. Proceedings of the Advanced Intelligent Computing Technology and Applications, Tianjin, China."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Wu, L.I., and Li, G. (2023, January 18\u201319). Zero-Shot Construction of Chinese Medical Knowledge Graph with ChatGPT. Proceedings of the 2023 IEEE International Conference on Medical Artificial Intelligence (MedAI), Beijing, China.","DOI":"10.1109\/MedAI59581.2023.00043"},{"key":"ref_93","doi-asserted-by":"crossref","unstructured":"Procko, T.T., and Ochoa, O. (October, January 30). Graph Retrieval-Augmented Generation for Large Language Models: A Survey. Proceedings of the 2024 Conference on AI, Science, Engineering, and Technology (AIxSET), Laguna Hills, CA, USA.","DOI":"10.1109\/AIxSET62544.2024.00030"},{"key":"ref_94","doi-asserted-by":"crossref","unstructured":"Su, Y., Liao, D., Xing, Z., Huang, Q., Xie, M., Lu, Q., and Xu, X. (2024, January 14\u201320). Enhancing Exploratory Testing by Large Language Model and Knowledge Graph. Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering, Lisbon, Portugal.","DOI":"10.1145\/3597503.3639157"},{"key":"ref_95","doi-asserted-by":"crossref","first-page":"8622","DOI":"10.1109\/TKDE.2024.3469578","article-title":"Large Language Models on Graphs: A Comprehensive Survey","volume":"36","author":"Jin","year":"2024","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_96","doi-asserted-by":"crossref","unstructured":"Procko, T.T., Elvira, T., and Ochoa, O. (2023, January 25\u201327). GPT-4: A Stochastic Parrot or Ontological Craftsman? Discovering Implicit Knowledge Structures in Large Language Models. Proceedings of the 2023 Fifth International Conference on Transdisciplinary AI (TransAI), Laguna Hills, CA, USA.","DOI":"10.1109\/TransAI60598.2023.00043"},{"key":"ref_97","doi-asserted-by":"crossref","unstructured":"Sun, Y., Yang, W., and Liu, Y. (2024, January 24\u201326). The Application of Constructing Knowledge Graph of Oral Historical Archives Resources Based on LLM-RAG. Proceedings of the 2024 8th International Conference on Information System and Data Mining, New York, NY, USA.","DOI":"10.1145\/3686397.3686420"},{"key":"ref_98","doi-asserted-by":"crossref","unstructured":"Wang, H., Han, X., Liu, M., Cheng, G., Liu, Y., and Zhang, N. (2023, January 24\u201327). Improving Adaptive Knowledge Graph Construction via Large Language Models with Multiple Views. Proceedings of the Knowledge Graph and Semantic Computing: Knowledge Graph Empowers Artificial General Intelligence, Shenyang, China.","DOI":"10.1007\/978-981-99-7224-1"},{"key":"ref_99","unstructured":"Khorashadizadeh, H., Amara, F.Z., Ezzabady, M., Ieng, F., Tiwari, S., Mihindukulasooriya, N., Groppe, J., Sahri, S., Benamara, F., and Groppe, S. (2024). Research Trends for the Interplay between Large Language Models and Knowledge Graphs. arXiv."},{"key":"ref_100","doi-asserted-by":"crossref","unstructured":"Dehal, R.S., Sharma, M., and de Souza Santos, R. (2024, January 14\u201320). Exposing Algorithmic Discrimination and Its Consequences in Modern Society: Insights from a Scoping Study. Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Society, New York, NY, USA.","DOI":"10.1145\/3639475.3640098"},{"key":"ref_101","unstructured":"Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., and Tang, S. (2024). Graph Retrieval-Augmented Generation: A Survey. arXiv."},{"key":"ref_102","unstructured":"Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., Zhou, L., Duan, N., Svyatkovskiy, A., and Fu, S. (2021). GraphCodeBERT: Pre-training Code Representations with Data Flow. arXiv."},{"key":"ref_103","first-page":"10436","article-title":"Enhanced Story Comprehension for Large Language Models through Dynamic Document-Based Knowledge Graphs","volume":"36","author":"Andrus","year":"2022","journal-title":"Proc. Aaai Conf. Artif. Intell."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/38\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:18:45Z","timestamp":1760030325000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/38"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,21]]},"references-count":103,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020038"],"URL":"https:\/\/doi.org\/10.3390\/make7020038","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,21]]}}}