{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T03:50:45Z","timestamp":1784692245443,"version":"3.55.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"2","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMOD Rec."],"published-print":{"date-parts":[[2025,7,14]]},"abstract":"<jats:p>The unification of large language models (LLMs) and knowledge graphs (KGs) has emerged as a hot topic. At the LLM+KG'24 workshop, co-located with VLDB 2024 in Guangzhou, China, the key theme explored was important data management challenges and opportunities due to the effective interaction between LLMs and KGs. The report outlines major directions and approaches presented by various speakers during the workshop.<\/jats:p>","DOI":"10.1145\/3749116.3749132","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T23:58:41Z","timestamp":1752537521000},"page":"60-65","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["LLM+KG@VLDB 24 Workshop Summary"],"prefix":"10.1145","volume":"54","author":[{"given":"Arijit","family":"Khan","sequence":"first","affiliation":[{"name":"Aalborg University, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianxing","family":"Wu","sequence":"additional","affiliation":[{"name":"Southeast University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"Platform and Content Group, Tencent, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,7,14]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"LlamaIndex Workshop. https:\/\/colab.research.google.com\/drive\/ 1tLjOg2ZQuIClfuWrAC2LdiZHCov8oUbs.","unstructured":"GraphRAG LlamaIndex Workshop. https:\/\/colab.research.google.com\/drive\/ 1tLjOg2ZQuIClfuWrAC2LdiZHCov8oUbs."},{"key":"e_1_2_1_2_1","unstructured":"LLM+Graph: The Second International Workshop on Data Management Opportunities in Bringing LLMS with Graph Data. https:\/\/seucoin.github.io\/workshop\/llmg2025\/."},{"key":"e_1_2_1_3_1","unstructured":"NebulaGraph. https:\/\/github.com\/vesoft-inc\/nebula."},{"key":"e_1_2_1_4_1","unstructured":"OpenKG. https:\/\/github.com\/OpenKG-ORG."},{"key":"e_1_2_1_5_1","volume-title":"'24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning. Association for Computing Machinery","author":"DEEM","year":"2024","unstructured":"DEEM '24: Proceedings of the Eighth Workshop on Data Management for End-to-End Machine Learning. Association for Computing Machinery, 2024."},{"key":"e_1_2_1_6_1","volume-title":"Semantic-enhanced Programmable Knowledge Graph (SPG) White paper (v1.0). https:\/\/spg.openkg.cn\/en-US","author":"Ant Group and OpenKG.","year":"2023","unstructured":"Ant Group and OpenKG. Semantic-enhanced Programmable Knowledge Graph (SPG) White paper (v1.0). https:\/\/spg.openkg.cn\/en-US, 2023."},{"key":"e_1_2_1_7_1","volume-title":"NLRSE@ACL","author":"Baek J.","year":"2023","unstructured":"J. Baek, A. F. Aji, and A. Saffari. Knowledge-Augmented Language Model Prompting for Zero-Shot Knowledge Graph Question Answering. In NLRSE@ACL, 2023."},{"key":"e_1_2_1_8_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Cavalleri E.","year":"2024","unstructured":"E. Cavalleri, M. Soto-Gomez, A. Pashaeibarough, D. Malchiodi, H. Caufield, J. Reese, C. J. Mungall, P. N. Robinson, E. Casiraghi, G. Valentini, and M. Mesiti. SPIREX: Improving LLM-based Relation Extraction from RNA-focused Scientific Literature using Graph Machine Learning. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_9_1","volume-title":"Knowledge Graphs, and Vector Databases: Synergy and Opportunities for Data Management (A Report on the LLM+KG@VLDB24 Workshop's Panel Discussion). https:\/\/wp.sigmod.org\/?p=3813","author":"Chen X.","year":"2024","unstructured":"X. Chen, W. Hu, A. Khan, S. Shankar, H. Wang, J. Wang, and T. Wu. Large Language Models, Knowledge Graphs, and Vector Databases: Synergy and Opportunities for Data Management (A Report on the LLM+KG@VLDB24 Workshop's Panel Discussion). https:\/\/wp.sigmod.org\/?p=3813, 2024."},{"key":"e_1_2_1_10_1","volume-title":"From Local to Global: A Graph RAG Approach to Query-Focused Summarization. CoRR, abs\/2404.16130","author":"Edge D.","year":"2024","unstructured":"D. Edge, H. Trinh, N. Cheng, J. Bradley, A. Chao, A. Mody, S. Truitt, and J. Larson. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. CoRR, abs\/2404.16130, 2024."},{"key":"e_1_2_1_11_1","unstructured":"E. Groves M. Wang Y. Abdulle H. Kunz J. Hoelscher-Obermaier R. Wu and H. Wu. Benchmarking and Analyzing In-Context Learning Fine-tuning and Supervised Learning for Biomedical Knowledge Curation: A Focused Study on Chemical Entities of Biological Interest. In Workshops at the International Conference on Very Large Data Bases (VLDB) 2024."},{"key":"e_1_2_1_12_1","first-page":"8145","volume-title":"ICML","author":"Guo L.","year":"2022","unstructured":"L. Guo, Q. Zhang, Z. Sun, M. Chen, W. Hu, and H. Chen. Understanding and Improving Knowledge Graph Embedding for Entity Alignment. In ICML, pages 8145--8156, 2022."},{"key":"e_1_2_1_13_1","volume-title":"HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. CoRR, abs\/2405.14831","author":"Guti\u00e9rrez B. J.","year":"2024","unstructured":"B. J. Guti\u00e9rrez, Y. Shu, Y. Gu, M. Yasunaga, and Y. Su. HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models. CoRR, abs\/2405.14831, 2024."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3587259.3627571"},{"key":"e_1_2_1_15_1","volume-title":"Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs. CoRR, abs\/2401.14640","author":"Hu N.","year":"2024","unstructured":"N. Hu, J. Chen, Y. Wu, G. Qi, S. Bi, T. Wu, and J. Z. Pan. Benchmarking Large Language Models in Complex Question Answering Attribution using Knowledge Graphs. CoRR, abs\/2401.14640, 2024."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-022-00745-1"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589304"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.408"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599885"},{"key":"e_1_2_1_20_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Khan A.","year":"2024","unstructured":"A. Khan, T. Wu, and X. Chen. LLM+KG: Data Management Opportunities in Unifying Large Language Models + Knowledge Graphs. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_21_1","volume-title":"International Conference on Learning Representations (ICLR)","author":"Khandelwal U.","year":"2020","unstructured":"U. Khandelwal, O. Levy, D. Jurafsky, L. Zettlemoyer, and M. Lewis. Generalization through Memorization: Nearest Neighbor Language Models. In International Conference on Learning Representations (ICLR), 2020."},{"key":"e_1_2_1_22_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Khorashadizadeh H.","year":"2024","unstructured":"H. Khorashadizadeh, F. Z. Amara, M. Ezzabady, F. Ieng, S. Tiwari, N. Mihindukulasooriya, J. Groppe, S. Sahri, and F. Benamara. Research Trends for the Interplay between Large Language Models and Knowledge Graphs. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_23_1","volume-title":"Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering. CoRR, abs\/2403.02966","author":"Ko S.","year":"2024","unstructured":"S. Ko, H. Cho, H. Chae, J. Yeo, and D. Lee. Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering. CoRR, abs\/2403.02966, 2024."},{"key":"e_1_2_1_24_1","volume-title":"A Declarative System for Optimizing AI Workloads. CoRR, abs\/2405.14696","author":"Liu C.","year":"2024","unstructured":"C. Liu, M. Russo, M. J. Cafarella, L. Cao, P. B. Chen, Z. Chen, M. J. Franklin, T. Kraska, S. Madden, and G. Vitagliano. A Declarative System for Optimizing AI Workloads. CoRR, abs\/2405.14696, 2024."},{"key":"e_1_2_1_25_1","volume-title":"TigerVector: Supporting Vector Search in Graph Databases for Advanced RAGs. CoRR, 2501.11216","author":"Liu S.","year":"2025","unstructured":"S. Liu, Z. Zeng, L. Chen, A. Ainihaer, A. Ramasami, S. Chen, Y. Xu, M. Wu, and J. Wang. TigerVector: Supporting Vector Search in Graph Databases for Advanced RAGs. CoRR, 2501.11216, 2025."},{"key":"e_1_2_1_26_1","volume-title":"Enhancing Large Language Models with Multimodality and Knowledge Graphs for Hallucination-free Open-set Object Recognition. In Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Liu X.","year":"2024","unstructured":"X. Liu, Y. Wu, Y. Zhou, J. Chen, H. Wang, Y. Liu, and S. Wan. Enhancing Large Language Models with Multimodality and Knowledge Graphs for Hallucination-free Open-set Object Recognition. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-25198-6_9"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-long.396"},{"key":"e_1_2_1_29_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Mou Y.","year":"2024","unstructured":"Y. Mou, L. Liu, S. Sowe, D. Collarana, and S. Decker. Leveraging LLMs Few-shot Learning to Improve Instruction-driven Knowledge Graph Construction. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_30_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Mustafa D. M.","year":"2024","unstructured":"D. M. Mustafa, A. Nadgeri, D. Collarana, B. T. Arnold, C. Quix, C. Lange, and S. Decker. From Instructions to ODRL Usage Policies: An Ontology Guided Approach. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_31_1","volume-title":"NebulaGraph Launches Industry-First Graph RAG: Retrieval-Augmented Generation with LLM Based on Knowledge Graphs. https:\/\/www.nebula-graph.io\/posts\/graph-RAG","year":"2023","unstructured":"NebulaGraph. NebulaGraph Launches Industry-First Graph RAG: Retrieval-Augmented Generation with LLM Based on Knowledge Graphs. https:\/\/www.nebula-graph.io\/posts\/graph-RAG, 2023."},{"key":"e_1_2_1_32_1","volume-title":"Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Nie J.","year":"2024","unstructured":"J. Nie, X. Hou, W. Song, X. Wang, X. Zhang, X. Jin, S. Zhang, and J. Shi. Knowledge Graph Efficient Construction: Embedding Chain-of-Thought into LLMs. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3352100"},{"key":"e_1_2_1_34_1","volume-title":"CIDR","author":"Parameswaran A. G.","year":"2024","unstructured":"A. G. Parameswaran, S. Shankar, P. Asawa, N. Jain, and Y. Wang. Revisiting Prompt Engineering via Declarative Crowdsourcing. In CIDR, 2024."},{"key":"e_1_2_1_35_1","volume-title":"LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data. CoRR, abs\/2407.11418","author":"Patel L.","year":"2024","unstructured":"L. Patel, S. Jha, C. Guestrin, and M. Zaharia. LOTUS: Enabling Semantic Queries with LLMs Over Tables of Unstructured and Structured Data. CoRR, abs\/2407.11418, 2024."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1250"},{"key":"e_1_2_1_37_1","volume-title":"KG-RAG: Bridging the Gap Between Knowledge and Creativity. CoRR, abs\/2405.12035","author":"Sanmartin D.","year":"2024","unstructured":"D. Sanmartin. KG-RAG: Bridging the Gap Between Knowledge and Creativity. CoRR, abs\/2405.12035, 2024."},{"key":"e_1_2_1_38_1","volume-title":"Proc. VLDB","author":"Shankar S.","unstructured":"S. Shankar, H. Li, P. Asawa, M. Hulsebos, Y. Lin, J. Zamfirescu-Pereira, H. Chase, W. Fu-Hinthorn, A. G. Parameswaran, and E. Wu. SPADE: Synthesizing Data Quality Assertions for Large Language Model Pipelines. Proc. VLDB"},{"key":"e_1_2_1_39_1","volume-title":"Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph. CoRR, abs\/2307.07697","author":"Sun J.","year":"2023","unstructured":"J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, H. Shum, and J. Guo. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model with Knowledge Graph. CoRR, abs\/2307.07697, 2023."},{"issue":"8","key":"e_1_2_1_40_1","first-page":"8461","article-title":"Revisiting Embedding-Based Entity Alignment","volume":"35","author":"Sun Z.","year":"2023","unstructured":"Z. Sun, W. Hu, C. Wang, Y. Wang, and Y. Qu. Revisiting Embedding-Based Entity Alignment: A Robust and Adaptive Method. IEEE Trans. Knowl. Data Eng., 35(8):8461--8475, 2023.","journal-title":"A Robust and Adaptive Method. IEEE Trans. Knowl. Data Eng."},{"key":"e_1_2_1_41_1","first-page":"32875","volume-title":"ICML","author":"Sun Z.","year":"2023","unstructured":"Z. Sun, J. Huang, X. Xu, Q. Chen, W. Ren, and W. Hu. What Makes Entities Similar? A Similarity Flooding Perspective for Multi-sourced Knowledge Graph Embeddings. In ICML, pages 32875--32885, 2023."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407828"},{"key":"e_1_2_1_43_1","first-page":"348","volume-title":"An In-Depth Analysis of the Question Answering Performance of the GPT LLM Family. In ISWC","author":"Tan Y.","year":"2023","unstructured":"Y. Tan, D. Min, Y. Li, W. Li, N. Hu, Y. Chen, and G. Qi. Can ChatGPT Replace Traditional KBQA Models? An In-Depth Analysis of the Question Answering Performance of the GPT LLM Family. In ISWC, pages 348--367, 2023."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00175"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-emnlp.209"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657661"},{"key":"e_1_2_1_47_1","volume-title":"Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study. CoRR, abs\/2406.17532","author":"Wang K.","year":"2024","unstructured":"K. Wang, G. Qi, J. Li, and S. Zhai. Can Large Language Models Understand DL-Lite Ontologies? An Empirical Study. CoRR, abs\/2406.17532, 2024."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.804"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19433-7_3"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.732"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.62"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.580"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/3641204.3641208"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3661370"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.632"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i10.21417"},{"key":"e_1_2_1_57_1","volume-title":"OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System. In Workshops at the International Conference on Very Large Data Bases (VLDB)","author":"Zhang N.","year":"2024","unstructured":"N. Zhang, Z. Xi, Y. Luo, P. Wang, B. Tian, Y. Yao, J. Zhang, S. Deng, M. Sun, L. Liang, Z. Zhang, X. Zhu, J. Zhou, and H. Chen. OneEdit: A Neural-Symbolic Collaboratively Knowledge Editing System. In Workshops at the International Conference on Very Large Data Bases (VLDB), 2024."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3664647.3681327"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE60146.2024.00280"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.296"}],"container-title":["ACM SIGMOD Record"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749116.3749132","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,15]],"date-time":"2025-07-15T16:22:10Z","timestamp":1752596530000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749116.3749132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,14]]},"references-count":60,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,7,14]]}},"alternative-id":["10.1145\/3749116.3749132"],"URL":"https:\/\/doi.org\/10.1145\/3749116.3749132","relation":{},"ISSN":["0163-5808"],"issn-type":[{"value":"0163-5808","type":"print"}],"subject":[],"published":{"date-parts":[[2025,7,14]]},"assertion":[{"value":"2025-07-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}