{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T01:58:00Z","timestamp":1777341480170,"version":"3.51.4"},"reference-count":30,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T00:00:00Z","timestamp":1757548800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Research Foundation","award":["490998901"],"award-info":[{"award-number":["490998901"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>This paper introduces GraphTrace, a novel retrieval framework that integrates a domain-specific knowledge graph (KG) with a large language model (LLM) to improve information retrieval for complex, multi-hop queries. Built on structured economic data related to the COVID-19 pandemic, GraphTrace adopts a modular architecture comprising entity extraction, path finding, query decomposition, semantic path ranking, and context aggregation, followed by LLM-based answer generation. GraphTrace is compared against baseline retrieval-augmented generation (RAG) and graph-based RAG (GraphRAG) approaches in both retrieval and generation settings. Experimental results show that GraphTrace consistently outperforms the baselines across evaluation metrics, particularly in handling mid-complexity (5\u20136-hop) queries and achieving top scores in directness during the generation evaluation. These gains are attributed to GraphTrace\u2019s alignment of semantic reasoning with structured KG traversal, combining modular components for more targeted and interpretable retrieval.<\/jats:p>","DOI":"10.3390\/computers14090382","type":"journal-article","created":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T10:50:04Z","timestamp":1757587804000},"page":"382","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["GraphTrace: A Modular Retrieval Framework Combining Knowledge Graphs and Large Language Models for Multi-Hop Question Answering"],"prefix":"10.3390","volume":"14","author":[{"given":"Anna","family":"Osipjan","sequence":"first","affiliation":[{"name":"The Institute of Information Systems, University of L\u00fcbeck, 23562 L\u00fcbeck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7767-8150","authenticated-orcid":false,"given":"Hanieh","family":"Khorashadizadeh","sequence":"additional","affiliation":[{"name":"The Institute of Information Systems, University of L\u00fcbeck, 23562 L\u00fcbeck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0278-8313","authenticated-orcid":false,"given":"Akasha-Leonie","family":"Kessel","sequence":"additional","affiliation":[{"name":"The Institute of Information Systems, University of L\u00fcbeck, 23562 L\u00fcbeck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5196-1117","authenticated-orcid":false,"given":"Sven","family":"Groppe","sequence":"additional","affiliation":[{"name":"The Institute of Information Systems, University of L\u00fcbeck, 23562 L\u00fcbeck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0295-7029","authenticated-orcid":false,"given":"Jinghua","family":"Groppe","sequence":"additional","affiliation":[{"name":"The Institute of Information Systems, University of L\u00fcbeck, 23562 L\u00fcbeck, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,11]]},"reference":[{"key":"ref_1","unstructured":"Zhang, Y., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., and Chen, Y. (2023). Siren\u2019s song in the AI ocean: A survey on hallucination in large language models. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1162\/tacl_a_00667","article-title":"Evaluating correctness and faithfulness of instruction-following models for question answering","volume":"12","author":"Adlakha","year":"2024","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, T., Zhang, Y., Brockett, C., Mao, Y., Sui, Z., Chen, W., and Dolan, B. (2021). A token-level reference-free hallucination detection benchmark for free-form text generation. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.464"},{"key":"ref_4","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_5","unstructured":"Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H., and Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv."},{"key":"ref_6","unstructured":"Tang, Y., and Yang, Y. (2024). Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries. arXiv."},{"key":"ref_7","unstructured":"Peng, B., Zhu, Y., Liu, Y., Bo, X., Shi, H., Hong, C., Zhang, Y., and Tang, S. (2024). Graph retrieval-augmented generation: A survey. arXiv."},{"key":"ref_8","unstructured":"Edge, D., Trinh, H., and Larson, J. (2025, July 01). LazyGraphRAG: Setting a New Standard for Quality and Cost. Available online: https:\/\/www.microsoft.com\/en-us\/research\/blog\/lazygraphrag-setting-a-new-standard-for-quality-and-cost\/."},{"key":"ref_9","unstructured":"Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R.O., and Larson, J. (2025). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv."},{"key":"ref_10","unstructured":"Sun, J., Xu, C., Tang, L., Wang, S., Lin, C., Gong, Y., Ni, L.M., Shum, H.Y., and Guo, J. (2023). Think-on-graph: Deep and responsible reasoning of large language model on knowledge graph. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Cao, Y., Gao, Z., Li, Z., Xie, X., Zhou, K., and Xu, J. (2025). LEGO-GraphRAG: Modularizing Graph-based Retrieval-Augmented Generation for Design Space Exploration. arXiv.","DOI":"10.14778\/3748191.3748194"},{"key":"ref_12","unstructured":"Liu, R., Jiang, H., Yan, X., Tang, B., and Li, J. (2025). PolyG: Effective and Efficient GraphRAG with Adaptive Graph Traversal. arXiv."},{"key":"ref_13","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_14","unstructured":"Zhu, W., Xiong, H., Cheng, X., Cui, L., Dou, Z., Dong, J., Pang, S., Wang, L., Kong, L., and Chen, Z. Evaluation of Retrieval-Augmented Generation: A Survey. Proceedings of the Big Data."},{"key":"ref_15","unstructured":"Gao, Y., Xiong, Y., Wang, M., and Wang, H. (2024). Modular RAG: Transforming RAG Systems into LEGO-like Reconfigurable Frameworks. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hu, Y., Lei, Z., Zhang, Z., Pan, B., Ling, C., and Zhao, L. (2024). GRAG: Graph Retrieval-Augmented Generation. arXiv.","DOI":"10.18653\/v1\/2025.findings-naacl.232"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Shi, Y., Tan, Q., Wu, X., Zhong, S., Zhou, K., and Liu, N. (2024, January 21\u201325). Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, New York, NY, USA. CIKM \u201924.","DOI":"10.1145\/3627673.3679722"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1561\/1500000102","article-title":"Multi-hop Question Answering","volume":"17","author":"Mavi","year":"2024","journal-title":"Found. Trends Inf. Retr."},{"key":"ref_19","unstructured":"Gruenwald, L., Jain, S., and Groppe, S. (2021). Leveraging Artificial Intelligence in Global Epidemics, Elsevier."},{"key":"ref_20","unstructured":"Khorashadizadeh, H., Mihindukulasooriya, N., Ranji, N., Ezzabady, M., Ieng, F., Groppe, J., Benamara, F., and Groppe, S. Construction and Canonicalization of Economic Knowledge Graphs with LLMs. Proceedings of the International Knowledge Graph and Semantic Web Conference."},{"key":"ref_21","unstructured":"Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., and Radford, A. (2024). Gpt-4o system card. arXiv."},{"key":"ref_22","unstructured":"(2025, June 30). Sentence Transformers Team msmarco-MiniLM-L6-en-de-v1. Available online: https:\/\/huggingface.co\/cross-encoder\/msmarco-MiniLM-L6-en-de-v1."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Khorashadizadeh, H., Tiwari, S., Benamara, F., Mihindukulasooriya, N., Groppe, J., Sahri, S., Ezzabady, M., Ieng, F., and Groppe, S. (2025, January 4\u20136). EcoRAG: A Multi-hop Economic QA Benchmark for Retrieval Augmented Generation Using Knowledge Graphs. Proceedings of the Natural Language Processing and Information Systems (NLDB), Kanazawa, Japan.","DOI":"10.1007\/978-3-031-97144-0_15"},{"key":"ref_24","unstructured":"(2025, June 30). Cross-Encoder Team all-MiniLM-L6-v2. Available online: https:\/\/huggingface.co\/sentence-transformers\/all-MiniLM-L6-v2."},{"key":"ref_25","unstructured":"Grootendorst, M. (2025, July 01). KeyBERT: Minimal Keyword Extraction with BERT. Available online: https:\/\/www.maartengrootendorst.com\/blog\/keybert\/."},{"key":"ref_26","unstructured":"Sanmartin, D. (2024). Kg-rag: Bridging the gap between knowledge and creativity. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Bollacker, K., Evans, C., Paritosh, P., Sturge, T., and Taylor, J. (2008, January 9\u201312). Freebase: A collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, BC, Canada.","DOI":"10.1145\/1376616.1376746"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1145\/2629489","article-title":"Wikidata: A free collaborative knowledgebase","volume":"57","year":"2014","journal-title":"Commun. ACM"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Wang, J., Liang, Y., Meng, F., Sun, Z., Shi, H., Li, Z., Xu, J., Qu, J., and Zhou, J. (2023). Is chatgpt a good nlg evaluator? a preliminary study. arXiv.","DOI":"10.18653\/v1\/2023.newsum-1.1"},{"key":"ref_30","first-page":"46595","article-title":"Judging llm-as-a-judge with mt-bench and chatbot arena","volume":"36","author":"Zheng","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/9\/382\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:43:49Z","timestamp":1760035429000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/9\/382"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,11]]},"references-count":30,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["computers14090382"],"URL":"https:\/\/doi.org\/10.3390\/computers14090382","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,11]]}}}