{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:47:42Z","timestamp":1782722862597,"version":"3.54.5"},"reference-count":64,"publisher":"PeerJ","license":[{"start":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T00:00:00Z","timestamp":1782691200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Dalian Customs","award":["2025HK184, 2025HK209"],"award-info":[{"award-number":["2025HK184, 2025HK209"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:p>\n                    Large Language Models (LLMs) often face challenges in performing reliable multi-hop reasoning due to issues such as incomplete evidence chains and hallucinations. Incorporating knowledge graphs (KGs) can mitigate these problems, but existing approaches either suffer from suboptimal accuracy or are computationally expensive. To address these issues, we propose Reasoning Path Retrieval for RAG (RPR-RAG), a novel KG-based retrieval framework that incrementally builds a subgraph from the knowledge graph, extracts explicit reasoning paths, and provides them as structured external evidence to downstream LLMs. The experimental results on WebQuestionsSP (WebQSP) and Complex WebQuestions (CWQ) indicate that RPR-RAG achieves competitive Hit and F1 in multi-hop reasoning tasks, while maintaining runtime, LLM call frequency, and token usage at reasonable levels. Moreover, without additional task-specific training, RPR-RAG also shows strong zero-shot performance on MetaQA. RPR-RAG is built on a lightweight embedding model which can be trained and executed on a single consumer-grade GPU (\n                    <jats:italic>e.g<\/jats:italic>\n                    ., RTX 3060, 6 GB). Ablation studies reveal that the path validity evaluation and stopping criterion play important roles in retrieval quality and efficiency. RPR-RAG is compatible with a range of backbone LLMs, from smaller 7B models to larger models such as GPT-5, providing a practical and interpretable framework for KG-grounded reasoning tasks. The source code is available at\n                    <jats:uri xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/doi.org\/10.5281\/zenodo.19334059\">https:\/\/doi.org\/10.5281\/zenodo.19334059<\/jats:uri>\n                    .\n                  <\/jats:p>","DOI":"10.7717\/peerj-cs.3950","type":"journal-article","created":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:27:02Z","timestamp":1782721622000},"page":"e3950","source":"Crossref","is-referenced-by-count":0,"title":["A cost-effective approach for knowledge graph reasoning path retrieval and enhanced large language model reliability"],"prefix":"10.7717","volume":"12","author":[{"given":"Zhe","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Jia","sequence":"additional","affiliation":[{"name":"Science and Technology Department, Dalian Customs, Dalian, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liang","family":"Zhao","sequence":"additional","affiliation":[{"name":"Dalian International Travel Healthcare Center (Dalian Customs Port Clinic), Dalian Customs, Dalian, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanming","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"4443","published-online":{"date-parts":[[2026,6,29]]},"reference":[{"issue":"16","key":"10.7717\/peerj-cs.3950\/ref-1","doi-asserted-by":"publisher","first-page":"17682","DOI":"10.1609\/aaai.v38i16.29720","article-title":"Graph of thoughts: solving elaborate problems with large language models","volume":"38","author":"Besta","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3950\/ref-2","first-page":"8409","article-title":"StePO-Rec: towards personalized outfit styling assistant via knowledge-guided multi-step reasoning","author":"Bi","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-3","first-page":"1247","article-title":"Freebase: a collaboratively created graph database for structuring human knowledge","author":"Bollacker","year":"2008"},{"key":"10.7717\/peerj-cs.3950\/ref-4","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2502.14902","article-title":"PathRAG: pruning graph-based retrieval augmented generation with relational paths","author":"Chen","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-5","first-page":"37665","article-title":"Plan-on-graph: self-correcting adaptive planning of large language model on knowledge graphs","volume-title":"Advances in Neural Information Processing Systems","volume":"37","author":"Chen","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2501.12948","article-title":"DeepSeek-R1: incentivizing reasoning capability in LLMs via reinforcement learning","author":"DeepSeek-AI","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-7","doi-asserted-by":"crossref","first-page":"14169","DOI":"10.18653\/v1\/2024.acl-long.764","article-title":"EWEK-QA: enhanced web and efficient knowledge graph retrieval for citation-based question answering systems","volume-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Dehghan","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-8","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.01238","article-title":"EffiQA: efficient question-answering with strategic multi-model collaboration on knowledge graphs","author":"Dong","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.16130","article-title":"From local to global: a graph rag approach to query-focused summarization","author":"Edge","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-10","first-page":"6178","article-title":"FRAG: a flexible modular framework for retrieval-augmented generation based on knowledge graphs","author":"Gao","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-11","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.10997","article-title":"Retrieval-augmented generation for large language models: a survey","author":"Gao","year":"2023"},{"key":"10.7717\/peerj-cs.3950\/ref-12","first-page":"2447","article-title":"Graph reasoning enhanced language models for text-to-SQL","author":"Gong","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-13","first-page":"10746","article-title":"LightRAG: simple and fast retrieval-augmented generation","author":"Guo","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-14","first-page":"553","article-title":"Improving multi-hop knowledge base question answering by learning intermediate supervision signals","author":"He","year":"2021"},{"key":"10.7717\/peerj-cs.3950\/ref-15","first-page":"72819","article-title":"Training chain-of-thought via latent-variable inference","volume":"36","author":"Hoffman","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10.7717\/peerj-cs.3950\/ref-16","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.1212303","article-title":"spaCY: industrial-strength natural language processing in python. Zenodo","author":"Honnibal","year":"2020"},{"key":"10.7717\/peerj-cs.3950\/ref-17","first-page":"4145","article-title":"GRAG: graph retrieval-augmented generation","author":"Hu","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-18","article-title":"Large language models cannot self-correct reasoning yet","author":"Huang","year":"2024"},{"issue":"16","key":"10.7717\/peerj-cs.3950\/ref-19","doi-asserted-by":"publisher","first-page":"18345","DOI":"10.1609\/aaai.v38i16.29794","article-title":"Chain-of-thought improves text generation with citations in large language models","volume":"38","author":"Ji","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3950\/ref-20","first-page":"9237","article-title":"StructGPT: a general framework for large language model to reason over structured data","author":"Jiang","year":"2023a"},{"key":"10.7717\/peerj-cs.3950\/ref-21","article-title":"UniKGQA: unified retrieval and reasoning for solving multi-hop question answering over knowledge graph","author":"Jiang","year":"2023b"},{"key":"10.7717\/peerj-cs.3950\/ref-22","article-title":"HippoRAG: neurobiologically inspired long-term memory for large language models","volume":"37","author":"Jim\u00e9nez Guti\u00e9rrez","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-23","doi-asserted-by":"crossref","first-page":"163","DOI":"10.18653\/v1\/2024.findings-acl.11","article-title":"Graph chain-of-thought: augmenting large language models by reasoning on graphs","volume-title":"Findings of the Association for Computational Linguistics: ACL 2024","author":"Jin","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-24","first-page":"3366","article-title":"Graph reasoning for question answering with triplet retrieval","author":"Li","year":"2023"},{"issue":"2","key":"10.7717\/peerj-cs.3950\/ref-25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3690635","article-title":"Structured chain-of-thought prompting for code generation","volume":"34","author":"Li","year":"2025","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"issue":"8","key":"10.7717\/peerj-cs.3950\/ref-26","doi-asserted-by":"publisher","first-page":"8688","DOI":"10.1609\/aaai.v38i8.28714","article-title":"UniGen: a unified generative framework for retrieval and question answering with large language models","volume":"38","author":"Li","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"17","key":"10.7717\/peerj-cs.3950\/ref-27","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s10489-025-06885-5","article-title":"Knowledge graph-extended retrieval augmented generation for question answering","volume":"45","author":"Linders","year":"2025","journal-title":"Applied Intelligence"},{"key":"10.7717\/peerj-cs.3950\/ref-28","article-title":"Reasoning on graphs: faithful and interpretable large language model reasoning","author":"Luo","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-29","article-title":"Graph-constrained reasoning: faithful reasoning on knowledge graphs with large language models","author":"Luo","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-30","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.20139","article-title":"GNN-RAG: graph neural retrieval for large language model reasoning","author":"Mavromatis","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-31","article-title":"Llama 4: advancing multimodal intelligence","author":"Meta-AI","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-32","first-page":"3366","article-title":"Dense retrieval of knowledge graphs for question answering","author":"Nangi","year":"2023"},{"key":"10.7717\/peerj-cs.3950\/ref-33","article-title":"GPT-4 technical report","author":"OpenAI","year":"2023"},{"key":"10.7717\/peerj-cs.3950\/ref-34","article-title":"GPT-5 system card","author":"OpenAI","year":"2025"},{"issue":"7","key":"10.7717\/peerj-cs.3950\/ref-35","doi-asserted-by":"publisher","first-page":"3580","DOI":"10.1109\/tkde.2024.3352100","article-title":"Unifying large language models and knowledge graphs: a roadmap","volume":"36","author":"Pan","year":"2024","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"10.7717\/peerj-cs.3950\/ref-36","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2506.08364","article-title":"Structure-augmented reasoning generation","author":"Parekh","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2009.02252","article-title":"KiLT: a benchmark for knowledge intensive language tasks","author":"Petroni","year":"2020"},{"key":"10.7717\/peerj-cs.3950\/ref-38","article-title":"QwQ-32B: a 32B-parameter reasoning model","author":"Qwen","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-39","doi-asserted-by":"crossref","DOI":"10.1145\/3677052.3698671","article-title":"HybridRAG: integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction","author":"Sarmah","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-40","first-page":"2380","article-title":"PullNet: open domain question answering with iterative retrieval on knowledge bases and text","author":"Sun","year":"2019"},{"key":"10.7717\/peerj-cs.3950\/ref-41","first-page":"4231","article-title":"Open domain question answering using early fusion of knowledge bases and text","author":"Sun","year":"2018"},{"key":"10.7717\/peerj-cs.3950\/ref-42","article-title":"Think-on-graph: deep and responsible reasoning of large language model on knowledge graph","author":"Sun","year":"2024a"},{"issue":"2","key":"10.7717\/peerj-cs.3950\/ref-43","doi-asserted-by":"publisher","first-page":"150","DOI":"10.1038\/s44284-023-00022-4","article-title":"Large-scale online job search behaviors reveal labor market shifts amid COVID-19","volume":"1","author":"Sun","year":"2024b","journal-title":"Nature Cities"},{"key":"10.7717\/peerj-cs.3950\/ref-44","first-page":"641","article-title":"The web as a knowledge-base for answering complex questions","author":"Talmor","year":"2018"},{"key":"10.7717\/peerj-cs.3950\/ref-45","first-page":"3505","article-title":"Paths-over-graph: knowledge graph empowered large language model reasoning","author":"Tan","year":"2025"},{"key":"10.7717\/peerj-cs.3950\/ref-46","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.09288","article-title":"Llama 2: open foundation and fine-tuned chat models","author":"Touvron","year":"2023"},{"issue":"17","key":"10.7717\/peerj-cs.3950\/ref-47","doi-asserted-by":"publisher","first-page":"19162","DOI":"10.1609\/aaai.v38i17.29884","article-title":"T-SciQ: teaching multimodal chain-of-thought reasoning via large language model signals for science question answering","volume":"38","author":"Wang","year":"2024a","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"3","key":"10.7717\/peerj-cs.3950\/ref-48","doi-asserted-by":"publisher","first-page":"1001","DOI":"10.1109\/tsc.2023.3326539","article-title":"Joint admission control and resource allocation of virtual network embedding via hierarchical deep reinforcement learning","volume":"17","author":"Wang","year":"2024c","journal-title":"IEEE Transactions on Services Computing"},{"key":"10.7717\/peerj-cs.3950\/ref-49","first-page":"740","article-title":"Dynamic sparse learning: a novel paradigm for efficient recommendation","author":"Wang","year":"2024b"},{"key":"10.7717\/peerj-cs.3950\/ref-50","article-title":"Self-consistency improves chain of thought reasoning in language models","author":"Wang","year":"2023"},{"key":"10.7717\/peerj-cs.3950\/ref-51","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022"},{"key":"10.7717\/peerj-cs.3950\/ref-52","first-page":"10370","article-title":"MindMap: knowledge graph prompting sparks graph of thoughts in large language models","author":"Wen","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-53","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2024.emnlp-main.205","article-title":"CoTKR: chain-of-thought enhanced knowledge rewriting for complex knowledge graph question answering","author":"Wu","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-54","doi-asserted-by":"crossref","DOI":"10.1145\/3626772.3657878","article-title":"C-pack: packed resources for general Chinese embeddings","author":"Xiao","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-55","first-page":"1","article-title":"WeKnow-RAG: an adaptive approach for retrieval-augmented generation integrating web search and knowledge graphs","author":"Xie","year":"2024"},{"key":"10.7717\/peerj-cs.3950\/ref-56","first-page":"11809","article-title":"Tree of thoughts: deliberate problem solving with large language models","volume":"36","author":"Yao","year":"2023a"},{"key":"10.7717\/peerj-cs.3950\/ref-57","article-title":"ReAct: synergizing reasoning and acting in language models","author":"Yao","year":"2023b"},{"key":"10.7717\/peerj-cs.3950\/ref-58","first-page":"6032","article-title":"RnG-KBQA: generation augmented iterative ranking for knowledge base question answering","author":"Ye","year":"2022"},{"key":"10.7717\/peerj-cs.3950\/ref-59","first-page":"201","article-title":"The value of semantic parse labeling for knowledge base question answering","author":"Yih","year":"2016"},{"key":"10.7717\/peerj-cs.3950\/ref-60","article-title":"DecAF: joint decoding of answers and logical forms for question answering over knowledge bases","author":"Yu","year":"2022"},{"key":"10.7717\/peerj-cs.3950\/ref-61","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.08860","article-title":"GreaseLM: graph reasoning enhanced language models for question answering","author":"Zhang","year":"2022b"},{"key":"10.7717\/peerj-cs.3950\/ref-62","doi-asserted-by":"publisher","first-page":"9359","DOI":"10.1609\/aaai.v38i8.28789","article-title":"Temporal graph contrastive learning for sequential recommendation","volume":"38","author":"Zhang","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"10.7717\/peerj-cs.3950\/ref-63","article-title":"Variational reasoning for question answering with knowledge graph","volume":"32","author":"Zhang","year":"2018"},{"key":"10.7717\/peerj-cs.3950\/ref-64","first-page":"5773","article-title":"Subgraph retrieval enhanced model for multi-hop knowledge base question answering","author":"Zhang","year":"2022a"}],"container-title":["PeerJ Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/peerj.com\/articles\/cs-3950.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3950.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3950.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/peerj.com\/articles\/cs-3950.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T08:27:11Z","timestamp":1782721631000},"score":1,"resource":{"primary":{"URL":"https:\/\/peerj.com\/articles\/cs-3950"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,29]]},"references-count":64,"alternative-id":["10.7717\/peerj-cs.3950"],"URL":"https:\/\/doi.org\/10.7717\/peerj-cs.3950","archive":["CLOCKSS","LOCKSS","Portico"],"relation":{},"ISSN":["2376-5992"],"issn-type":[{"value":"2376-5992","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,29]]},"article-number":"e3950"}}