{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T17:04:51Z","timestamp":1772471091258,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T00:00:00Z","timestamp":1772409600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100007129","name":"Natural Science Foundation of Shandong Province","doi-asserted-by":"publisher","award":["ZR2024ZD20"],"award-info":[{"award-number":["ZR2024ZD20"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62072469"],"award-info":[{"award-number":["62072469"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Large language models (LLMs) have demonstrated significant capabilities in natural language processing (NLP), but they often encounter challenges in the medical domain. This can result in insufficient alignment between generated answers and user intent, as well as factual deviations. To address these issues, we propose Co-MedGraphRAG, a novel framework combining knowledge graph reasoning with large\u2013small model collaboration, aimed at improving the structural grounding and interpretability of medical responses. The framework operates through a multi-stage collaborative mechanism to augment question answering. First, a large language model constructs a question-specific knowledge graph (KG) containing pending entities (denoted as \u201cnone\u201d) to explicitly define known and unknown variables. Subsequently, a hybrid reasoning strategy is employed to populate the pending entities, thereby completing the question-specific knowledge graph. Finally, this graph serves as critical structured evidence, combined with the original question, to augment the large language model in generating the final answer, implemented using Qwen2.5-7B and GLM4-9B in this paper. To evaluate the generated answers, we introduce a larger-parameter LLM(GPT-4o) to assess performance across five dimensions and compute an overall score. Experiments on three medical datasets demonstrate that Co-MedGraphRAG achieves consistent improvements in relevance, practicality, and structured knowledge support compared with mainstream Retrieval-Augmented Generation (RAG) frameworks. This work serves as a reference for researchers and developers designing medical question-answering frameworks and exploring decision-support applications.<\/jats:p>","DOI":"10.3390\/info17030247","type":"journal-article","created":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T16:06:59Z","timestamp":1772467619000},"page":"247","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Co-MedGraphRAG: A Collaborative Large\u2013Small Model Medical Question-Answering Framework Enhanced by Knowledge Graph Reasoning"],"prefix":"10.3390","volume":"17","author":[{"given":"Sizhe","family":"Chen","sequence":"first","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Data Open Innovative Application Laboratory, Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}]},{"given":"Tao","family":"Chen","sequence":"additional","affiliation":[{"name":"Qingdao Institute of Software, College of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China"},{"name":"Shandong Data Open Innovative Application Laboratory, Qingdao 266580, China"},{"name":"Shandong Key Laboratory of Intelligent Oil & Gas Industrial Software, Qingdao 266580, China"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"8459","DOI":"10.1007\/s12652-021-03612-z","article-title":"Artificial intelligence in disease diagnosis: A systematic literature review, synthesizing framework and future research agenda","volume":"14","author":"Kumar","year":"2023","journal-title":"J. Ambient Intell. Human Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial intelligence in radiology","volume":"18","author":"Hosny","year":"2018","journal-title":"Nat. Rev. Cancer"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e38397","DOI":"10.2196\/38397","article-title":"The application of artificial intelligence in health care resource allocation before and during the COVID-19 pandemic: Scoping review","volume":"2","author":"Wu","year":"2023","journal-title":"JMIR AI"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"795","DOI":"10.1016\/j.amjmed.2019.01.017","article-title":"Artificial intelligence transforms the future of health care","volume":"132","author":"Zand","year":"2019","journal-title":"Am. J. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"711","DOI":"10.21037\/atm.2019.11.108","article-title":"Using artificial intelligence to improve medical services in China","volume":"8","author":"Li","year":"2020","journal-title":"Ann. Transl. Med."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Guo, M., Guo, M., Dougherty, E.T., and Qian, X. (May, January 30). MSQ-BioBERT: Ambiguity resolution to enhance BioBERT medical question-answering. Proceedings of the ACM Web Conference 2023, Austin, TX, USA.","DOI":"10.1145\/3543507.3583878"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Jain, S., and Dodiya, T. (2014). Rule based architecture for medical question answering system. Proceedings of the Second International Conference on Soft Computing for Problem Solving (SocProS 2012), New Delhi, India, 28\u201330 December 2012, Springer.","DOI":"10.1007\/978-81-322-1602-5_128"},{"key":"ref_8","unstructured":"Lee, M., Cimino, J., Zhu, H.R., Sable, C., Shanker, V., Ely, J., and Yu, H. (2006). Beyond information retrieval\u2014Medical question answering. AMIA Annual Symposium Proceedings, American Medical Informatics Association."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zhang, S., Zhang, X., Wang, H., Cheng, J., Li, P., and Ding, Z. (2017). Chinese medical question answer matching using end-to-end character-level multi-scale CNNs. Appl. Sci., 7.","DOI":"10.3390\/app7080767"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder\u2013decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Association for Computational Linguistics.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sundermeyer, M., Alkhouli, T., Wuebker, J., and Ney, H. (2014, January 25\u201329). Translation modeling with bidirectional recurrent neural networks. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1003"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ben Abacha, A., and Demner-Fushman, D. (2019). A question-entailment approach to question answering. BMC Bioinform., 20.","DOI":"10.1186\/s12859-019-3119-4"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yan, Y., Zhang, B.W., Li, X.F., and Liu, Z. (2020). List-wise learning to rank biomedical question-answer pairs with deep ranking recursive autoencoders. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0242061"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: A pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"ref_15","unstructured":"Buck, C., Bulian, J., Ciaramita, M., Gajewski, W., Gesmundo, A., Houlsby, N., and Wang, W. (May, January 30). Ask the right questions: Active question reformulation with reinforcement learning. Proceedings of the Sixth International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Dong, L., Mallinson, J., Reddy, S., and Lapata, M. (2017, January 9\u201311). Learning to paraphrase for question answering. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (EMNLP), Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1091"},{"key":"ref_17","unstructured":"Khandelwal, U., Levy, O., Jurafsky, D., Zettlemoyer, L., and Lewis, M. (2020, January 26\u201330). Generalization through memorization: Nearest neighbor language models. Proceedings of the Eighth International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia."},{"key":"ref_18","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","volume":"Volume 33","author":"Lewis","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_19","unstructured":"Guu, K., Lee, K., Tung, Z., Pasupat, P., and Chang, M. (2020). Retrieval augmented language model pre-training. Proceedings of the International Conference on Machine Learning, Virtual, 13\u201318 July 2020, PMLR."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Izacard, G., and Grave, E. (2021, January 19\u201323). Leveraging passage retrieval with generative models for open domain question answering. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, Online.","DOI":"10.18653\/v1\/2021.eacl-main.74"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Xu, S., Yan, Z., Dai, C., and Wu, F. (2025). MEGA-RAG: A retrieval-augmented generation framework with multi-evidence guided answer refinement for mitigating hallucinations of LLMs in public health. Front. Public Health, 13.","DOI":"10.3389\/fpubh.2025.1635381"},{"key":"ref_22","unstructured":"Zheng, L., Li, Z., Zhang, H., Zhuang, Y., Chen, Z., Huang, Y., Wang, Y., Xu, Y., Zhuo, D., and Xing, E.P. (2022, January 11\u201313). Alpa: Automating inter- and intra-operator parallelism for distributed deep learning. Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), Carlsbad, CA, USA."},{"key":"ref_23","first-page":"1","article-title":"PaLM: Scaling language modeling with pathways","volume":"24","author":"Chowdhery","year":"2023","journal-title":"J. Mach. Learn. Res."},{"key":"ref_24","unstructured":"Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., Lin, Q., and Jiang, D. (2024, January 7\u201311). WizardLM: Empowering large pre-trained language models to follow complex instructions. Proceedings of the Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ranaldi, L., and Freitas, A. (2024). Aligning large and small language models via chain-of-thought reasoning. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), St. Julian\u2019s, Malta, 17\u201322 March 2024, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.eacl-long.109"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chen, D., Zhuang, Y., Zhang, S., Liu, J., Dong, S., and Tang, S. (2024, January 20\u201327). Data shunt: Collaboration of small and large models for lower costs and better performance. Proceedings of the AAAI Conference on Artificial Intelligence, Vancouver, BC, Canada.","DOI":"10.1609\/aaai.v38i10.29003"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"108276","DOI":"10.1016\/j.neunet.2025.108276","article-title":"Improving large models with small models: Lower costs and better performance","volume":"195","author":"Chen","year":"2026","journal-title":"Neural Netw."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Xu, C., Xu, Y., Wang, S., Liu, Y., Zhu, C., and McAuley, J. (2024). Small Models are Valuable Plug-ins for Large Language Models. Findings of the Association for Computational Linguistics: ACL 2024, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.findings-acl.18"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"108843","DOI":"10.1016\/j.knosys.2022.108843","article-title":"Step by step: A hierarchical framework for multi-hop knowledge graph reasoning with reinforcement learning","volume":"248","author":"Zhu","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_30","unstructured":"Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A.J., Welihinda, A., and Hayes, A. (2024). GPT-4o System Card. arXiv."},{"key":"ref_31","first-page":"46595","article-title":"Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena","volume":"36","author":"Zheng","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","unstructured":"(2026, January 30). Chinese-Medical-Dialogue-Data: A Large-Scale Chinese Medical Dialogue Dataset. GitHub Repository. Available online: https:\/\/github.com\/Toyhom\/Chinese-medical-dialogue-data."},{"key":"ref_33","unstructured":"wjjingtian (2026, January 30). cMQA: Chinese Medical Question Answering Dataset. GitHub Repository. Available online: https:\/\/github.com\/wjjingtian\/cMQA."},{"key":"ref_34","unstructured":"Li, J., Wang, X., Wu, X., Zhang, Z., Xu, X., Fu, J., Tiwari, P., Wan, X., and Wang, B. (2023). Huatuo-26M: A Large-Scale Chinese Medical Question Answering Dataset. arXiv."},{"key":"ref_35","unstructured":"Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., and Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv."},{"key":"ref_36","unstructured":"Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R.O., and Larson, J. (2024). From local to global: A graph RAG approach to query-focused summarization. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Guo, Z., Xia, L., Yu, Y., Ao, T., and Huang, C. (2025). LightRAG: Simple and fast retrieval-augmented generation. Findings of the Association for Computational Linguistics: EMNLP 2025, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2025.findings-emnlp.568"},{"key":"ref_38","unstructured":"Team GLM, Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., and Zhao, H. (2024). ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools (Technical Report). arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024). M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. Findings of the Association for Computational Linguistics: ACL 2024, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.findings-acl.137"},{"key":"ref_40","unstructured":"Hu, H., Feng, Y., Li, R., Xue, R., Hou, X., Tian, Z., Gao, Y., and Du, S. (2026, January 20\u201327). Cog-RAG: Cognitive-Inspired Dual-Hypergraph with Theme Alignment Retrieval-Augmented Generation. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI 2026), Singapore."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/3\/247\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,2]],"date-time":"2026-03-02T16:23:12Z","timestamp":1772468592000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/3\/247"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,2]]},"references-count":40,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["info17030247"],"URL":"https:\/\/doi.org\/10.3390\/info17030247","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,2]]}}}