{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T17:45:06Z","timestamp":1793727906218,"version":"4.6.1"},"reference-count":58,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T00:00:00Z","timestamp":1773273600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) offer significant potential for intelligent question answering (Q&amp;amp;A) in healthcare, yet traditional knowledge representation methods fail to capture the complex, hierarchical nature of Traditional Chinese Medicine (TCM) knowledge systems. The lack of effective retrieval-augmented generation (RAG) frameworks specifically tailored for TCM\u2019s unique epistemology limits applications.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>This study aims to evaluate the effectiveness of a novel Tree-Organized Self-Reflective Retrieval (TOSRR) framework in enhancing LLM performance on TCM Q&amp;amp;A tasks through innovative knowledge organization and dynamic self-correction mechanisms.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We developed a hierarchical knowledge representation system that structures TCM knowledge as subject-predicate-object-text (SPO-T) units within a tree-like architecture, enabling multi-dimensional relationships while preserving semantic context. Our iterative self-reflection mechanism implements dynamic knowledge retrieval and validation across textbook chapters and disciplines. Performance was evaluated using randomly selected questions from the TCM Medical Licensing Examination (MLE) and college Classics Course Exam (CCE), representing both standardized clinical knowledge and classical theory assessment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>When integrated with GPT-4, the TOSRR framework demonstrated a 19.85% improvement in absolute accuracy on the TCM MLE benchmark and increased recall accuracy from 27 to 38% on CCE datasets. Expert manual evaluation revealed substantial enhancements across critical dimensions: safety, consistency, explainability, compliance, and coherence, with a comprehensive improvement of 18.64 points. Retrieval-Augmented Generation Assessment (RAGAs) metrics confirmed the framework\u2019s superior knowledge utilization, retrieval precision, and resistance to information noise compared to standard RAG approaches.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The TOSRR framework enhances LLM performance in TCM knowledge tasks through its hierarchical knowledge representation and self-reflective retrieval approach. And the framework has potential for application in teaching.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2026.1752778","type":"journal-article","created":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T14:35:32Z","timestamp":1773326132000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Improving TCM question answering through tree-organized self-reflective retrieval with LLMs"],"prefix":"10.3389","volume":"13","author":[{"given":"Chang","family":"Liu","sequence":"first","affiliation":[{"name":"School of Basic Medical Sciences, Zhejiang Chinese Medical University","place":["Hangzhou, China"]},{"name":"Breast Disease Specialist Hospital of Guangdong Provincial Hospital of Chinese Medicine, Guangdong Provincial Hospital of Chinese Medicine","place":["Guangzhou, China"]},{"name":"Gancao Doctor Institute of Artificial Intelligence for Chinese Medicine, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ying","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Basic Medical Sciences, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianmin","family":"Li","sequence":"additional","affiliation":[{"name":"Gancao Doctor Institute of Artificial Intelligence for Chinese Medicine, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiqian","family":"Qu","sequence":"additional","affiliation":[{"name":"School of Basic Medical Sciences, Zhejiang Chinese Medical University","place":["Hangzhou, China"]},{"name":"Gancao Doctor Institute of Artificial Intelligence for Chinese Medicine, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Li","sequence":"additional","affiliation":[{"name":"Faculty of Chinese Medicine, Macau University of Science and Technology","place":["Macau, Macau SAR, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lingyong","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Basic Medical Sciences, Zhejiang Chinese Medical University","place":["Hangzhou, China"]},{"name":"Gancao Doctor Institute of Artificial Intelligence for Chinese Medicine, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyuan","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Basic Medical Sciences, Zhejiang Chinese Medical University","place":["Hangzhou, China"]},{"name":"Gancao Doctor Institute of Artificial Intelligence for Chinese Medicine, Zhejiang Chinese Medical University","place":["Hangzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,3,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1386085","DOI":"10.3389\/fpubh.2024.1386085","article-title":"The mediating effect of traditional Chinese medicine (tcm) health literacy between tcm culture promotion and residents\u2019 health status.","volume":"12","author":"Zhang","year":"2024","journal-title":"Front Public Health"},{"key":"B2","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1186\/s12889-022-13309-3","article-title":"Do medical treatment choices affect the health of chronic patients in middle and old age in China?-Evidence from charls 2018.","volume":"22","author":"Tang","year":"2022","journal-title":"Bmc Public Health"},{"key":"B3","doi-asserted-by":"publisher","first-page":"425037","DOI":"10.1155\/2015\/425037","article-title":"Developing traditional Chinese medicine in the era of evidence-based medicine: current evidences and challenges.","volume":"2015","author":"Fung","year":"2015","journal-title":"Evid Based Complement Alternat Med"},{"key":"B4","doi-asserted-by":"publisher","first-page":"107530","DOI":"10.1016\/j.phrs.2024.107530","article-title":"Tcmchat: a generative large language model for traditional Chinese medicine.","volume":"210","author":"Dai","year":"2024","journal-title":"Pharmacol Res"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e12658","DOI":"10.1111\/jebm.12658","article-title":"Large language models in traditional Chinese medicine: a scoping review.","volume":"18","author":"Ren","year":"2025","journal-title":"J Evid Based Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1097\/HM9.0000000000000143","article-title":"Large language models in traditional Chinese medicine: a systematic review.","volume":"5","author":"Chen","year":"2025","journal-title":"Acupunct Herbal Med."},{"key":"B7","doi-asserted-by":"publisher","first-page":"1512329","DOI":"10.3389\/fmed.2024.1512329","article-title":"Research on a traditional Chinese medicine case-based question-answering system integrating large language models and knowledge graphs.","volume":"11","author":"Duan","year":"2024","journal-title":"Front Med"},{"key":"B8","doi-asserted-by":"publisher","first-page":"109887","DOI":"10.1016\/j.compbiomed.2025.109887","article-title":"Tcm-kllama: intelligent generation model for traditional Chinese medicine prescriptions based on knowledge graph and large language model.","volume":"189","author":"Zhuang","year":"2025","journal-title":"Comput Biol Med"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2409.14924","article-title":"Retrieval augmented generation (rag) and beyond: a comprehensive survey on how to make your llms use external data more wisely.","author":"Zhao","year":"2024","journal-title":"arXiv [Preprints]."},{"key":"B10","article-title":"Chinese medicine question answering robot based on rag and self-built dataset.","volume-title":"Proceedings of the 2nd International Conference on Machine Learning and Automation.","author":"Zuo","year":"2024"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.12288","article-title":"The reversal curse: llms trained on \u201ca is b\u201d fail to learn \u201cb is a\u201d.","author":"Berglund","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.11401","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks.","author":"Lewis","year":"2020","journal-title":"arXiv [Preprint]."},{"key":"B13","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.1007\/s10439-023-03327-6","article-title":"Potential for gpt technology to optimize future clinical decision-making using retrieval-augmented generation.","volume":"52","author":"Wang","year":"2024","journal-title":"Ann Biomed Eng"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-96-4298-4_2","article-title":"Traditional Chinese medicine case analysis system for high-level semantic abstraction: optimized with prompt and rag.","author":"Xu","year":"2025","journal-title":"arXiv [Preprint]."},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2401.18059","article-title":"Raptor: recursive abstractive processing for tree-organized retrieval.","author":"Sarthi","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B16","unstructured":"Xuanqiwendui.\n          \n          2025"},{"key":"B17","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge.","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B18","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models.","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat Med"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.14454","article-title":"Pmc-llama: further finetuning llama on medical papers.","author":"Wu","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.01097","article-title":"Doctorglm: fine-tuning your Chinese doctor is not a herculean task.","author":"Xiong","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.06975","article-title":"Huatuo: tuning llama model with Chinese medical knowledge.","author":"Wang","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2305.17100","article-title":"Biomedgpt: a unified and generalist biomedical generative pre-trained transformer for vision, language, and multimodal tasks.","author":"Zhang","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2310.09089","article-title":"Qilin-med: multi-stage knowledge injection advanced medical large language model.","author":"Ye","year":"2023","journal-title":"arXiv [Preprint]."},{"key":"B24","doi-asserted-by":"publisher","first-page":"2159","DOI":"10.1609\/aaai.v38i17.29907","article-title":"Zhongjing: enhancing the Chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue.","volume":"38","author":"Yang","year":"2024"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2403.16056","article-title":"Qibo: a large language model for traditional Chinese medicine.","author":"Zhang","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2501.02471","article-title":"Hengqin-ra-v1: advanced large language model for diagnosis and treatment of rheumatoid arthritis with dataset based traditional Chinese medicine.","author":"Liu","year":"2025","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2406.01126","article-title":"Tcmbench: a comprehensive benchmark for evaluating large language models in traditional Chinese medicine.","author":"Yue","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2402.01733","article-title":"Development and testing of retrieval augmented generation in large language models - a case study report.","author":"Ke","year":"2024","journal-title":"arXiv [Preprint]."},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.11973","article-title":"Exploring the landscape of large language models: foundations, techniques, and challenges.","author":"Moradi","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B30","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2022.naacl-srw.7","article-title":"Retrieval-augmented generation across heterogeneous knowledge.","volume-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Student Research Workshop.","author":"Yu","year":"2022"},{"key":"B31","first-page":"4471","article-title":"Study on Chinese medicine digital syndrome differentiation with its knowledge representation and reasoning methods.","volume":"34","author":"Cangfa","year":"2019","journal-title":"China J Tradit Chin Med Pharm"},{"key":"B32","doi-asserted-by":"publisher","first-page":"240","DOI":"10.16690\/j.cnki.1007-9203.2023.17.036","article-title":"Tacit knowledge and its implication for Chinese medicine education.","volume":"31","author":"Yang","year":"2023","journal-title":"J Tradit Chin Med Manage"},{"key":"B33","doi-asserted-by":"publisher","first-page":"979","DOI":"10.14148\/j.issn.1672-0482.2023.0979","article-title":"Discussion on classical formula artificial intelligence research path based on graph theory.","volume":"39","author":"Liu","year":"2023","journal-title":"J Nanjing Univ Tradit Chin Med"},{"key":"B34","first-page":"2547","article-title":"Construction and application of knowledge atlas of traditional Chinese medicine for removing blood stasis.","volume":"34","author":"Li","year":"2023","journal-title":"Lishizhen Med Materia Med Res"},{"key":"B35","first-page":"204","article-title":"Design and development of intelligent question answering system based on knowledge map of Chinese medicine.","volume":"20","author":"Chen","year":"2018","journal-title":"China New Telecommun"},{"key":"B36","first-page":"98","article-title":"Design and implementation of intelligent inquiry system of TCM medical records based on knowledge graph.","volume":"29","author":"Tang","year":"2023","journal-title":"Modern Comput"},{"key":"B37","first-page":"105","article-title":"Research on the construction and application of the knowledge graph of Prescriptions for Fifty-two Diseases.","volume":"18","author":"Lu","year":"2023","journal-title":"China Digital Med"},{"key":"B38","doi-asserted-by":"publisher","first-page":"1147677","DOI":"10.3389\/fphar.2023.1147677","article-title":"The construction of a tcm knowledge graph and application of potential knowledge discovery in diabetic kidney disease by integrating diagnosis and treatment guidelines and real-world clinical data.","volume":"14","author":"Zhao","year":"2023","journal-title":"Front Pharmacol"},{"key":"B39","doi-asserted-by":"publisher","first-page":"232","DOI":"10.3390\/info12060232","article-title":"A comprehensive survey of knowledge graph-based recommender systems: technologies, development, and contributions.","volume":"12","author":"Chicaiza","year":"2021","journal-title":"Information"},{"key":"B40","doi-asserted-by":"publisher","first-page":"2180","DOI":"10.1609\/aaai.v38i17.29928","article-title":"Tree-of-reasoning question decomposition for complex question answering with large language models.","volume":"38","author":"Zhang","year":"2024","journal-title":"Proc. AAAI Confer Artif Intell."},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.14464","article-title":"Tree of reviews: a tree-based dynamic iterative retrieval framework for multi-hop question answering.","author":"Jiapeng","year":"2024","journal-title":"arXiv [Preprint]."},{"key":"B42","doi-asserted-by":"publisher","first-page":"1575","DOI":"10.3390\/s22041575","article-title":"Learning to reason on tree structures for knowledge-based visual question answering.","volume":"22","author":"Li","year":"2022","journal-title":"Sensors."},{"key":"B43","doi-asserted-by":"publisher","first-page":"1395","DOI":"10.3390\/electronics13071395","article-title":"Traditional Chinese medicine knowledge graph construction based on large language models.","volume":"13","author":"Zhang","year":"2024","journal-title":"Electronics"},{"key":"B44","first-page":"15","article-title":"T-know: a knowledge graph-based question answering and infor-mation retrieval system for traditional Chinese medicine.","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics: System Demonstrations.","author":"Liu","year":"2018"},{"key":"B45","doi-asserted-by":"publisher","first-page":"102535","DOI":"10.1016\/j.artmed.2023.102535","article-title":"Knowledge graph assisted end-to-end medical dialog generation.","volume":"139","author":"Varshney","year":"2023","journal-title":"Artif Intell Med"},{"key":"B46","unstructured":"Kang\n              M\n            \n            \n              Kwak\n              JM\n            \n            \n              Baek\n              J\n            \n            \n              Hwang\n              SJ.\n            \n          \n          Knowledge-Consistent Dialogue Generation with Knowledge Graphs.\n          \n          2022"},{"key":"B47","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.04136","article-title":"Knowledge-augmented language model prompting for zero-shot knowledge graph question answering.","author":"Baek","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2405.12035","article-title":"Kg-rag: bridging the gap between knowledge and creativity.","author":"Sanmartin","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.16130","article-title":"From local to global: a graph rag approach to query-focused summarization.","author":"Edge","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B50","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2025.findings-emnlp.168","article-title":"Cot-rag: integrating chain of thought and retrieval-augmented generation to enhance reasoning in large language models.","author":"Li","year":"2025","journal-title":"arXiv [Preprint]."},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.06155","article-title":"Ernie-layout: layout knowledge enhanced pre-training for visually-rich document understanding.","author":"Peng","year":"2022","journal-title":"arXiv [Preprint]."},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.11633","article-title":"Evaluating chatgpt\u2019s information extraction capabilities: an assessment of performance, explainability, calibration, and faithfulness.","author":"Li","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B53","doi-asserted-by":"publisher","first-page":"e55090","DOI":"10.2196\/55090","article-title":"Research on traditional Chinese medicine domain knowledge graph completion and quality evaluation.","volume":"2","author":"Liu","year":"2023","journal-title":"Jmir Med Inform"},{"key":"B54","doi-asserted-by":"publisher","first-page":"7050","DOI":"10.1109\/TPAMI.2021.3121646","article-title":"Logicenn: a neural based knowledge graphs embedding model with logical rules.","volume":"45","author":"Nayyeri","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell."},{"key":"B55","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511763113","volume-title":"Recommender Systems: an Introduction.","author":"Jannach","year":"2010"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.09774","article-title":"Huatuogpt-ii, one-stage training for medical adaption of llms.","author":"Chen","year":"2023","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.03302","article-title":"How easily do irrelevant inputs skew the responses of large language models?","author":"Wu","year":"2024","journal-title":"arXiv E-Prints [Preprint]."},{"key":"B58","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.05391","article-title":"Pp-structurev2: a stronger document analysis system.","author":"Li","year":"2022","journal-title":"arXiv E-Prints [Preprint]."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1752778\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T14:35:46Z","timestamp":1773326146000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1752778\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,12]]},"references-count":58,"alternative-id":["10.3389\/fmed.2026.1752778"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1752778","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,12]]},"article-number":"1752778"}}