{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T13:53:03Z","timestamp":1775742783520,"version":"3.50.1"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T00:00:00Z","timestamp":1775692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"\n Background<\/jats:title>\n Jin San Zhen acupuncture therapy is a classical Traditional Chinese Medicine (TCM) school originating from the Lingnan region of China. It is widely used in China for central nervous system diseases, internal medical conditions, and various pain disorders, benefiting a large number of patients. However, the related clinical evidence and expert experience are scattered across journal articles and monographs, without systematic curation or structured presentation, making it difficult for frontline clinicians and trainees to access in a timely manner. Although general-purpose large language models (LLMs) can generate answers, they are prone to \u201challucinations\u201d and lack traceable evidence-based support.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n Based on Chinese clinical research literature and authoritative monographs from the past decade, this study aimed to construct a Knowledge Graph (KG) for Jin San Zhen and to develop an intelligent question\u2013answering (QA) system that combines the KG with LLMs to answer clinical and educational questions related to Jin San Zhen.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n We searched Chinese databases such as China National Knowledge Infrastructure (CNKI), Wanfang, and CQVIP for clinical studies published between 2016 and 2025 in which Jin San Zhen was the main intervention, and incorporated information on point combinations and clinical practice from four authoritative monographs. Following a PRISMA-style selection process (905 initial records \u2192 416 after deduplication \u2192 191 included studies) we designed an ontology comprising seven entity types (diseases, acupoints, acupoint combinations, treatment plans, etc.) and nine relation types. We used the Qwen3-MAX LLM for information extraction, supplemented by manual verification, and ultimately constructed the KG in Neo4j. We evaluated the KG intrinsically using Precision, Recall, and F1 metrics against a human-annotated gold standard derived from stratified sampling (\n n<\/jats:italic>\n = 149 treatment plans from\n N<\/jats:italic>\n = 298, 95% confidence level, 5% margin of error), with inter-annotator agreement assessed on 81 overlapping annotations. We then designed a retrieval-augmented generation (RAG) workflow, in which user queries are parsed by an LLM into a limited set of query types, Cipher templates are used to retrieve the graph, structured records are returned, and the LLM generates natural language answers that can be traced back to the original literature. We described the scale and characteristics of the KG using node and relation statistics, and developed 60 evaluation questions covering common query types and major disease categories. Two TCM acupuncture experts were invited to rate, under a double-blind design, the answers produced by three systems\u2014a \u201cKG+LLM template model,\u201d a \u201cKG+LLM hybrid model\u201d incorporating fuzzy entity matching and enhanced retrieval strategies, and an \u201cLLM-only model\u201d\u2014on three dimensions (correctness, professionalism, and completeness) using a 1\u20135 scale. Paired\n t<\/jats:italic>\n -tests were used to compare differences across all pairwise model combinations.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n The final KG contained 921 nodes and 3,745 relations, including more than 80 diseases, over 360 standardized acupoints, 55 core acupoint combinations, and 298 treatment plans, systematically representing the \u201cdisease\u2013plan\u2013acupoint\u201d relationships and efficacy characteristics of Jin San Zhen. Intrinsic evaluation showed that the KG achieved post-refinement F1 scores of 0.952 for main acupoints (\n P<\/jats:italic>\n = 0.959,\n R<\/jats:italic>\n = 0.949) and 0.859 for auxiliary acupoints (\n P<\/jats:italic>\n = 0.984,\n R<\/jats:italic>\n = 0.858), with inter-annotator F1 of 0.991 and 0.999, respectively. Across 60 evaluation questions, the KG \u00b1 LLM hybrid model achieved the highest mean scores on all three dimensions (correctness: 5.00; professionalism: 5.00; completeness: 4.40), significantly outperforming both the KG \u00b1 LLM template model (4.75, 4.77, 4.03) and the LLM-only model (4.05, 3.65, 4.12; all pairwise comparisons\n p<\/jats:italic>\n &lt; 0.01). Notably, the hybrid model resolved the completeness limitation observed in the template-based approach, while both KG-enhanced systems produced answers fully traceable to source literature across all 60 questions, with no fabricated claims detected by expert reviewers. Expert feedback indicated that the hybrid model's layered presentation\u2014distinguishing high-confidence graph-derived content from supplementary general knowledge\u2014provides particularly strong clinical reference value.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusion<\/jats:title>\n Compared with a general-purpose LLM, the Jin San Zhen knowledge-graph\u2013based QA system\u2014particularly with the tiered confidence generation strategy\u2014markedly improves the accuracy, professionalism, and completeness of answers while providing traceable evidence with explicit confidence labeling. The system thus has the potential to serve as an auxiliary tool for primary care and general practitioners to rapidly access information on Jin San Zhen, perform evidence integration, and support teaching. Future prospective studies in real-world clinical settings are needed to evaluate its actual impact on decision quality and patient outcomes.<\/jats:p>\n <\/jats:sec>","DOI":"10.3389\/fmed.2026.1755583","type":"journal-article","created":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T12:54:21Z","timestamp":1775739261000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Construction and evaluation of the knowledge graph and large model question-answering system for Jin San Zhen therapy: a tool study for primary care and general practice"],"prefix":"10.3389","volume":"13","author":[{"given":"Junjie","family":"Chen","sequence":"first","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minting","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]},{"name":"Panyu Hospital of Traditional Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianchao","family":"Chen","sequence":"additional","affiliation":[{"name":"Australian Chinese Medicine Commerce Association","place":["Sydney, NSW, Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guangming","family":"Luo","sequence":"additional","affiliation":[{"name":"Guangzhou Zihetang Traditional Chinese Medicine Co., Ltd.","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Genxin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chubin","family":"Lei","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongjing","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jin","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kunru","family":"Gu","sequence":"additional","affiliation":[{"name":"School of Acupuncture and Rehabilitation Clinical Medicine, Guangzhou University of Chinese Medicine","place":["Guangzhou, Guangdong, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2026,4,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"853","DOI":"10.13460\/j.issn.1005-0957.2022.08.085","article-title":"A brief analysis of the academic characteristics of Jin San Zhen therapy","volume":"41","author":"Fan","year":"2022","journal-title":"Shanghai J Acupunct Moxibustion."},{"key":"B2","doi-asserted-by":"publisher","first-page":"26","DOI":"10.19787\/j.issn.2097-3128.2024.04.006","article-title":"Clinical efficacy of Temporal Three Needles combined with repetitive transcranial magnetic stimulation in the treatment of post-stroke aphasia","volume":"1","author":"Shi","year":"2024","journal-title":"Chin J Tradit Med Rehabil"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.13703\/j.0255-2930.20250723-k0001","article-title":"\u201cIntelligence Three Needles\u201d for Alzheimer's disease: a systematic review and mechanistic exploration","author":"Zhou RJ Li","year":"2025","journal-title":"Chin Acupunct Moxibustion."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1182","DOI":"10.13359\/j.cnki.gzxbtcm.2025.05.021","article-title":"Summary of clinical experience of Zhuang Lixing's regulating-spirit needling method in the treatment of perimenopausal generalized anxiety disorder","volume":"42","author":"Zhou","year":"2025","journal-title":"J Guangzhou Univ Chin Med."},{"key":"B5","doi-asserted-by":"publisher","first-page":"93","DOI":"10.13193\/j.issn.1673-7717.2023.02.019","article-title":"Effects of Jin San Zhen on TCM syndrome scores, cerebrovascular resistance, and carotid IMT in patients with post-stroke sequelae","volume":"41","author":"Zhang","year":"2023","journal-title":"Chin Arch Tradit Chin Med."},{"key":"B6","doi-asserted-by":"publisher","first-page":"10","DOI":"10.19917\/j.cnki.1005-0779.024085","article-title":"Clinical study of Jin San Zhen Tiaoshen needling method for mild to moderate depression in Parkinson's disease","volume":"40","author":"Chu","year":"2024","journal-title":"J Clin Acupunct Moxibustion."},{"key":"B7","volume-title":"Network Meta-analysis and clinical study of Jin San Zhen acupoint embedding therapy for pediatric cerebral palsy","author":"Li","year":"2025"},{"key":"B8","unstructured":"Meta-analysis of Jin San Zhen therapy for insomnia\n \n 238\n 45\n \n \n Zhang\n WX\n \n \n Chen\n L\n \n \n Tang\n CZ\n \n \n Chin J Libr Inform Sci Tradit Chin Med.\n 48\n 2024"},{"key":"B9","unstructured":"Clinical efficacy of warm needle moxibustion at Shoulder Three Needles combined with Zhengqing Fengtongning in treating senile frozen shoulder of wind-cold-damp impediment type and its effects on PRI scores and pain mediators\n \n 2901\n 5\n \n \n Jiang\n CL\n \n \n Zhou\n Y\n \n \n Lou\n F\n \n \n Qi\n XJ\n \n \n Chin J Gerontol.\n 45\n 2025"},{"key":"B10","unstructured":"Analysis of the therapeutic effect of Jin San Zhen on sciatica caused by lumbar disc herniation\n \n 103\n 5\n \n \n Tian\n C\n \n \n China Commun Physician.\n 40\n 2024"},{"key":"B11","doi-asserted-by":"publisher","first-page":"77","DOI":"10.19621\/j.cnki.11-3555\/r.2024.1623","article-title":"Clinical observation of Jin San Zhen acupuncture combined with intra-articular injection for senile knee osteoarthritis","volume":"32","author":"Dong","year":"2024","journal-title":"China's Naturopathy."},{"key":"B12","volume-title":"Jin Rui's Academic Thoughts and Integrated Experience of Jin San Zhen Therapy","author":"Zhuang","year":"2016"},{"key":"B13","volume-title":"Comprehensive Clinical Experience Atlas of Jin San Zhen Therapy","author":"Zhuang","year":"2017"},{"key":"B14","volume-title":"Atlas of Jin San Zhen Point Combinations.","author":"Zhuang","year":"2021"},{"key":"B15","doi-asserted-by":"publisher","first-page":"932","DOI":"10.1038\/s41591-024-03416-6","article-title":"A generalist medical language model for disease diagnosis assistance","volume":"31","author":"Liu","year":"2025","journal-title":"Nat Med."},{"key":"B16","doi-asserted-by":"publisher","first-page":"e2371","DOI":"10.52225\/narra.v5i1.2371","article-title":"Chinese generative AI models (DeepSeek and Qwen) rival ChatGPT-4 in ophthalmology queries with excellent performance in Arabic and English","volume":"5","author":"Sallam","year":"2025","journal-title":"Narra J."},{"key":"B17","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1088\/1361-6560\/adbff1","article-title":"A feasibility study of automating radiotherapy planning with large language model agents","author":"Wang","year":"2025","journal-title":"Phys Med Biol."},{"key":"B18","doi-asserted-by":"publisher","first-page":"e62857","DOI":"10.2196\/62857","article-title":"Ability of ChatGPT to replace doctors in patient education: cross-sectional comparative analysis of inflammatory bowel disease","volume":"27","author":"Yan","year":"2025","journal-title":"J Med Internet Res."},{"key":"B19","doi-asserted-by":"publisher","first-page":"PP","DOI":"10.1109\/JBHI.2025.3612415","article-title":"BianCang: a traditional Chinese Medicine Large Language Model","volume":"24","author":"Wei","year":"2025","journal-title":"IEEE J Biomed Health Inform"},{"key":"B20","doi-asserted-by":"publisher","first-page":"2019","DOI":"10.1093\/jamia\/ocae087","article-title":"Lingdan: enhancing encoding of traditional Chinese medicine knowledge for clinical reasoning tasks with large language models","volume":"31","author":"Hua","year":"2024","journal-title":"J Am Med Inform Assoc."},{"key":"B21","doi-asserted-by":"publisher","first-page":"e48433","DOI":"10.2196\/48433","article-title":"Examining real-world medication consultations and drug-herb interactions: ChatGPT performance evaluation","volume":"9","author":"Hsu","year":"2023","journal-title":"JMIR Med Educ."},{"key":"B22","doi-asserted-by":"publisher","first-page":"359","DOI":"10.1093\/jamia\/ocaf179","article-title":"AcuKG: a comprehensive knowledge graph for medical acupuncture","volume":"33","author":"Li","year":"2026","journal-title":"J Am Med Inform Assoc."},{"key":"B23","doi-asserted-by":"publisher","first-page":"2886","DOI":"10.1038\/s41591-024-03139-8","article-title":"Integrated image-based deep learning and language models for primary diabetes care","volume":"30","author":"Li","year":"2024","journal-title":"Nat Med."},{"key":"B24","doi-asserted-by":"publisher","first-page":"1512329","DOI":"10.3389\/fmed.2024.1512329","article-title":"Research on a traditional Chinese medicine case-based question-answering system integrating large language models and knowledge graphs","volume":"11","author":"Duan","year":"2024","journal-title":"Front Med."},{"key":"B25","doi-asserted-by":"publisher","first-page":"e48443","DOI":"10.2196\/48443","article-title":"Unlocking the secrets behind advanced artificial intelligence language models in deidentifying Chinese-English mixed clinical text: development and validation study","volume":"26","author":"Lee","year":"2024","journal-title":"J Med Internet Res."},{"key":"B26","doi-asserted-by":"publisher","first-page":"e75279","DOI":"10.2196\/75279","article-title":"Leveraging retrieval-augmented large language models for dietary recommendations with Traditional Chinese Medicine's Medicine food homology: algorithm development and validation","volume":"13","author":"Sha","year":"2025","journal-title":"JMIR Med Inform."},{"key":"B27","doi-asserted-by":"publisher","first-page":"9076","DOI":"10.1038\/s41467-025-64142-2","article-title":"LINS: a general medical Q&A framework for enhancing the quality and credibility of LLM-generated responses","volume":"16","author":"Wang","year":"2025","journal-title":"Nat Commun."},{"key":"B28","doi-asserted-by":"publisher","first-page":"107530","DOI":"10.1016\/j.phrs.2024.107530","article-title":"TCMChat: a generative large language model for traditional Chinese medicine","volume":"210","author":"Dai","year":"2024","journal-title":"Pharmacol Res."},{"key":"B29","doi-asserted-by":"publisher","first-page":"1390","DOI":"10.1136\/bjo-2023-324526","article-title":"Development and evaluation of a large language model of ophthalmology in Chinese","volume":"108","author":"Zheng","year":"2024","journal-title":"Br J Ophthalmol."},{"key":"B30","doi-asserted-by":"publisher","first-page":"e57674","DOI":"10.2196\/57674","article-title":"Data Set and Benchmark (MedGPTEval) to Evaluate Responses From Large Language Models in Medicine: Evaluation Development and Validation","volume":"12","author":"Xu","year":"2024","journal-title":"JMIR Med Inform."},{"key":"B31","doi-asserted-by":"publisher","first-page":"e74177","DOI":"10.2196\/74177","article-title":"Large language models in lung cancer: systematic review","volume":"27","author":"Zhong","year":"2025","journal-title":"J Med Internet Res."},{"key":"B32","doi-asserted-by":"publisher","first-page":"4019","DOI":"10.1016\/j.csbj.2024.10.050","article-title":"Assessment of Large Language Models (LLMs) in decision-making support for gynecologic oncology","volume":"23","author":"Gumilar","year":"2024","journal-title":"Comput Struct Biotechnol J."},{"key":"B33","doi-asserted-by":"publisher","first-page":"108290","DOI":"10.1016\/j.compbiomed.2024.108290","article-title":"MedChatZH: A tuning LLM for traditional Chinese medicine consultations","volume":"172","author":"Tan","year":"2024","journal-title":"Comput Biol Med."},{"key":"B34","doi-asserted-by":"publisher","first-page":"eads8932","DOI":"10.1126\/sciadv.ads8932","article-title":"Discovery of antimicrobial peptides with notable antibacterial potency by an LLM-based foundation model","volume":"11","author":"Wang","year":"2025","journal-title":"Sci Adv."},{"key":"B35","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s13326-025-00331-8","article-title":"Unveiling differential adverse event profiles in vaccines via LLM text embeddings and ontology semantic analysis","volume":"16","author":"Wang","year":"2025","journal-title":"J Biomed Semantics."},{"key":"B36","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1038\/s41746-025-01457-w","article-title":"Language models for data extraction and risk of bias assessment in complementary medicine","volume":"8","author":"Lai","year":"2025","journal-title":"NPJ Digit Med."},{"key":"B37","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1186\/1472-6882-14-362","article-title":"The quality of reports of randomized clinical trials on traditional Chinese medicine treatments: a systematic review of articles indexed in the China National Knowledge Infrastructure database from 2005 to 2012","volume":"14","author":"Li","year":"2014","journal-title":"BMC Complement Altern Med."},{"key":"B38","doi-asserted-by":"publisher","first-page":"e64963","DOI":"10.2196\/64963","article-title":"Comparing Diagnostic Accuracy of Clinical Professionals and Large Language Models: Systematic Review and Meta-Analysis","volume":"13","author":"Shan","year":"2025","journal-title":"JMIR Med Inform."},{"key":"B39","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1038\/s43856-025-00802-0","article-title":"Large language model comparisons between English and Chinese query performance for cardiovascular prevention","volume":"5","author":"Ji","year":"2025","journal-title":"Commun Med."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1755583\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T12:54:28Z","timestamp":1775739268000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1755583\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,9]]},"references-count":39,"alternative-id":["10.3389\/fmed.2026.1755583"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1755583","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,9]]},"article-number":"1755583"}}