{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:52:48Z","timestamp":1791478368729,"version":"4.3.4"},"reference-count":78,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"\n Background<\/jats:title>\n Traditional Chinese medicine (TCM) has received increasing attention in evidence-based medicine. However, its largely experience-based and practice-oriented nature has slowed modernization and standardization. Advances in AI provide new opportunities for integrating AI with TCM.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study evaluates the application of large language models (LLMs), including ChatGPT (OpenAI) and DeepSeek, in generating acupuncture treatment protocols. We compare the validity and clinical effectiveness of physician-generated protocols with those produced by LLMs, examine differences in outputs across language corpora (Chinese and English), and establish a standardized assessment framework for systematically assessing the effectiveness of acupuncture point selection. Ultimately, this research aims to support the standardization of acupoint selection and promote greater consistency, reliability, and evidence-based development in acupuncture practice.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Qualified acupuncture treatment cases published in Acupuncture in Medicine were identified and translated from English into Chinese to create a bilingual case dataset. Standardized case prompts were independently input into DeepSeek and ChatGPT to generate acupuncture treatment protocols. Ten senior TCM physicians evaluated the AI-generated and physician-generated protocols using 7 criteria: local point selection, distal point selection, syndrome-based point selection, meridian-tracing point selection, neuroanatomical point selection, core acupoint selection, and synergistic effects. Repeated-measures ANOVA with sphericity assessment was used to examine differences across LLMs, language corpora, and protocol groups.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Blinded expert evaluations revealed significant differences among the 5 acupuncture groups for local point selection (P=.006), syndrome-based point selection (P=.005), neuroanatomical point selection (P<.001), and core acupoints coverage principle (P=.03). For local point selection, DeepSeek Chinese output received higher scores than the other evaluated outputs, while ChatGPT English output scored higher than ChatGPT Chinese output. For syndrome-based point selection, the journal physician-generated protocols received lower scores than the DeepSeek Chinese, DeepSeek English, and ChatGPT Chinese outputs. For neuroanatomical point selection, DeepSeek English output achieved the highest scores among the 5 groups. For core acupoints coverage, the journal physician-generated protocols received lower scores than the DeepSeek Chinese, DeepSeek English, and ChatGPT English outputs. No significant differences were observed among the 5 groups for distal point selection, meridian-tracing point selection, or therapeutic synergy. Overall, DeepSeek performed better in the Chinese-language setting, whereas ChatGPT performed better in the English-language setting.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n AI has demonstrated substantial progress in encoding explicit acupuncture knowledge and performing systematic acupoint selection. However, challenges remain in individualized treatment and consistency across linguistic contexts. Enhancing bilingual or multilingual training corpora and developing syndrome-specific reasoning modules may improve the clinical applicability of AI-assisted TCM systems.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/95851","type":"journal-article","created":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:05:09Z","timestamp":1791475509000},"page":"e95851","source":"Crossref","is-referenced-by-count":0,"title":["Comparative Evaluation of ChatGPT, DeepSeek, and Physician-Generated Acupuncture Treatment Protocols Across Chinese and English Language Settings: Cross-Sectional Evaluation Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2384-8330","authenticated-orcid":false,"given":"Xin","family":"Liu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-1063-9047","authenticated-orcid":false,"given":"Zining","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,8]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1177\/09645284221146200"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1001\/archinternmed.2012.3654"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.actpsy.2025.104728"},{"key":"ref4","unstructured":"NHE fact sheetCenters for Medicare & Medicaid Services20266242026-09-11https:\/\/www.cms.gov\/data-research\/statistics-trends-and-reports\/national-health-expenditure-data\/nhe-fact-sheet"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1142\/S0192415X24500617"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1136\/acupmed-2012-010178"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1155\/2013\/710892"},{"key":"ref8","unstructured":"BurmanTAmericans are using AI to diagnose their health issuesNewsweek20257202026-09-11https:\/\/www.newsweek.com\/ai-healthcare-diagnosis-chatgpt-doctor-2100091"},{"key":"ref9","unstructured":"HuJRuderSSiddhantANeubigGFiratOJohnsonMXTREME: a massively multilingual multi-task benchmark for evaluating cross-lingual generalisation2020Proceedings of the 37th International Conference on Machine LearningJuly 13 to July 18, 2020Virtual event44114421"},{"key":"ref10","unstructured":"Traditional medicineWorld Health Organization2026-09-11https:\/\/www.who.int\/standards\/classifications\/frequently-asked-questions\/traditional-medicine"},{"key":"ref11","unstructured":"Use of complementary health approaches for pain by U.S. adults increased from 2002 to 2022National Center for Complementary and Integrative Health2026-09-11https:\/\/www.nccih.nih.gov\/research\/research-results\/use-of-complementary-health-approaches-for-pain-by-us-adults-increased-from-2002-to-2022"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1007\/s11606-020-05871-6"},{"key":"ref13","unstructured":"Acupuncture for chronic lower back pain (cLBP) (30.3.3)Centers for Medicare & Medicaid Services2026-09-11https:\/\/www.cms.gov\/medicare-coverage-database\/view\/ncd.aspx?NCDId=373"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.5365\/wpsar.2025.16.3.1245"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1038\/480S94a"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpain.2017.11.005"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1097\/hm9.0000000000000089"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1002\/cam4.5962"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1016\/j.imr.2025.101123"},{"key":"ref20","unstructured":"Battlefield acupunctureU.S. Department of Veterans Affairs20218262026-09-11https:\/\/www.research.va.gov\/currents\/0821-Battlefield-acupuncture.cfm"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1016\/j.jtcms.2017.08.005"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1186\/s12910-025-01316-z"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1016\/j.jep.2012.01.033"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.312.7023.71"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.2147\/TCRM.S523279"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1089\/acm.2019.0197"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1000261"},{"key":"ref28","unstructured":"The state of AI in early 2024: Gen AI adoption spikes and starts to generate valueQuantumBlack, AI by McKinsey20245302026-09-11https:\/\/www.mckinsey.com\/capabilities\/quantumblack\/our-insights\/the-state-of-ai-2024"},{"key":"ref29","unstructured":"The 2025 AI index reportStanford Institute for Human-Centered Artificial Intelligence2026-09-11https:\/\/hai.stanford.edu\/ai-index\/2025-ai-index-report"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2017.07.007"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1056\/cat.23.0404"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1016\/j.slast.2025.100263"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1136\/bmjhci-2022-100549"},{"key":"ref34","doi-asserted-by":"publisher","DOI":"10.1177\/23333928241234863"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.38166"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.1097\/WNO.0000000000002274"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1186\/s13054-025-05468-7"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.2147\/JMDH.S538253"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2025.1601529"},{"key":"ref41","unstructured":"BathgateRMicrosoft is doubling down on multilingual large language models and Europe stands to benefit the mostITPro20257212026-09-11https:\/\/www.itpro.com\/technology\/artificial-intelligence\/microsoft-is-doubling-down-on-multilingual-large-language-models-and-europe-stands-to-benefit-the-most"},{"key":"ref42","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-026-01631-5"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1177\/21582440231204124"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.2196\/47427"},{"key":"ref45","unstructured":"CurryDAI app revenue and usage statisticsBusiness of Apps20266112026-09-11https:\/\/www.businessofapps.com\/data\/ai-app-market\/"},{"key":"ref46","unstructured":"Introducing GPT-5OpenAI2025872026-09-11https:\/\/openai.com\/index\/introducing-gpt-5\/"},{"key":"ref47","unstructured":"Introducing DeepSeek-V3DeepSeek2026-07-14https:\/\/api-docs.deepseek.com\/news\/news1226"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2412.19437"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.4103\/jin.jin_145_24"},{"issue":"3","key":"ref50","doi-asserted-by":"crossref","first-page":"E269","DOI":"10.36076\/ppj.2021\/24\/E269","volume":"24","author":"Atalay, SG","year":"2021","journal-title":"Pain Physician"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/410979"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1007\/s11726-007-0314-z"},{"key":"ref53","unstructured":"WHO international standard terminologies on traditional Chinese medicineWorld Health Organization2022332026-09-11https:\/\/www.who.int\/publications\/i\/item\/9789240042322"},{"key":"ref54","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2024.108074"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1016\/j.joim.2025.09.004"},{"key":"ref56","doi-asserted-by":"publisher","DOI":"10.21037\/lcm-21-48"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1096\/fj.01-0925fje"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1016\/j.pneurobio.2008.05.004"},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1155\/2020\/2948292"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/310591"},{"key":"ref61","author":"Liang, F","year":"2021","journal-title":"Zhenjiu Xue Acupuncture and Moxibustion"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.1089\/acm.2016.0016"},{"key":"ref63","doi-asserted-by":"publisher","DOI":"10.1155\/2012\/409568"},{"key":"ref64","author":"Rang, HP","year":"2011","journal-title":"Rang & Dale's Pharmacology. 7th ed"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1016\/j.ctcp.2025.102019"},{"key":"ref66","unstructured":"WHO standard acupuncture point locations in the Western Pacific regionWorld Health Organization: Regional Office for the Western Pacific20082026-09-11https:\/\/iris.who.int\/items\/f188654a-d8a7-4519-9979-8e2de713c060"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.14165"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btz682"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v39i27.35038"},{"key":"ref70","author":"Polanyi, M","year":"2009","journal-title":"The Tacit Dimension"},{"key":"ref71","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780195092691.001.0001","author":"Nonaka, I","year":"1995","journal-title":"The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation"},{"key":"ref72","doi-asserted-by":"publisher","DOI":"10.1016\/j.socscimed.2008.03.006"},{"key":"ref73","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2"},{"key":"ref74","doi-asserted-by":"publisher","DOI":"10.1089\/acu.2018.1297"},{"key":"ref75","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gkh061"},{"key":"ref76","doi-asserted-by":"publisher","DOI":"10.1016\/j.dcmed.2022.12.004"},{"key":"ref77","doi-asserted-by":"publisher","DOI":"10.21037\/atm-22-6431"},{"key":"ref78","doi-asserted-by":"publisher","DOI":"10.54254\/2977-3903\/2025.25627"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e95851","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T16:05:12Z","timestamp":1791475512000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e95851"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,8]]},"references-count":78,"URL":"https:\/\/doi.org\/10.2196\/95851","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.95851","asserted-by":"object"}]},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,8]]}}}