{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:28:08Z","timestamp":1787027288796,"version":"build-2736575974"},"reference-count":19,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T00:00:00Z","timestamp":1779667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models show promise in medical tasks, but their systematic error patterns in high-stakes clinical settings remain poorly understood, limiting safe deployment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      A three-phase simulation study was conducted. In phase 1, researchers selected 103 multiple-choice questions and 30 clinical scenario questions, derived from an LBP examination question bank and clinical guidelines and systematically evaluated five mainstream LLMs (GPT-5, GPT-4o, GPT-o3, Deepseek-V2.5, and Grok-4) across six dimensions: accuracy, completeness, practicality, readability, safety, and output stability. In Phase 2, two clinical coders independently performed qualitative content analysis on responses with low scores from Phase 1 (\u2264 3 any dimension), classified error types, and calculated inter-rater reliability (Cohen's \u03ba = 0.84); consensus was reached through discussion. In Phase 3, targeted safety-oriented prompts were designed for the high-risk error categories identified in Phase 2, and a separate linear mixed model was fitted for each of the five evaluation dimensions (\n                      <jats:italic>n<\/jats:italic>\n                      = 7 questions). Given the small sample size, the effect size (Hedges' g) serves as the primary indicator of practical significance.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      All five models achieved accuracy rates exceeding 90% on the general LBP knowledge test, demonstrating solid foundational knowledge. GPT-4o exhibited the highest overall clinical quality score and output stability. Error attribution revealed that lower-performing models, particularly Deepseek-V2.5, produced more safety-critical errors, including factual hallucinations and omissions of safety warnings. Targeted prompt engineering produced significant improvements for Deepseek-V2.5 across all five evaluation dimensions (\n                      <jats:italic>p<\/jats:italic>\n                      &amp;lt; 0.001), with the largest gains in safety and completeness. GPT-4o showed significant improvements in four dimensions but not in safety (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.227). A significant model \u00d7 condition interaction was observed only for safety (\n                      <jats:italic>p<\/jats:italic>\n                      = 0.002).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Although all models demonstrated strong foundational medical knowledge, their ability to translate this knowledge into reliable clinical guidance varied substantially. Critical concerns extend beyond factual accuracy to encompass safety and completeness in real-world clinical contexts. Human oversight remains indispensable. Clinicians should recognize the distinct strengths and limitations of different models and select tools according to their specific clinical use cases. Structured prompt design and systematic fact-checking represent the most practical and scalable approaches to enhancing safety, particularly in resource-limited settings. This study contributes to a more nuanced understanding of LLMs' capabilities and risks in chronic disease management and provides a replicable methodological foundation for future clinical AI evaluation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/frai.2026.1811701","type":"journal-article","created":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T06:06:08Z","timestamp":1779689168000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Deficiencies in clinical reasoning of LLMs in low back pain management and remediation via prompt engineering: from performance evaluation to error diagnosis"],"prefix":"10.3389","volume":"9","author":[{"given":"Jia-Hui","family":"Luo","sequence":"first","affiliation":[{"name":"School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Medicine of Shanghai Pudong New Area People's Hospital","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Therapy, The Second Rehabilitation Hospital of Shanghai","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi-Lin","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Orthopedic, Guanghua Hospital of Integrative Chinese and Western Medicine","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min-Jun","family":"Zhao","sequence":"additional","affiliation":[{"name":"School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Medicine of Shanghai Pudong New Area People's Hospital","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Therapy, The Second Rehabilitation Hospital of Shanghai","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian-Li","family":"Yin","sequence":"additional","affiliation":[{"name":"School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Medicine of Shanghai Pudong New Area People's Hospital","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Therapy, The Second Rehabilitation Hospital of Shanghai","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dao-Fang","family":"Ding","sequence":"additional","affiliation":[{"name":"School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Therapy, The Second Rehabilitation Hospital of Shanghai","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu-Bo","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Rehabilitation Science, Shanghai University of Traditional Chinese Medicine","place":["Shanghai, China"]},{"name":"Department of Rehabilitation Medicine of Shanghai Pudong New Area People's Hospital","place":["Shanghai, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,25]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"600","DOI":"10.1038\/s41746-025-01963-x","article-title":"The evaluation illusion of large language models in medicine","volume":"8","author":"Agrawal","year":"2025","journal-title":"NPJ Digit. Med."},{"key":"B2","doi-asserted-by":"publisher","first-page":"S192","DOI":"10.1007\/s00586-006-1072-1","article-title":"European guidelines for the management of chronic nonspecific low back pain","volume":"2","author":"Airaksinen","year":"2006","journal-title":"Euro. Spine J."},{"key":"B3","doi-asserted-by":"publisher","first-page":"1499","DOI":"10.1186\/s12909-025-08084-9","article-title":"Evaluating GPT-4o in high-stakes medical assessments: performance and error analysis on a Chilean anesthesiology exam","volume":"25","author":"Altermatt","year":"2025","journal-title":"BMC Med. Educ."},{"key":"B4","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1038\/s41746-025-01670-7","article-title":"A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation","volume":"8","author":"Asgari","year":"2025","journal-title":"NPJ Digit. Med."},{"key":"B5","article-title":"Clinical knowledge in LLMs does not translate to human interactions","author":"Bean","year":"2025","journal-title":"arXiv [Preprint]"},{"key":"B6","doi-asserted-by":"publisher","first-page":"132","DOI":"10.16499\/j.cnki.1003-5397.2020.01.014","article-title":"Analysis of key factors influencing text reading difficulty grading and research on readability formulas based on a corpus of chinese language textbooks","volume":"1","author":"Cheng","year":"2020","journal-title":"Lang. Writ. Appl."},{"key":"B7","doi-asserted-by":"publisher","first-page":"e670","DOI":"10.1016\/S2665-9913(23)00232-1","article-title":"Global, regional, and national burden of other musculoskeletal disorders, 1990\u20132020, and projections to 2050: a systematic analysis of the global burden of disease study 2021","volume":"5","year":"2023","journal-title":"Lancet Rheumatol."},{"key":"B8","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1038\/s41746-025-01996-2","article-title":"Benchmarking large language models for personalized, biomarker-based health intervention recommendations","volume":"8","author":"Jarchow","year":"2025","journal-title":"NPJ Digit. Med."},{"key":"B9","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit. Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"e70535","DOI":"10.2196\/70535","article-title":"Unveiling the potential of large language models in transforming chronic disease management: mixed methods systematic review","volume":"27","author":"Li","year":"2025","journal-title":"J. Med. Internet Res."},{"key":"B11","doi-asserted-by":"publisher","first-page":"715","DOI":"10.1055\/a-2375-3671","volume":"86","author":"Moser","year":"2024","journal-title":"Gesundheitswesen"},{"key":"B12","unstructured":"Low Back Pain and Sciatica in Over 16s: assessment and Management\n          \n          2016"},{"key":"B13","article-title":"A real-world evaluation of LLM medication safety reviews in NHS primary care","author":"Normand","year":"2025","journal-title":"arXiv [Preprint]"},{"key":"B14","doi-asserted-by":"publisher","first-page":"514","DOI":"10.7326\/M16-2367","article-title":"Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American college of physicians","volume":"166","author":"Qaseem","year":"2017","journal-title":"Ann. of Intern. Med."},{"key":"B15","doi-asserted-by":"publisher","first-page":"e58329","DOI":"10.2196\/58329","article-title":"Evaluation framework of large language models in medical documentation: development and usability study","volume":"26","author":"Seo","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"B16","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1038\/s41591-024-03423-7","article-title":"Toward expert-level medical question answering with large language models","volume":"31","author":"Singhal","year":"2025","journal-title":"Nat. Med."},{"key":"B17","doi-asserted-by":"publisher","first-page":"1658316","DOI":"10.3389\/frai.2025.1658316","article-title":"Prompt engineering for accurate statistical reasoning with large language models in medical research","volume":"8","author":"Vilakati","year":"2025","journal-title":"Front. Artif. Intell."},{"key":"B18","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1038\/s41746-025-02277-8","article-title":"A novel evaluation benchmark for medical LLMs illuminating safety and effectiveness in clinical domains","volume":"9","author":"Wang","year":"2025","journal-title":"NPJ Digit. Med."},{"key":"B19","doi-asserted-by":"publisher","first-page":"1573828","DOI":"10.3389\/fpubh.2025.1573828","article-title":"Global burden of occupational ergonomic factor-induced low back pain, 1990~2021: data analysis and projections of the global burden of disease","volume":"13","author":"Wu","year":"2025","journal-title":"Front. Public Health"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1811701\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,25]],"date-time":"2026-05-25T06:06:10Z","timestamp":1779689170000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2026.1811701\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,25]]},"references-count":19,"alternative-id":["10.3389\/frai.2026.1811701"],"URL":"https:\/\/doi.org\/10.3389\/frai.2026.1811701","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,25]]},"article-number":"1811701"}}