{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T15:26:43Z","timestamp":1787239603662,"version":"build-2736575974"},"reference-count":41,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) are rapidly evolving from text-based agents to multimodal systems capable of interpreting medical images. While their textual reasoning has improved, the safety implications of this shift remain underexplored, specifically regarding the alignment between visual interpretation and textual advice in low back pain (LBP) management.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to evaluate the performance of 10 commonly used and highly representative LLMs, including paid models such as ChatGPT 5 Plus (OpenAI) and reasoning-oriented models like Qwen3-Thinking (Alibaba\/Qwen team) in the context of LBP. Specifically, it assesses their clinical accuracy, readability, and the \u201casymmetric evolution\u201d in cross-modal diagnostic consistency.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a cross-sectional quantitative evaluation integrated with an exploratory multimodal stress test. First, 10 LLMs, including ChatGPT 5 and ChatGPT 5 Plus, Gemini 3 Pro (Google DeepMind), Claude 4.5 Sonnet (Anthropic), Grok 4 (xAI), DeepSeek, Kimi K2 (Moonshot AI), Doubao (ByteDance Seed [Doubao Team]), Qwen3 (Base), and Qwen3-Thinking, addressed 25 standardized LBP inquiries. A multidisciplinary expert panel extracted 1816 individual recommendations by segmenting the models\u2019 point-by-point responses. These were independently coded for clinical accuracy against international guidelines on a 5-point Likert scale. Outcomes also included readability (Flesch Reading Ease [FRE] and Simple Measure of Gobbledygook [SMOG]), understandability (Patient Education Materials Assessment Tool [PEMAT]), and the presence of disclaimers. Second, to probe multimodal capabilities, we conducted an exploratory stress test (N=5) using a curated set of challenging clinical cases, including representative but complex LBP pathologies to assess baseline image recognition, and adversarial cases with deliberate clinical-radiological mismatches to evaluate cross-modal alignment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>In textual tasks, models achieved an overall accuracy of 88.4% (1605\/1816). ChatGPT 5 Plus demonstrated near-perfect performance, with zero serious errors. Within the only directly matched reasoning comparison, Qwen3-Thinking showed a lower serious-error rate than Qwen3 (Base). However, textual advice remained difficult to read, although understandability scores were robust. In contrast, multimodal performance in the exploratory stress test exhibited a negative divergence. In pure imaging tasks featuring complex but clinically representative pathologies, the average diagnostic score was poor, with models failing to identify core pathologies. In comprehensive diagnostic analysis tasks (Q29-Q30), models exhibited \u201ctextual masking,\u201d aligning visual findings with text cues rather than imaging evidence. Crucially, safety disclaimer coverage dropped precipitously from 82.8% (207\/250) in textual tasks to 28% (14\/50) in multimodal interactions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Under adversarial stress conditions, current LLMs exhibit a capability mismatch: expert-level textual accuracy masks vulnerable visual interpretation. This \u201chigh-confidence trap\u201d may cause users to misapply trust in textual logic to visual diagnostics. Although the matched Qwen3 comparison suggested a lower serious-error rate in the reasoning-enhanced setting, poor cross-modal alignment and the systemic absence of medical disclaimers in visual tasks pose substantial risks. Thus, current multimodal LLMs remain unready for direct patient use in LBP management.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/93522","type":"journal-article","created":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T15:35:06Z","timestamp":1783956906000},"page":"e93522-e93522","source":"Crossref","is-referenced-by-count":0,"title":["Discordance Between Textual Reasoning and Visual Interpretation in Large Language Models for Low Back Pain: Cross-Sectional Quantitative Evaluation and Exploratory Multimodal Stress Test"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-3100-2748","authenticated-orcid":false,"given":"Ziyu","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3563-4666","authenticated-orcid":false,"given":"Longhao","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3140-4721","authenticated-orcid":false,"given":"Zhizhen","family":"Lv","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6047-3165","authenticated-orcid":false,"given":"Hanzhe","family":"Lv","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-9652-6047","authenticated-orcid":false,"given":"Wei","family":"Sheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3008-7739","authenticated-orcid":false,"given":"Zicheng","family":"Wei","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0067-3145","authenticated-orcid":false,"given":"Binghao","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9348-1244","authenticated-orcid":false,"given":"Ying","family":"Shen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1907-6606","authenticated-orcid":false,"given":"Yu","family":"Tian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8220-1128","authenticated-orcid":false,"given":"Jingwen","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-1207-7212","authenticated-orcid":false,"given":"Zhifang","family":"Shen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-2755-6999","authenticated-orcid":false,"given":"Lijiang","family":"Lv","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,20]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1016\/S2665-9913(23)00098-X","article-title":"Global, regional, and national burden of low back pain, 1990\u20132020, its attributable risk factors, and projections to 2050: a systematic analysis of the Global Burden of Disease Study 2021","volume":"5","author":"Ferreira","journal-title":"Lancet Rheumatol"},{"issue":"1","key":"R2","doi-asserted-by":"publisher","DOI":"10.2196\/19928","article-title":"Utilization of self-diagnosis health chatbots in real-world settings: case study","volume":"23","author":"Fan","journal-title":"J Med Internet Res"},{"issue":"1","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s13054-025-05479-4","article-title":"Primer on large language models: an educational overview for intensivists","volume":"29","author":"Idan","journal-title":"Crit Care"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1136\/ard-2024-226202","article-title":"Assessing the performance of AI chatbots in answering patients\u2019 common questions about low back pain","volume":"84","author":"Scaff","journal-title":"Ann Rheum Dis"},{"issue":"8","key":"R5","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.28276","article-title":"Enhancing postmarketing surveillance of medical products with large language models","volume":"7","author":"Matheny","journal-title":"JAMA Netw Open"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2024.106956","article-title":"Large language model enhanced logic tensor network for stance detection","volume":"183","author":"Dai","journal-title":"Neural Netw"},{"issue":"7956","key":"R7","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","journal-title":"Nature"},{"issue":"8","key":"R8","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","journal-title":"Nat Med"},{"issue":"1","key":"R9","doi-asserted-by":"publisher","DOI":"10.1186\/s12873-025-01396-5","article-title":"Evaluation of ChatGPT-5 responses in obstetric and gynecological emergencies: concordance, readability, and clinical reliability","volume":"25","author":"Arslan","journal-title":"BMC Emerg Med"},{"issue":"9","key":"R10","doi-asserted-by":"publisher","first-page":"4815","DOI":"10.1007\/s00405-025-09524-4","article-title":"ChatGPT versus DeepSeek in head and neck cancer staging and treatment planning: guideline-based study","volume":"282","author":"Vural Camalan","journal-title":"Eur Arch Otorhinolaryngol"},{"issue":"12","key":"R11","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwae403","article-title":"A survey on multimodal large language models","volume":"11","author":"Yin","journal-title":"Natl Sci Rev"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.3389\/fcell.2025.1564054","article-title":"Evaluating multimodal ChatGPT for emergency decision-making of ocular trauma cases","volume":"13","author":"Xue","journal-title":"Front Cell Dev Biol"},{"issue":"7972","key":"R13","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","journal-title":"Nature"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1186\/s12903-025-06619-6","article-title":"A multi-dimensional performance evaluation of large language models in dental implantology: comparison of ChatGPT, DeepSeek, Grok, Gemini and Qwen across diverse clinical scenarios","volume":"25","author":"Wu","journal-title":"BMC Oral Health"},{"issue":"11","key":"R15","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1007\/s11604-024-01619-y","article-title":"Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in \"diagnosis please\" cases","volume":"42","author":"Sonoda","journal-title":"Jpn J Radiol"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-02003-4","article-title":"HONeYBEE: enabling scalable multimodal AI in oncology through foundation model-driven embeddings","volume":"8","author":"Tripathi","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"258","DOI":"10.3969\/j.issn.1004-406X.2022.03.09","article-title":"Clinical guidelines for the diagnosis and treatment of nonspecific low back pain in China","volume":"32","author":"Gu","journal-title":"Chin J Spine Spinal Cord"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n1627","article-title":"Low back pain and sciatica: summary of NICE guidance","volume":"356","author":"Bernstein","journal-title":"BMJ"},{"issue":"7","key":"R19","doi-asserted-by":"publisher","first-page":"514","DOI":"10.7326\/M16-2367","article-title":"Noninvasive treatments for acute, subacute, and chronic low back pain: a clinical practice guideline from the American College of Physicians","volume":"166","author":"Qaseem","journal-title":"Ann Intern Med"},{"issue":"10137","key":"R20","doi-asserted-by":"publisher","first-page":"2356","DOI":"10.1016\/S0140-6736(18)30480-X","article-title":"What low back pain is and why we need to pay attention","volume":"391","author":"Hartvigsen","journal-title":"Lancet"},{"issue":"10137","key":"R21","doi-asserted-by":"publisher","first-page":"2384","DOI":"10.1016\/S0140-6736(18)30488-4","article-title":"Low back pain: a call for action","volume":"391","author":"Buchbinder","journal-title":"Lancet"},{"issue":"10137","key":"R22","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(18)30725-6","article-title":"Low back pain: a major global challenge","volume":"391","author":"Clark","journal-title":"Lancet"},{"issue":"3","key":"R23","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","journal-title":"J Appl Psychol"},{"issue":"4","key":"R24","doi-asserted-by":"publisher","first-page":"399","DOI":"10.1038\/s41443-023-00682-9","article-title":"Evaluating the readability of online testosterone search results","volume":"36","author":"Pominville","journal-title":"Int J Impot Res"},{"issue":"3","key":"R25","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"Shoemaker","journal-title":"Patient Educ Couns"},{"issue":"2","key":"R26","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.8538","article-title":"Clinical validity, understandability, and actionability of online cardiovascular disease risk calculators: systematic review","volume":"20","author":"Bonner","journal-title":"J Med Internet Res"},{"issue":"1","key":"R27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000417","article-title":"Peer review of GPT-4 technical report and systems card","volume":"3","author":"Gallifant","journal-title":"PLOS Digit Health"},{"key":"R28","doi-asserted-by":"crossref","unstructured":"McGinness L Baumgartner P . Large language models imitate logical reasoning, but at what cost? In: Liu M Yu X Xu C Song Y , editors. AI 2025: Advances in Artificial Intelligence: 38th Australasian Joint Conference on Artificial Intelligence, AI 2025, Canberra, ACT, Australia, December 1\u20135, 2025, Proceedings, Part I. Springer; 2025:80-96. [doi: 10.1007\/978-981-95-4969-6_7]","DOI":"10.1007\/978-981-95-4969-6_7"},{"key":"R29","unstructured":"Hagendorff T Fabi S . Beyond chains of thought: benchmarking latent-space reasoning abilities in large language models. arXiv. Preprint posted online on Apr 14, 2025. [doi: 10.48550\/arXiv.2504.10615]"},{"issue":"4","key":"R30","doi-asserted-by":"publisher","first-page":"3129","DOI":"10.1007\/s12652-021-03439-8","article-title":"Chatbot interaction with artificial intelligence: human data augmentation with T5 and language transformer ensemble for text classification","volume":"14","author":"Bird","journal-title":"J Ambient Intell Human Comput"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Saeid Y Ne\u00fcburger F Kr\u00fcgl S H\u00fcster H Kopinski T Lanwehr R . Harnessing large language models: fine-tuned BERT for detecting charismatic leadership tactics in natural language. Presented at: 2024 IEEE 3rd Conference on Information Technology and Data Science (CITDS); Aug 26-28, 2024; Debrecen, Hungary. [doi: 10.1109\/CITDS62610.2024.10791373]","DOI":"10.1109\/CITDS62610.2024.10791373"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. Presented at: NIPS\u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems; Nov 28 to Dec 9, 2022; New Orleans, Louisiana, USA. [doi: 10.52202\/068431-1800]","DOI":"10.52202\/068431-1800"},{"key":"R33","unstructured":"Ding J Long S Pu C et al. NL2Repo-bench: towards long-horizon repository generation evaluation of coding agents. arXiv. Preprint posted online on Dec 14, 2025. [doi: 10.48550\/arXiv.2512.12730]"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/84120","article-title":"Knowledge-practice performance gap in clinical large language models: systematic review of 39 benchmarks","volume":"27","author":"Gong","journal-title":"J Med Internet Res"},{"issue":"8","key":"R35","doi-asserted-by":"publisher","first-page":"2005","DOI":"10.1007\/s40279-024-02054-9","article-title":"Judgement and decision making in clinical and return-to-sports decision making: a narrative review","volume":"54","author":"Yung","journal-title":"Sports Med"},{"key":"R36","unstructured":"Sepehri MS Fabian Z Soltanolkotabi M Soltanolkotabi M . MediConfusion: can you trust your AI radiologist? Probing the reliability of multimodal medical foundation models. Presented at: The Thirteenth International Conference on Learning Representations (ICLR 2025); Apr 24-28, 2025; Singapore. URL: https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2025\/hash\/b37c2e26b75ee02fcabd65a2a0367136-Abstract-Conference.html [Accessed 24-07-2026]"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-08987-4","article-title":"Unmasking Clever Hans predictors and assessing what machines really learn","volume":"10","author":"Lapuschkin","journal-title":"Nat Commun"},{"issue":"2","key":"R38","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1007\/s11263-019-01228-7","article-title":"Grad-CAM: visual explanations from deep networks via gradient-based localization","volume":"128","author":"Selvaraju","journal-title":"Int J Comput Vis"},{"key":"R39","doi-asserted-by":"crossref","unstructured":"Chakraborty T Shayegani E Cai Z et al. Can textual unlearning solve cross-modality safety alignment? In: Al-Onaizan Y Bansal M Chen YN , editors. Findings of the Association for Computational Linguistics: EMNLP 2024. Association for Computational Linguistics; 2024:9830-9844. [doi: 10.18653\/v1\/2024.findings-emnlp.574]","DOI":"10.18653\/v1\/2024.findings-emnlp.574"},{"issue":"1","key":"R40","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-30761-2","article-title":"Towards artificial general intelligence via a multimodal foundation model","volume":"13","author":"Fei","journal-title":"Nat Commun"},{"issue":"3","key":"R41","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1177\/0194599815626146","article-title":"Readability and understandability of online vocal cord paralysis materials","volume":"154","author":"Balakrishnan","journal-title":"Otolaryngol Head Neck Surg"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T14:41:43Z","timestamp":1787236903000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e93522"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,20]]},"references-count":41,"URL":"https:\/\/doi.org\/10.2196\/93522","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,20]]},"article-number":"v14i1e93522"}}