{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T12:58:46Z","timestamp":1792241926414,"version":"4.4.0"},"reference-count":37,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2025,12,16]],"date-time":"2025-12-16T00:00:00Z","timestamp":1765843200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Accurate deformity classification and fusion-level planning are essential in adolescent idiopathic scoliosis (AIS) surgery and are traditionally guided by Cobb angle measurement and the Lenke system. Multimodal large language models (LLMs) (e.g., ChatGPT-4.0; Claude 3.7 Sonnet, Gemini 2.5 Pro, DeepSeek-R1-0528 Chat) are increasingly used for image interpretation despite limited validation for radiographic decision-making. This study evaluated the agreement and reproducibility of contemporary multimodal LLMs for AIS assessment compared with expert spine surgeons. Methods: This single-center retrospective study included 125 AIS patients (94 females, 31 males; mean age 14.8 \u00b1 1.9 years) who underwent posterior instrumentation (2020\u20132024). Two experienced spine surgeons independently performed Lenke classification (including lumbar and sagittal modifiers) and selected fusion levels (UIV\u2013LIV) on standing AP, lateral, and side-bending radiographs; discrepancies were resolved by consensus to establish the reference standard. The same radiographs were analyzed by four paid multimodal LLMs using standardized zero-shot prompts. Because LLMs showed inconsistent end-vertebra selection, LLM-derived Cobb angles lacked a common anatomical reference frame and were excluded from quantitative analysis. Agreement with expert consensus and test\u2013retest reproducibility (repeat analyses one week apart) were assessed using Cohen\u2019s \u03ba. Evaluation times were recorded. Results: Surgeon agreement was high for Lenke classification (92.0%, \u03ba = 0.913) and fusion-level selection (88.8%, \u03ba = 0.879). All LLMs demonstrated chance-level test\u2013retest reproducibility and very low agreement with expert consensus (Lenke: 1.6\u201310.2%, \u03ba = 0.001\u20130.036; fusion: 0.8\u201312.0%, \u03ba = 0.003\u20130.053). Claude produced missing outputs in 17 Lenke and 29 fusion-level cases. Although LLMs completed assessments far faster than surgeons (seconds vs. ~11\u201312 min), speed did not translate into clinically acceptable reliability. Conclusions: Current general-purpose multimodal LLMs do not provide reliable Lenke classification or fusion-level planning in AIS. Their poor agreement with expert surgeons and marked internal inconsistency indicate that LLM-generated interpretations should not be used for surgical decision-making or patient self-assessment without task-specific validation.<\/jats:p>","DOI":"10.3390\/diagnostics15243219","type":"journal-article","created":{"date-parts":[[2025,12,16]],"date-time":"2025-12-16T15:41:12Z","timestamp":1765899672000},"page":"3219","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Reliability of Large Language Model-Based Artificial Intelligence in AIS Assessment: Lenke Classification and Fusion-Level Suggestion"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8245-2187","authenticated-orcid":false,"given":"Cemil","family":"Aktan","sequence":"first","affiliation":[{"name":"Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya 07100, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8813-0162","authenticated-orcid":false,"given":"Ak\u0131n","family":"Ko\u015far","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya 07100, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8002-6408","authenticated-orcid":false,"given":"Melih","family":"\u00dcnal","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya 07100, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2809-6721","authenticated-orcid":false,"given":"Murat","family":"Korkmaz","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Faculty of Medicine, Istanbul University, Istanbul 34093, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4547-3042","authenticated-orcid":false,"given":"\u00d6zcan","family":"Kaya","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, \u0130stinye University Medical Park TEM Hospital, Istanbul 34250, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0704-3797","authenticated-orcid":false,"given":"Turgut","family":"Akg\u00fcl","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Faculty of Medicine, Istanbul University, Istanbul 34093, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6832-064X","authenticated-orcid":false,"given":"Ferhat","family":"G\u00fcler","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Antalya Training and Research Hospital, Antalya 07100, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,12,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.nec.2012.12.006","article-title":"Adolescent scoliosis classification and treatment","volume":"24","author":"Hoashi","year":"2013","journal-title":"Neurosurg. Clin. N. Am."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"15030","DOI":"10.1038\/nrdp.2015.30","article-title":"Adolescent idiopathic scoliosis","volume":"1","author":"Cheng","year":"2015","journal-title":"Nat. Rev. Dis. Primers"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s11832-012-0457-4","article-title":"Epidemiology of adolescent idiopathic scoliosis","volume":"7","author":"Konieczny","year":"2013","journal-title":"J. Child. Orthop."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1097\/01.brs.0000162280.95076.bb","article-title":"Idiopathic scoliosis in Singapore schoolchildren: A prevalence study 15 years into the screening program","volume":"30","author":"Wong","year":"2005","journal-title":"Spine"},{"key":"ref_5","first-page":"529","article-title":"Adolescent idiopathic scoliosis: Etiology, anatomy, natural history, and bracing","volume":"54","author":"Parent","year":"2005","journal-title":"Instr. Course Lect."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1097\/01.blo.0000198725.54891.73","article-title":"Scoliosis: Surgical versus nonsurgical treatment","volume":"443","author":"Lonstein","year":"2006","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1512","DOI":"10.1056\/NEJMoa1307337","article-title":"Effects of bracing in adolescents with idiopathic scoliosis","volume":"369","author":"Weinstein","year":"2013","journal-title":"N. Engl. J. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s13013-017-0145-8","article-title":"2016 SOSORT guidelines: Orthopaedic and rehabilitation treatment of idiopathic scoliosis during growth","volume":"13","author":"Negrini","year":"2018","journal-title":"Scoliosis Spinal Disord."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1148\/radiology.134.2.6986054","article-title":"Computerized measurement and analysis of scoliosis: A more accurate representation of the shape of the curve","volume":"134","author":"Jeffries","year":"1980","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.2106\/00004623-200108000-00006","article-title":"Adolescent idiopathic scoliosis: A new classification to determine extent of spinal arthrodesis","volume":"83","author":"Lenke","year":"2001","journal-title":"J. Bone Joint Surg. Am."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1097\/00007632-200204150-00014","article-title":"Interobserver and intraobserver reliability of Lenke\u2019s new scoliosis classification system","volume":"27","author":"Ogon","year":"2002","journal-title":"Spine"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1097\/BSD.0b013e3181831ef7","article-title":"Interobserver and intraobserver variability in the identification of the Lenke classification lumbar modifier in adolescent idiopathic scoliosis","volume":"22","author":"Duong","year":"2009","journal-title":"J. Spinal Disord. Tech."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2040","DOI":"10.1111\/os.14144","article-title":"Validation of Artificial Intelligence in the Classification of Adolescent Idiopathic Scoliosis and the Compairment to Clinical Manual Handling","volume":"16","author":"Lu","year":"2024","journal-title":"Orthop. Surg."},{"key":"ref_14","unstructured":"OpenAI (2024, January 01). Introducing ChatGPT. Available online: https:\/\/openai.com\/blog\/chatgpt."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1093\/eurjcn\/zvad038","article-title":"What if your patient switches from Dr. Google to Dr. ChatGPT? A vignette-based survey of the trustworthiness, value and danger of ChatGPT-generated responses to health questions","volume":"23","author":"Moons","year":"2024","journal-title":"Eur. J. Cardiovasc. Nurs."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1016\/j.arthro.2023.01.015","article-title":"ChatGPT, an artificial intelligence chatbot, is impacting medical literature","volume":"39","author":"Lubowitz","year":"2023","journal-title":"Arthroscopy"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2058","DOI":"10.1016\/j.arthro.2023.12.009","article-title":"ChatGPT responses to common questions about anterior cruciate ligament reconstruction are frequently satisfactory","volume":"40","author":"Li","year":"2024","journal-title":"Arthroscopy"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1519","DOI":"10.2106\/JBJS.23.00209","article-title":"Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty","volume":"105","author":"Mika","year":"2023","journal-title":"J. Bone Joint Surg. Am."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1097\/BPO.0000000000002681","article-title":"ChatGPT responses to common questions about slipped capital femoral epiphysis: A reliable resource for parents?","volume":"44","author":"Adelstein","year":"2024","journal-title":"J. Pediatr. Orthop."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1037\/0022-3514.77.6.1121","article-title":"Unskilled and unaware of it: How difficulties in recognizing one\u2019s own incompetence lead to inflated self-assessments","volume":"77","author":"Kruger","year":"1999","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2360","DOI":"10.1007\/s00586-013-2693-9","article-title":"Measuring procedures to determine the Cobb angle in idiopathic scoliosis: A systematic review","volume":"22","author":"Langensiepen","year":"2013","journal-title":"Eur. Spine J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e22.00086","DOI":"10.2106\/JBJS.RVW.22.00086","article-title":"Artificial Intelligence in Modern Orthopaedics: Current and Future Applications","volume":"10","author":"Hui","year":"2022","journal-title":"JBJS Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Farhadi, F., Barnes, M.R., Sugito, H.R., Sin, J.M., Henderson, E.R., and Levy, J.J. (2022). Applications of artificial intelligence in orthopaedic surgery. Front. Med. Technol., 4.","DOI":"10.3389\/fmedt.2022.995526"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1545","DOI":"10.1007\/s43390-024-00940-w","article-title":"Applications of artificial intelligence for adolescent idiopathic scoliosis: Mapping the evidence","volume":"12","author":"Goldman","year":"2024","journal-title":"Spine Deform."},{"key":"ref_25","first-page":"821","article-title":"Lenke and King classification systems for adolescent idiopathic scoliosis: Interobserver agreement and postoperative results","volume":"4","author":"Soleimanpour","year":"2011","journal-title":"Int. J. Gen. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1097\/01.BRS.0000067265.52473.C3","article-title":"Comparison of reliability between the Lenke and King classification systems for adolescent idiopathic scoliosis using radiographs that were not premeasured","volume":"28","author":"Richards","year":"2003","journal-title":"Spine"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2103","DOI":"10.1097\/01.brs.0000231434.93884.c9","article-title":"Interobserver and intraobserver agreement of Lenke and King classifications for idiopathic scoliosis and the influence of level of professional training","volume":"31","author":"Niemeyer","year":"2006","journal-title":"Spine"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Park, S.J., Park, J.S., Kang, D.H., and Lee, C.S. (2024). The Optimal Lowest Instrumented Vertebra to Prevent the Distal Adding-On Phenomenon in Patients Undergoing Selective Thoracic Fusion for Adolescent Idiopathic Scoliosis with Lenke Type 1A and 1B Curves: Comparison of Nine Selection Criteria. J. Clin. Med., 13.","DOI":"10.3390\/jcm13133859"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1007\/s12178-023-09876-6","article-title":"Selection of Fusion Levels in Adolescent Idiopathic Scoliosis","volume":"17","author":"Baghdadi","year":"2024","journal-title":"Curr. Rev. Musculoskelet. Med."},{"key":"ref_30","first-page":"24824","article-title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1186\/s13018-025-05620-7","article-title":"A deep learning model for radiological measurement of adolescent idiopathic scoliosis using biplanar radiographs","volume":"20","author":"Xie","year":"2025","journal-title":"J. Orthop. Surg. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"5880","DOI":"10.1007\/s00330-022-08692-9","article-title":"A multi-stage ensemble network system to diagnose adolescent idiopathic scoliosis","volume":"32","author":"Wu","year":"2022","journal-title":"Eur. Radiol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1969","DOI":"10.1007\/s00586-021-07025-6","article-title":"Comparison of manual versus automated measurement of Cobb angle in idiopathic scoliosis based on a deep learning keypoint detection technology","volume":"31","author":"Sun","year":"2022","journal-title":"Eur. Spine J."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/s13018-024-05383-7","article-title":"Intelligent measurement of adolescent idiopathic scoliosis x-ray coronal imaging parameters based on VB-Net neural network: A retrospective analysis of 2092 cases","volume":"20","author":"Liu","year":"2025","journal-title":"J. Orthop. Surg Res."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Bartsch, H., Jorgensen, O., Rosati, D., Hoelscher-Obermaier, J., and Pfau, J. (2023). Self-Consistency of Large Language Models under Ambiguity. arXiv.","DOI":"10.18653\/v1\/2023.blackboxnlp-1.7"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"4164","DOI":"10.1007\/s00586-023-08052-1","article-title":"Deep learning prediction of curve severity from rasterstereographic back images in adolescent idiopathic scoliosis","volume":"33","author":"Minotti","year":"2024","journal-title":"Eur. Spine J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"100857","DOI":"10.1016\/j.landig.2025.01.013","article-title":"Importance of sample size on the quality and utility of AI-based prediction models for healthcare","volume":"7","author":"Riley","year":"2025","journal-title":"Lancet Digit. Health"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/24\/3219\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,18]],"date-time":"2025-12-18T05:15:53Z","timestamp":1766034953000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/15\/24\/3219"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,16]]},"references-count":37,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["diagnostics15243219"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics15243219","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,16]]}}}