{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T04:56:43Z","timestamp":1788411403675,"version":"build-2803163510"},"reference-count":62,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T00:00:00Z","timestamp":1746662400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background\/Objectives: Open-source AI models are increasingly applied in medical imaging, yet their effectiveness in detecting and classifying spinal stabilization systems remains underexplored. This study compares ChatGPT-4o (a large language model) and BiomedCLIP (a multimodal model) in their analysis of posturographic X-ray images (AP projection) to assess their accuracy in identifying the presence, type (growing vs. non-growing), and specific system (MCGR vs. PSF). Methods: A dataset of 270 X-ray images (93 without stabilization, 80 with MCGR, and 97 with PSF) was analyzed manually by neurosurgeons and evaluated using a three-stage AI-based questioning approach. Performance was assessed via classification accuracy, Gwet\u2019s Agreement Coefficient (AC1) for inter-rater reliability, and a two-tailed z-test for statistical significance (p &lt; 0.05). Results: The results indicate that GPT-4o demonstrates high accuracy in detecting spinal stabilization systems, achieving near-perfect recognition (97\u2013100%) for the presence or absence of stabilization. However, its consistency is reduced when distinguishing complex growing-rod (MCGR) configurations, with agreement scores dropping significantly (AC1 = 0.32\u20130.50). In contrast, BiomedCLIP displays greater response consistency (AC1 = 1.00) but struggles with detailed classification, particularly in recognizing PSF (11% accuracy) and MCGR (4.16% accuracy). Sensitivity analysis revealed GPT-4o\u2019s superior stability in hierarchical classification tasks, while BiomedCLIP excelled in binary detection but showed performance deterioration as the classification complexity increased. Conclusions: These findings highlight GPT-4o\u2019s robustness in clinical AI-assisted diagnostics, particularly for detailed differentiation of spinal stabilization systems, whereas BiomedCLIP\u2019s precision may require further optimization to enhance its applicability in complex radiographic evaluations.<\/jats:p>","DOI":"10.3390\/jcm14103282","type":"journal-article","created":{"date-parts":[[2025,5,8]],"date-time":"2025-05-08T07:41:47Z","timestamp":1746690107000},"page":"3282","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Comparative Evaluation of Large Language and Multimodal Models in Detecting Spinal Stabilization Systems on X-Ray Images"],"prefix":"10.3390","volume":"14","author":[{"given":"Bartosz","family":"Polis","sequence":"first","affiliation":[{"name":"Department of Neurosurgery, Polish-Mother\u2019s Memorial Hospital Research Institute, 93-338 Lodz, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4031-0930","authenticated-orcid":false,"given":"Agnieszka","family":"Zawadzka-Fabijan","sequence":"additional","affiliation":[{"name":"Department of Rehabilitation Medicine, Faculty of Health Sciences, Medical University of Lodz, 90-419 Lodz, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Fabijan","sequence":"additional","affiliation":[{"name":"Independent Researcher, Luton LU2 0GS, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"R\u00f3\u017ca","family":"Kosi\u0144ska","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Polish-Mother\u2019s Memorial Hospital Research Institute, 93-338 Lodz, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emilia","family":"Nowos\u0142awska","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Polish-Mother\u2019s Memorial Hospital Research Institute, 93-338 Lodz, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0998-4435","authenticated-orcid":false,"given":"Artur","family":"Fabijan","sequence":"additional","affiliation":[{"name":"Department of Neurosurgery, Polish-Mother\u2019s Memorial Hospital Research Institute, 93-338 Lodz, Poland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,5,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Daeschler, S.C., Bourget, M.H., Derakhshan, D., Sharma, V., Asenov, S.I., Gordon, T., Cohen-Adad, J., and Borschel, G.H. (2022). Rapid, automated nerve histomorphometry through open-source artificial intelligence. Sci. Rep., 12.","DOI":"10.1038\/s41598-022-10066-6"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hentschel, S., Kobs, K., and Hotho, A. (2022). CLIP knows image aesthetics. Front. Artif. Intell., 5.","DOI":"10.3389\/frai.2022.976235"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s12032-022-01711-1","article-title":"Artificial intelligence and machine learning in precision and genomic medicine","volume":"39","author":"Quazi","year":"2022","journal-title":"Med. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.jebdp.2016.07.002","article-title":"Artificial intelligence expert systems with neural network machine learning may assist decision-making for extractions in orthodontic treatment planning","volume":"16","author":"Takada","year":"2016","journal-title":"J. Evid. Based Dent. Pract."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"baaa010","DOI":"10.1093\/database\/baaa010","article-title":"Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine","volume":"2020","author":"Ahmed","year":"2020","journal-title":"Database"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1213\/ANE.0000000000006047","article-title":"Artificial Intelligence and Machine Learning in Patient Blood Management: A Scoping Review","volume":"135","author":"Meier","year":"2022","journal-title":"Anesth. Analg."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Polis, B., Zawadzka-Fabijan, A., Fabijan, R., Kosi\u0144ska, R., Nowos\u0142awska, E., and Fabijan, A. (2025). Exploring BiomedCLIP\u2019s Capabilities in Medical Image Analysis: A Focus on Scoliosis Detection and Severity Assessment. Appl. Sci., 15.","DOI":"10.3390\/app15010398"},{"key":"ref_8","unstructured":"OpenAI (2024, July 29). CLIP: Connecting Text and Images. Available online: https:\/\/openai.com\/research\/clip."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"6989","DOI":"10.21037\/qims-23-376","article-title":"SDA-CLIP: Surgical visual domain adaptation using video and text labels","volume":"13","author":"Li","year":"2023","journal-title":"Quant. Imaging Med. Surg."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yang, W., Wang, Y., Hu, J., and Yuan, T. (2023). Sleep CLIP: A Multimodal Sleep Staging Model Based on Sleep Signals and Sleep Staging Labels. Sensors, 23.","DOI":"10.3390\/s23177341"},{"key":"ref_11","unstructured":"(2024, September 02). BiomedCLIP-PubMedBERT_256-vit_base_patch16_224. Available online: https:\/\/huggingface.co\/microsoft\/BiomedCLIP-PubMedBERT_256-vit_base_patch16_224."},{"key":"ref_12","first-page":"e43958","article-title":"Artificial Intelligence (AI) in Radiology: A Deep Dive Into ChatGPT 4.0\u2019s Accuracy with the American Journal of Neuroradiology\u2019s (AJNR) \u201cCase of the Month\u201d","volume":"15","author":"Suthar","year":"2023","journal-title":"Cureus"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Lee, K.-H., Lee, R.-W., and Kwon, Y.-E. (2024). Validation of a Deep Learning Chest X-ray Interpretation Model: Integrating Large-Scale AI and Large Language Models for Comparative Analysis with ChatGPT. Diagnostics, 14.","DOI":"10.3390\/diagnostics14010090"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1131","DOI":"10.1007\/s10439-023-03335-6","article-title":"A Promising Start and Not a Panacea: ChatGPT\u2019s Early Impact and Potential in Medical Science and Biomedical Engineering Research","volume":"52","author":"Sohail","year":"2023","journal-title":"Ann. Biomed. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Fabijan, A., Zawadzka-Fabijan, A., Fabijan, R., Zakrzewski, K., Nowos\u0142awska, E., and Polis, B. (2024). Artificial Intelligence in Medical Imaging: Analyzing the Performance of ChatGPT and Microsoft Bing in Scoliosis Detection and Cobb Angle Assessment. Diagnostics, 14.","DOI":"10.3390\/diagnostics14070773"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Maharathi, S., Iyengar, R., and Chandrasekhar, P. (2023). Biomechanically designed Curve Specific Corrective Exercise for Adolescent Idiopathic Scoliosis gives significant outcomes in an Adult: A case report. Front. Rehabil. Sci., 4.","DOI":"10.3389\/fresc.2023.1127222"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.compbiomed.2016.01.026","article-title":"Improvement of automated image stitching system for DR X-ray images","volume":"71","author":"Yang","year":"2016","journal-title":"Comput. Biol. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.ejrad.2018.09.014","article-title":"Radiation dose for pediatric scoliosis patients undergoing whole spine radiography: Effect of the radiographic length in an auto-stitching digital radiography system","volume":"108","author":"Hwang","year":"2018","journal-title":"Eur. J. Radiol."},{"key":"ref_19","unstructured":"(2024, April 10). Patient examination AO Surgery Reference. Available online: https:\/\/surgeryreference.aofoundation.org\/spine\/deformities\/adolescent-idiopathic-scoliosis\/further-reading\/patient-examination."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"534","DOI":"10.1186\/s40001-024-02052-7","article-title":"Long-term outcomes of spinal fusion in adolescent idiopathic scoliosis: A literature review","volume":"29","author":"Pishnamaz","year":"2024","journal-title":"Eur. J. Med. Res."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Maccaferri, B., Vommaro, F., Cini, C., Filardo, G., Boriani, L., and Gasbarrini, A. (2024). Surgical Treatment of Early-Onset Scoliosis: Traditional Growing Rod vs. Magnetically Controlled Growing Rod vs. Vertical Expandable Prosthesis Titanium Ribs. J. Clin. Med., 14.","DOI":"10.3390\/jcm14010177"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1348\/000711006X126600","article-title":"Computing inter-rater reliability and its variance in the presence of high agreement","volume":"61","author":"Gwet","year":"2008","journal-title":"Br. J. Math. Stat. Psychol."},{"key":"ref_23","unstructured":"Gwet, K.L. (2008). Handbook of Inter-Rater Reliability: The Definitive Guide to Measuring the Extent of Agreement Among Raters, Advanced Analytics, LLC."},{"key":"ref_24","unstructured":"Gwet, K.L. (2014). Agreement Coefficients for Nominal and Ordinal Data, Advanced Analytics, LLC."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wongpakaran, N., Wongpakaran, T., Wedding, D., and Gwet, K.L. (2013). A comparison of Cohen\u2019s Kappa and Gwet\u2019s AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples. BMC Med. Res. Methodol., 13.","DOI":"10.1186\/1471-2288-13-61"},{"key":"ref_26","unstructured":"Gwet, K.L. (2025, March 01). irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC). R Package Version 1.0, CRAN 2019. Available online: https:\/\/CRAN.R-project.org\/package=irrCAC."},{"key":"ref_27","unstructured":"Makowski, D., L\u00fcdecke, D., Patil, I., Th\u00e9riault, R., Ben-Shachar, M., and Wiernik, B. (2025, March 01). Automated Results Reporting as a Practical Tool to Improve Reproducibility and Methodological Best Practices Adoption. CRAN 2023. Available online: https:\/\/easystats.github.io\/report\/."},{"key":"ref_28","unstructured":"Pedersen, T. (2025, March 01). Patchwork: The Composer of Plots. R Package Version 1.2.0. Available online: https:\/\/CRAN.R-project.org\/package=patchwork."},{"key":"ref_29","unstructured":"R Core Team (2021). R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing. Available online: https:\/\/www.R-project.org\/."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"570","DOI":"10.32614\/RJ-2021-053","article-title":"Reproducible Summary Tables with the gtsummary Package","volume":"13","author":"Sjoberg","year":"2021","journal-title":"R. J."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wickham, H. (2016). Ggplot2: Elegant Graphics for Data Analysis, Springer.","DOI":"10.1007\/978-3-319-24277-4_9"},{"key":"ref_32","unstructured":"Hoopes, A., Butoi, V.I., Guttag, J.V., and Dalca, A.V. (2024). Voxelprompt: A Vision-Language Agent for Grounded Medical Image Analysis. arXiv, Available online: https:\/\/arxiv.org\/abs\/2410.08397."},{"key":"ref_33","unstructured":"Awais, M., Alharthi, A.H.S.A., Kumar, A., and Cholakkal, H. (2024). AgroGPT: Efficient Agricultural Vision-Language Model with Expert Tuning. arXiv, Available online: https:\/\/arxiv.org\/abs\/2410.08405."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"298","DOI":"10.5858\/arpa.2024-0215-RA","article-title":"Generative Artificial Intelligence in Anatomic Pathology","volume":"149","author":"Brodsky","year":"2025","journal-title":"Arch. Pathol. Lab. Med."},{"key":"ref_35","unstructured":"Jegham, N., Abdelatti, M., and Hendawi, A. (2025). Visual Reasoning Evaluation of Grok, Deepseek Janus, Gemini, Qwen, Mistral, and ChatGPT. arXiv, Available online: https:\/\/arxiv.org\/abs\/2502.16428."},{"key":"ref_36","unstructured":"Koleilat, T., Asgariandehkordi, H., and Rivaz, H. (2024). Medclip-samv2: Towards universal text-driven medical image segmentation. arXiv."},{"key":"ref_37","unstructured":"Neveditsin, N., Lingras, P., and Mago, V. (2024). Clinical insights: A comprehensive review of language models in medicine. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Soni, N., Ora, M., Agarwal, A., Yang, T., and Bathla, G. (2024). A Review of The Opportunities and Challenges with Large Language Models in Radiology: The Road Ahead. Am. J. Neuroradiol.","DOI":"10.3174\/ajnr.A8589"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hartsock, I., and Rasool, G. (2024). Vision-language models for medical report generation and visual question answering: A review. Front. Artif. Intell., 7.","DOI":"10.3389\/frai.2024.1430984"},{"key":"ref_40","unstructured":"Liu, Y., Cao, X., Chen, T., Jiang, Y., You, J., and Wu, M. (2025). A Survey of Embodied AI in Healthcare: Techniques, Applications, and Opportunities. arXiv."},{"key":"ref_41","unstructured":"Buess, L., Keicher, M., Navab, N., and Maier, A. (2025). From large language models to multimodal AI: A scoping review on the potential of generative AI in medicine. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"AIoa2400640","DOI":"10.1056\/AIoa2400640","article-title":"A multimodal biomedical foundation model trained from fifteen million image\u2013text pairs","volume":"2","author":"Zhang","year":"2025","journal-title":"NEJM AI"},{"key":"ref_43","unstructured":"Wang, C., Li, M., He, J., Wang, Z., Darzi, E., and Chen, Z. (2024). A survey for large language models in biomedicine. arXiv."},{"key":"ref_44","first-page":"152","article-title":"Developing ChatGPT for biology and medicine: A complete review of biomedical question answering","volume":"10","author":"Li","year":"2024","journal-title":"PMC"},{"key":"ref_45","unstructured":"Chen, Z., Pekis, A., and Brown, K. (2024). Advancing high resolution vision-language models in biomedicine. arXiv."},{"key":"ref_46","unstructured":"Chaves, J.M.Z., Huang, S.C., Xu, Y., and Xu, H. (2024). Towards a clinically accessible radiology foundation model: Open-access and lightweight, with automated evaluation. arXiv."},{"key":"ref_47","unstructured":"Bansal, H., Israel, D., Zhao, S., Li, S., and Nguyen, T. (2024). MedMax: Mixed-Modal Instruction Tuning for Training Biomedical Assistants. arXiv."},{"key":"ref_48","first-page":"103805","article-title":"Candidate-Heuristic In-Context Learning: A new framework for enhancing medical visual question answering with LLMs","volume":"61","author":"Liang","year":"2024","journal-title":"ScienceDirect"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Khan, W., Leem, S., See, K.B., Wong, J.K., Zhang, S., and Fang, R. (2025). A comprehensive survey of foundation models in medicine. IEEE Rev Biomed Eng.","DOI":"10.1109\/RBME.2025.3531360"},{"key":"ref_50","unstructured":"Ikezogwo, W.O., Zhang, K., and Seyfioglu, M.S. (2025). MedicalNarratives: Connecting Medical Vision and Language with Localized Narratives. arXiv."},{"key":"ref_51","unstructured":"Nie, Y., He, S., Bie, Y., Wang, Y., Chen, Z., and Yang, S. (2025). ConceptCLIP: Towards Trustworthy Medical AI via Concept-Enhanced Contrastive Language-Image Pre-training. arXiv."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"10147","DOI":"10.1038\/s41467-024-54424-6","article-title":"Large-scale long-tailed disease diagnosis on radiology images","volume":"15","author":"Zheng","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_53","unstructured":"Han, Y., Liu, X., Zhang, X., and Ding, C. (2024). Foundation Models in Electrocardiogram: A Review. arXiv."},{"key":"ref_54","unstructured":"Zhao, Z., Liu, Y., Wu, H., Wang, M., Li, Y., and Wang, S. (2023). CLIP in Medical Imaging: A Comprehensive Survey. arXiv, Available online: https:\/\/arxiv.org\/abs\/2312.07353."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Lee, R.W., Lee, K.H., Yun, J.S., Kim, M.S., and Choi, H.S. (2024). Comparative Analysis of M4CXR, an LLM-Based Chest X-Ray Report Generation Model, and ChatGPT in Radiological Interpretation. J. Clin. Med., 13.","DOI":"10.3390\/jcm13237057"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Wang, J., Wang, T., Xu, J., and Zhang, Z. (2024, January 3\u20136). Zero-Shot Diagnosis of Unseen Pulmonary Diseases via Spatial Domain Adaptive Correction and Guidance by ChatGPT-4o. Proceedings of the2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbon, Portugal. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/10822114\/.","DOI":"10.1109\/BIBM62325.2024.10822114"},{"key":"ref_57","unstructured":"Gu, D., Gao, Y., Zhou, Y., Zhou, M., and Metaxas, D. (2025). RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment. arXiv, Available online: https:\/\/arxiv.org\/abs\/2501.07525."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Yu, S., Sun, W., Mi, D., Jin, S., Wu, X., Xin, B., and Zhang, H. (2024). Artificial Intelligence Diagnosing of Oral Lichen Planus: A Comparative Study. Bioengineering, 11.","DOI":"10.3390\/bioengineering11111159"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Ghanta, S.N., Al\u2019\u2019Aref, S.J., Lala-Trinidade, A., and Nadkarni, G.N. (2024). Applications of ChatGPT in Heart Failure Prevention, Diagnosis, Management, and Research: A Narrative Review. Diagnostics, 14.","DOI":"10.3390\/diagnostics14212393"},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Huang, H., Li, T., Wang, Z., Seldon, D., and Rodrigo, A. (2025, March 06). TaxonGPT: Taxonomic Classification Using Generative Artificial Intelligence. bioRxiv, Available online: https:\/\/www.biorxiv.org\/content\/10.1101\/2024.10.28.618575.","DOI":"10.1101\/2024.10.28.618575"},{"key":"ref_61","unstructured":"Peng, B., Chen, K., Li, M., Feng, P., Bi, Z., and Liu, J. (2024). Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attacks. arXiv, Available online: https:\/\/arxiv.org\/abs\/2409.08087."},{"key":"ref_62","first-page":"1662","article-title":"Negation scope detection for sentiment analysis: A reinforcement learning framework for replicating human interpretations","volume":"512","author":"Feuerriegel","year":"2020","journal-title":"Inf. Sci."}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/10\/3282\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:29:31Z","timestamp":1760030971000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/10\/3282"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,8]]},"references-count":62,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,5]]}},"alternative-id":["jcm14103282"],"URL":"https:\/\/doi.org\/10.3390\/jcm14103282","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,8]]}}}