{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T12:08:49Z","timestamp":1778501329371,"version":"3.51.4"},"reference-count":15,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T00:00:00Z","timestamp":1778457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Large language models (LLMs) are increasingly applied to medical image interpretation; however, their diagnostic accuracy and reliability in musculoskeletal radiology remain uncertain. This study evaluates the diagnostic performance and confidence calibration of LLMs in detecting and classifying bone tumors on radiographs. Methods: This retrospective observational study analyzed a dataset of 257 radiographs with confirmed diagnoses obtained from Radiopaedia, including normal studies and a spectrum of benign and malignant bone tumors. Cases were selected to ensure representation across multiple tumor types. Three LLMs (ChatGPT 5.3, X-ray Interpreter GPT-4.1, and X-ray Interpreter Gemini) evaluated each image using a standardized prompt assessing abnormality detection, tumor detection, classification, and confidence. Outcomes included diagnostic accuracy, false positive abnormality rates, false negative rates, tumor hallucination rates, and confidence calibration. Results: Abnormality detection was high across models, with Gemini demonstrating the highest sensitivity (up to 100%). Tumor detection was strongest in lesions with characteristic features, including osteosarcoma and osteochondroma. False negative rates varied substantially, with GPT-4.1 demonstrating the highest rate (29.9%), followed by ChatGPT (24.8%) and Gemini (6.6%). Primary diagnostic accuracy was highest for osteosarcoma in GPT-4.1 (80%), while ChatGPT 5.3 performed best in benign lesions, including osteochondroma (84.6%) and non-ossifying fibroma (76.9%). Tumor subtype classification remained limited across all models and was poorest for Ewing sarcoma (0% in ChatGPT and GPT-4.1; 10.3% in Gemini). False positive abnormality rates were highest in GPT-4.1 (40.7%), followed by Gemini (25.9%) and ChatGPT (13.5%). Tumor hallucination occurred only in Gemini (12.3%). All models demonstrated confidence miscalibration, with higher confidence observed in incorrect predictions and in tumor-negative cases. Conclusions: LLMs demonstrate strong performance in detecting radiographic abnormalities but remain limited in tumor subtype classification, particularly for diagnostically challenging lesions such as Ewing sarcoma. Elevated false positive and false negative rates, along with systematic overconfidence\u2014especially in GPT-4.1\u2014highlight important limitations for clinical use. These findings support the role of LLMs as adjunctive tools rather than independent diagnostic systems.<\/jats:p>","DOI":"10.3390\/diagnostics16101460","type":"journal-article","created":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T10:50:52Z","timestamp":1778496652000},"page":"1460","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Diagnostic Performance and Confidence Calibration of Large Language Models for Bone Tumor Radiographs"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-2551-2190","authenticated-orcid":false,"given":"Sanjana","family":"Arun","sequence":"first","affiliation":[{"name":"College of Medicine Phoenix, University of Arizona, Phoenix, AZ 85004, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eujung","family":"Park","sequence":"additional","affiliation":[{"name":"College of Medicine Phoenix, University of Arizona, Phoenix, AZ 85004, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katja","family":"Klosterman","sequence":"additional","affiliation":[{"name":"College of Medicine Phoenix, University of Arizona, Phoenix, AZ 85004, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2684-9930","authenticated-orcid":false,"given":"Carissa","family":"Zhu","sequence":"additional","affiliation":[{"name":"College of Medicine Phoenix, University of Arizona, Phoenix, AZ 85004, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ronak","family":"Arun","sequence":"additional","affiliation":[{"name":"Language School and Arts Department, University of Michigan, Ann Arbor, MI 48109, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Palmer Wrigley","family":"Stratton","sequence":"additional","affiliation":[{"name":"Biological Sciences, Arizona State University, Tempe, AZ 85287, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hamsa","family":"Gangaswamiah","sequence":"additional","affiliation":[{"name":"Robert Wood Johnson Barnabas Health, New Brunswick, NJ 08901, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,5,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1007\/s00256-024-04854-6","article-title":"Conventional radiography for the assessment of focal bone lesions of the appendicular skeleton: Fundamental concepts in the modern imaging era","volume":"54","author":"Matcuk","year":"2025","journal-title":"Skeletal. Radiol."},{"key":"ref_2","first-page":"1768","article-title":"Accuracy of plain radiographs in diagnosing biopsy-proven bone lesions","volume":"23","author":"Gerber","year":"2019","journal-title":"SA Orthop. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e2024S131","DOI":"10.1590\/1806-9282.2024s131","article-title":"Biopsy of bone tumors: A literature review","volume":"70","author":"Guedes","year":"2024","journal-title":"Rev. Assoc. Med. Bras."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Crenn, V., Vezole, L., Bouhamama, A., Meurgey, A., Karanian, M., Marec-B\u00e9rard, P., Gouin, F., and Vaz, G. (2021). Percutaneous Core Needle Biopsy Can Efficiently and Safely Diagnose Most Primary Bone Tumors. Diagnostics, 11.","DOI":"10.3390\/diagnostics11091552"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1007\/s00330-024-10902-5","article-title":"ChatGPT\u2019s diagnostic performance based on textual vs. visual information compared to radiologists\u2019 diagnostic performance in musculoskeletal radiology","volume":"35","author":"Horiuchi","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"100525","DOI":"10.1016\/j.jbo.2024.100525","article-title":"Large-Scale assessment of ChatGPT\u2019s performance in benign and malignant bone tumors imaging report diagnosis and its potential for clinical applications","volume":"44","author":"Yang","year":"2024","journal-title":"J. Bone Oncol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e22409","DOI":"10.1016\/j.heliyon.2023.e22409","article-title":"ChatGPT-assisted deep learning for diagnosing bone metastasis in bone scans: Bridging the AI Gap for Clinicians","volume":"9","author":"Son","year":"2023","journal-title":"Heliyon"},{"key":"ref_8","unstructured":"(2026, March 30). Radiopaedia.org. Available online: https:\/\/radiopaedia.org."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1038\/nature21056","article-title":"Dermatologist-level classification of skin cancer with deep neural networks","volume":"542","author":"Esteva","year":"2017","journal-title":"Nature"},{"key":"ref_10","unstructured":"Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., Langlotz, C., and Shpanskaya, K. (2017). CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/j.ro.2004.06.013","article-title":"Imaging, diagnosis, and staging of bone tumors: A primer","volume":"39","author":"Pommersheim","year":"2004","journal-title":"Semin. Roentgenol."},{"key":"ref_12","unstructured":"Guo, C., Pleiss, G., Sun, Y., and Weinberger, K.Q. (2017, January 6\u201311). On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning (ICML 2017), Sydney, Australia."},{"key":"ref_13","unstructured":"Minderer, M., Djolonga, J., Romijnders, R., Hubis, F., Zhai, X., Houlsby, N., Tran, D., and Lucic, M. (2021, January 6\u201314). Revisiting the calibration of modern neural networks. Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS 2021), Online."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref_15","unstructured":"Nori, H., King, N., McKinney, S.M., Carignan, D., and Horvitz, E. (2023). Capabilities of GPT-4 on medical challenge problems. arXiv."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/10\/1460\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:17:46Z","timestamp":1778498266000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/10\/1460"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,11]]},"references-count":15,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2026,5]]}},"alternative-id":["diagnostics16101460"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16101460","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,11]]}}}