{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T13:42:59Z","timestamp":1786542179684,"version":"build-2736575974"},"reference-count":54,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T00:00:00Z","timestamp":1786492800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Tomography"],"abstract":"<jats:p>Background: Accurate fracture interpretation on plain radiographs is critical for both trauma care and medico-legal decision-making, where reproducibility is as important as point accuracy. Although vision language models (VLMs) have shown promising diagnostic performance, their temporal stability in forensic radiography remains unclear. Methods: We analyzed 300 forensic radiographs (150 fracture-positive, 150 fracture-negative) from six long bones, independently evaluated by three emergency medicine physicians, three forensic medicine physicians, and three VLMs (ChatGPT-5.2, Gemini 3 Pro, Claude Sonnet 4.5) using an identical task format. Assessments included fracture presence and structured fracture subtype description (bone, morphology, displacement). VLM evaluations were repeated after one month under identical conditions. Results: Physician accuracy ranged from 79.7% to 98.7%. Emergency physicians reached the higher median sensitivity (92.0% against 76.7%), while specificity among the forensic readers was the more tightly clustered (median 92.0%, range 87.3\u2013100.0%). ChatGPT-5.2 was the most accurate model (83.0%; sensitivity 70.0%, specificity 96.0%), followed by Gemini 3 Pro (77.7%), whereas Claude Sonnet 4.5 reached only 46.3% because of an extreme false-positive tendency (specificity 11.3%). Over one month, accuracy changed by \u22125.2, \u22124.1 and +8.6 percentage points, but these net figures concealed considerable case-level movement: within-model agreement ranged from near chance to substantial (mean Cohen \u03ba 0.131 to 0.622), and the F1-score of Claude Sonnet 4.5 fell by 13.1 points despite its higher accuracy. Subtype descriptions were frequently correct once a fracture had been detected, but end-to-end subtype accuracy remained low. Conclusions: Current vision language models demonstrate encouraging diagnostic performance; however, their temporal reproducibility remains inadequate for independent medico-legal fracture interpretation. These findings highlight that reproducibility, in addition to diagnostic accuracy, should be considered a core benchmark when evaluating VLMs for high-stakes clinical and forensic use. Larger multicenter studies using independent external datasets are needed before forensic application is considered.<\/jats:p>","DOI":"10.3390\/tomography12080113","type":"journal-article","created":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T12:53:48Z","timestamp":1786539228000},"page":"113","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Temporal Reproducibility of Fracture Interpretation in Forensic Radiography: A Multispecialty Comparison of Physicians and Vision Language Models Including Fracture Subtype Description"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7861-4297","authenticated-orcid":false,"given":"Halit Canberk","family":"Aydogan","sequence":"first","affiliation":[{"name":"Department of Forensic Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adem","family":"K\u00f6ksal","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Ayg\u00fcn","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hacer","family":"Ya\u015far Teke","sequence":"additional","affiliation":[{"name":"Department of Forensic Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feyza Nur","family":"\u00c7atalba\u015f","sequence":"additional","affiliation":[{"name":"Department of Forensic Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"\u0130brahim","family":"\u00c7altekin","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Ordu University Training and Research Hospital, Ordu 52200, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e580","DOI":"10.1016\/S2666-7568(21)00172-0","article-title":"Global, regional, and national burden of bone fractures in 204 countries and territories, 1990\u20132019: A systematic analysis from the Global Burden of Disease Study 2019","volume":"2","author":"Wu","year":"2021","journal-title":"Lancet Healthy Longev."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1136\/emj.18.4.263","article-title":"Diagnostic errors in an accident and emergency department","volume":"18","author":"Guly","year":"2001","journal-title":"Emerg. Med. J."},{"key":"ref_3","first-page":"111","article-title":"Traumatic fractures in adults: Missed diagnosis on plain radiographs in the Emergency Department","volume":"89","author":"Pinto","year":"2018","journal-title":"Acta Biomed."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"548","DOI":"10.1148\/radiol.12111119","article-title":"The causes of medical malpractice suits against radiologists in the United States","volume":"266","author":"Whang","year":"2013","journal-title":"Radiology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"317","DOI":"10.2214\/ajr.176.2.1760317","article-title":"Defending the \u201cmissed\u201d radiographic diagnosis","volume":"176","author":"Berlin","year":"2001","journal-title":"AJR Am. J. Roentgenol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1186\/s13244-021-01011-8","article-title":"Investigating errors in medical imaging: Medical malpractice cases in Finland","volume":"12","author":"Tarkiainen","year":"2021","journal-title":"Insights Imaging"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kutbi, M. (2024). Artificial intelligence-based applications for bone fracture detection using medical images: A systematic review. Diagnostics, 14.","DOI":"10.3390\/diagnostics14171879"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"11591","DOI":"10.1073\/pnas.1806905115","article-title":"Deep neural network improves fracture detection by clinicians","volume":"115","author":"Lindsey","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1148\/radiol.211785","article-title":"Artificial intelligence in fracture detection: A systematic review and meta-analysis","volume":"304","author":"Kuo","year":"2022","journal-title":"Radiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1007\/s00256-025-05078-y","article-title":"Diagnostic performance of X-ray-based deep learning models for detecting ankle and foot fractures: A systematic review and meta-analysis","volume":"55","author":"Hosseinzadeh","year":"2026","journal-title":"Skelet. Radiol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Devnath, H., Devnath, L., and Hossain, M.M. (2026). Explainable artificial intelligence for automated flatfoot detection in foot X-ray images. Discov. Artif. Intell.","DOI":"10.1007\/s44163-026-01342-9"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1148\/radiol.210937","article-title":"Improving radiographic fracture recognition performance and efficiency using artificial intelligence","volume":"302","author":"Guermazi","year":"2022","journal-title":"Radiology"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"869","DOI":"10.2214\/AJR.22.27873","article-title":"Artificial intelligence (AI) for fracture diagnosis: An overview of current products and considerations for clinical adoption, from the AJR special series on AI applications","volume":"219","author":"Zech","year":"2022","journal-title":"AJR Am. J. Roentgenol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e78041","DOI":"10.2196\/78041","article-title":"Trends and trajectories in the rise of large language models in radiology: Scoping review","volume":"13","author":"Alshibli","year":"2025","journal-title":"JMIR Med. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4374","DOI":"10.1007\/s00330-024-11339-6","article-title":"CXR-LLAVA: A multimodal large language model for interpreting chest X-ray images","volume":"35","author":"Lee","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e240955","DOI":"10.1148\/radiol.240955","article-title":"Toward foundation models in radiology? Quantitative assessment of GPT-4V\u2019s multimodal and multianatomic region capabilities","volume":"313","author":"Strotzer","year":"2024","journal-title":"Radiology"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Eauchai, L., Gonz\u00e1lez, L.O., Shi, Y., McGinnis, M.T., Yovchev, A., Herasevich, S., Pickering, B.W., and Herasevich, V. (2026). Do multimodal vision-language models enhance the medical diagnostic process? A systematic review. Healthcare, 14.","DOI":"10.3390\/healthcare14131877"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Han, H. (2025). Challenges of reproducible AI in biomedical data science. BMC Med. Genom., 18.","DOI":"10.1186\/s12920-024-02072-6"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"100409","DOI":"10.1016\/j.clinsp.2024.100409","article-title":"Bibliographic research with large language model ChatGPT-4: Instability, hallucinations and sometimes alerts","volume":"79","author":"Metze","year":"2024","journal-title":"Clinics"},{"key":"ref_21","first-page":"1587","article-title":"The current state of forensic imaging: Clinical forensic imaging","volume":"139","author":"Dedouit","year":"2025","journal-title":"Int. J. Leg. Med."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Orsini, F., Cioffi, A., Cipolloni, L., Bibb\u00f2, R., Montana, A., De Simone, S., and Cecannecchia, C. (2025). The application of artificial intelligence in forensic pathology: A systematic literature review. Front. Med., 12.","DOI":"10.3389\/fmed.2025.1583743"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Contaldo, M.T., Pasceri, G., Vignati, G., Bracchi, L., Triggiani, S., and Carrafiello, G. (2024). AI in radiology: Navigating medical responsibility. Diagnostics, 14.","DOI":"10.3390\/diagnostics14141506"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e078378","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD + AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"Collins","year":"2024","journal-title":"BMJ"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e250913","DOI":"10.1148\/radiol.250913","article-title":"Guidelines for reporting studies on large language models in radiology: An international Delphi expert survey","volume":"318","author":"Kottlors","year":"2026","journal-title":"Radiology"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Jung, J., Dai, J., Liu, B., and Wu, Q. (2024). Artificial intelligence in fracture detection with different image modalities and data types: A systematic review and meta-analysis. PLoS Digit. Health, 3.","DOI":"10.1371\/journal.pdig.0000438"},{"key":"ref_27","first-page":"1","article-title":"Advancements and initial experiences in AI-assisted X-ray based fracture diagnosis: A narrative review","volume":"17","author":"Gurabi","year":"2025","journal-title":"Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1016\/j.crad.2024.04.009","article-title":"Artificial intelligence diagnostic accuracy in fracture detection from plain radiographs and comparing it with clinicians: A systematic review and meta-analysis","volume":"79","author":"Nowroozi","year":"2024","journal-title":"Clin. Radiol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1148\/radiol.2021203886","article-title":"Assessment of an AI aid in detection of adult appendicular skeletal fractures by emergency physicians and radiologists: A multicenter cross-sectional diagnostic study","volume":"300","author":"Duron","year":"2021","journal-title":"Radiology"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1007\/s13244-018-0645-y","article-title":"Artificial intelligence as a medical device in radiology: Ethical and regulatory issues in Europe and the United States","volume":"9","author":"Pesapane","year":"2018","journal-title":"Insights Imaging"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1148\/radiol.2019191586","article-title":"Ethics of artificial intelligence in radiology: Summary of the joint European and North American multisociety statement","volume":"293","author":"Geis","year":"2019","journal-title":"Radiology"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Naik, N., Hameed, B.M.Z., Shetty, D.K., Swain, D., Shah, M., Paul, R., Aggarwal, K., Ibrahim, S., Patil, V., and Smriti, K. (2022). Legal and ethical consideration in artificial intelligence in healthcare: Who takes responsibility?. Front. Surg., 9.","DOI":"10.3389\/fsurg.2022.862322"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.acra.2019.09.017","article-title":"How the FDA regulates AI","volume":"27","author":"Harvey","year":"2020","journal-title":"Acad. Radiol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e59607","DOI":"10.2196\/59607","article-title":"Examining the role of large language models in orthopedics: Systematic review","volume":"26","author":"Zhang","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3576","DOI":"10.1007\/s00330-020-06672-5","article-title":"Integrating artificial intelligence into the clinical practice of radiology: Challenges and recommendations","volume":"30","author":"Recht","year":"2020","journal-title":"Eur. Radiol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e230725","DOI":"10.1148\/radiol.230725","article-title":"Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting: A multilingual feasibility study","volume":"307","author":"Adams","year":"2023","journal-title":"Radiology"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1007\/s00330-024-11115-6","article-title":"Revolution or risk? Assessing the potential and challenges of GPT-4V in radiologic image interpretation","volume":"35","author":"Huppertz","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"108013","DOI":"10.1016\/j.cmpb.2024.108013","article-title":"ChatGPT in healthcare: A taxonomy and systematic review","volume":"245","author":"Li","year":"2024","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1186\/s41935-025-00445-1","article-title":"Assessing ChatGPT-4\u2019s ability to generate forensic reports: A study of artificial intelligence in forensics","volume":"15","author":"Aydogan","year":"2025","journal-title":"Egypt. J. Forensic Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"14604582251414578","DOI":"10.1177\/14604582251414578","article-title":"Inferential performance and temporal stability of large language models in suicide method prediction: A forensic psychiatric analysis","volume":"32","author":"Aydogan","year":"2026","journal-title":"Health Inform. J."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1007\/s11604-025-01884-5","article-title":"Accuracy and reproducibility of large language model measurements of liver metastases: Comparison with radiologist measurements","volume":"44","author":"Sugawara","year":"2026","journal-title":"Jpn. J. Radiol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"e240790","DOI":"10.1148\/ryai.240790","article-title":"Retrieval-augmented generation with large language models in radiology: From theory to practice","volume":"7","author":"Fink","year":"2025","journal-title":"Radiol. Artif. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e231259","DOI":"10.1148\/radiol.231259","article-title":"Evaluating GPT-4 on impressions generation in radiology reports","volume":"307","author":"Sun","year":"2023","journal-title":"Radiology"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"e5143","DOI":"10.1002\/nbm.5143","article-title":"A review of self-supervised, generative, and few-shot deep learning methods for data-limited magnetic resonance imaging segmentation","volume":"37","author":"Liu","year":"2024","journal-title":"NMR Biomed."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"6006","DOI":"10.1038\/s41598-021-85570-2","article-title":"Critical evaluation of deep neural networks for wrist fracture detection","volume":"11","author":"Raisuddin","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1007\/s10278-018-0167-7","article-title":"Ankle fracture detection utilizing a convolutional neural network ensemble implemented with a small sample, de novo training, and multiview incorporation","volume":"32","author":"Kitamura","year":"2019","journal-title":"J. Digit. Imaging"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"e216096","DOI":"10.1001\/jamanetworkopen.2021.6096","article-title":"Development and validation of a deep learning model using convolutional neural networks to identify scaphoid fractures in radiographs","volume":"4","author":"Yoon","year":"2021","journal-title":"JAMA Netw. Open"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1186\/s13018-021-02845-0","article-title":"Artificial intelligence to diagnosis distal radius fracture using biplane plain X-rays","volume":"16","author":"Oka","year":"2021","journal-title":"J. Orthop. Surg. Res."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Hardala\u00e7, F., Uysal, F., Peker, O., \u00c7i\u00e7eklida\u011f, M., Tolunay, T., Tokg\u00f6z, N., Kutbay, U., Demirciler, B., and Mert, F. (2022). Fracture detection in wrist X-ray images using deep learning-based object detection models. Sensors, 22.","DOI":"10.3390\/s22031285"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1066","DOI":"10.1038\/s41467-021-21311-3","article-title":"A scalable physician-level deep learning algorithm detects universal trauma on pelvic radiographs","volume":"12","author":"Cheng","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Aldhafeeri, F.M. (2025). Governing artificial intelligence in radiology: A systematic review of ethical, legal, and regulatory frameworks. Diagnostics, 15.","DOI":"10.3390\/diagnostics15182300"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2547","DOI":"10.1177\/21925682251314379","article-title":"Artificial intelligence for cervical spine fracture detection: A systematic review of diagnostic performance and clinical potential","volume":"15","author":"Liawrungrueang","year":"2025","journal-title":"Glob. Spine J."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"100452","DOI":"10.1016\/j.imu.2020.100452","article-title":"Bone fracture detection through the two-stage system of crack-sensitive convolutional neural network","volume":"22","author":"Ma","year":"2021","journal-title":"Inform. Med. Unlocked"},{"key":"ref_54","first-page":"e97674","article-title":"Artificial intelligence in bone fracture detection: A review of evidence, limitations, and clinical integration","volume":"17","author":"Elkohail","year":"2025","journal-title":"Cureus"}],"container-title":["Tomography"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2379-139X\/12\/8\/113\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,12]],"date-time":"2026-08-12T13:24:56Z","timestamp":1786541096000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2379-139X\/12\/8\/113"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,12]]},"references-count":54,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["tomography12080113"],"URL":"https:\/\/doi.org\/10.3390\/tomography12080113","relation":{},"ISSN":["2379-139X"],"issn-type":[{"value":"2379-139X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,12]]}}}