{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T15:31:01Z","timestamp":1786375861446,"version":"build-2736575974"},"reference-count":39,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Multimodal large language models (MLLMs) have emerging potential for interpreting medical images and text, but their performance in orthopedic imaging tasks and the influence of prompt configuration remain insufficiently studied.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the performance of commercial and open-source MLLMs for diagnosing and staging osteonecrosis of the femoral head (ONFH) and to assess how different prompt configurations affect model performance.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n This single-center retrospective diagnostic accuracy study included 159 radiograph patients contributing 318 hip-level observations and 170 magnetic resonance imaging (MRI) patients contributing 340 hip-level observations; 55 patients with both modalities formed the multi-image (MI) subgroup between July 2023 and December 2024. Four MLLMs were evaluated: GPT-4o, Claude 3.7 Sonnet, Qwen2.5-VL 72B, and Gemma 3 27B. Three prompt configurations were tested: single image (SI), image plus radiology description (ID), and MI. Model performance was assessed for ONFH detection; early- versus late-stage differentiation; detailed grading using the Ficat, Association Research Circulation Osseous (ARCO), and Steinberg systems; and grading reliability using intraclass correlation coefficients (ICCs).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Model performance varied by prompt configuration and imaging input. For ONFH detection, the SI configuration yielded a mean detection area under the receiver operating characteristic curve (AUC) of 0.55 (SD 0.03), whereas the ID configuration achieved a mean detection AUC of 0.91 (SD 0.01). In radiograph-based ONFH detection, ID input achieved a mean accuracy of 0.88 (SD 0.01); in MRI-based ONFH detection, ID input achieved a mean accuracy of 0.85 (SD 0.01). For early- versus late-stage differentiation, the mean accuracy was 0.65 (SD 0.11) with SI input, 0.78 (SD 0.04) with ID input, and approximately 0.59 (SD 0.10) with MI input. For detailed grading, ID input improved mean accuracy across the Ficat, ARCO, and Steinberg systems compared with SI input. In ARCO grading reliability analysis, the mean MLLM ICC was 0.51 (SD 0.18) for SI, 0.97 (SD 0.02) for ID, and 0.49 (SD 0.15) for MI; surgeon interrater and intrarater ICCs were 0.70 and 0.81, respectively. In the commercial versus open-source model comparison, no significant overall difference was observed between model groups (\n P<\/jats:italic>\n =.83).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Prompt configuration strongly influenced MLLM performance in ONFH diagnosis and staging. Pairing images with deidentified radiology descriptions improved diagnostic and grading performance, whereas MI input did not provide consistent additional benefit in this retrospective single-center evaluation. These findings support the potential role of MLLMs as assistive tools in human-AI orthopedic imaging workflows, but external validation, careful input standardization, and prospective clinical evaluation are needed before clinical deployment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/92919","type":"journal-article","created":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T15:12:11Z","timestamp":1786374731000},"page":"e92919-e92919","source":"Crossref","is-referenced-by-count":0,"title":["Prompt Configurations for Multimodal Large Language Models in Diagnosing and Staging Osteonecrosis of the Femoral Head: Multimodel Retrospective Observational Diagnostic Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-9854-7134","authenticated-orcid":false,"given":"Jiesheng","family":"Zhu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7043-9794","authenticated-orcid":false,"given":"Xingxing","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0371-9109","authenticated-orcid":false,"given":"Jincheng","family":"Shi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9441-0873","authenticated-orcid":false,"given":"Shaoming","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0223-6063","authenticated-orcid":false,"given":"Daosen","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3787-1515","authenticated-orcid":false,"given":"Zhihan","family":"Gao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7832-3019","authenticated-orcid":false,"given":"Yi","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-1688-3161","authenticated-orcid":false,"given":"Tao","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6178-6316","authenticated-orcid":false,"given":"Xue","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2402-1848","authenticated-orcid":false,"given":"Yimu","family":"Lin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0153-8846","authenticated-orcid":false,"given":"Peiyu","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2155-4623","authenticated-orcid":false,"given":"Li","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6592-3071","authenticated-orcid":false,"given":"Pei","family":"Fan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,10]]},"reference":[{"issue":"3","key":"R1","doi-asserted-by":"crossref","first-page":"473","DOI":"10.2106\/00004623-199603000-00022","volume":"78","author":"Maillefert","journal-title":"J Bone Joint Surg Am"},{"issue":"3","key":"R2","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1302\/2058-5241.4.180036","article-title":"Osteonecrosis of the femoral head: pathophysiology and current concepts of treatment","volume":"4","author":"Petek","journal-title":"EFORT Open Rev"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","first-page":"590","DOI":"10.5312\/wjo.v6.i8.590","article-title":"Current concepts on osteonecrosis of the femoral head","volume":"6","author":"Moya-Angeler","journal-title":"World J Orthop"},{"issue":"12","key":"R4","doi-asserted-by":"publisher","first-page":"2589","DOI":"10.2106\/00004623-200412000-00001","article-title":"Fate of very small asymptomatic stage-I osteonecrotic lesions of the hip","volume":"86","author":"Hernigou","journal-title":"J Bone Joint Surg Am"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"6","DOI":"10.4103\/0019-5413.45318","article-title":"Management of avascular necrosis of femoral head at pre-collapse stage","volume":"43","author":"Sen","journal-title":"Indian J Orthop"},{"issue":"1","key":"R6","first-page":"23","volume":"35","author":"Stoica","journal-title":"Curr Health Sci J"},{"key":"R7","doi-asserted-by":"publisher","DOI":"10.2106\/JBJS.F.00457","article-title":"Systematic analysis of classification systems for osteonecrosis of the femoral head","volume":"88 Suppl 3","author":"Mont","journal-title":"J Bone Joint Surg Am"},{"issue":"307","key":"R8","first-page":"124","author":"Kay","journal-title":"Clin Orthop Relat Res"},{"issue":"11","key":"R9","doi-asserted-by":"publisher","first-page":"1702","DOI":"10.2106\/00004623-199611000-00010","article-title":"Interobserver reliability and intraobserver reproducibility of the modified Ficat classification system of osteonecrosis of the femoral head","volume":"78","author":"Smith","journal-title":"J Bone Joint Surg Am"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01208-3","article-title":"Evaluating multimodal AI in medical diagnostics","volume":"7","author":"Kaczmarczyk","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240689","article-title":"Impact of multimodal prompt elements on diagnostic performance of GPT-4V in challenging brain MRI cases","volume":"314","author":"Schramm","journal-title":"Radiology"},{"issue":"5","key":"R12","doi-asserted-by":"publisher","first-page":"1911","DOI":"10.1002\/ksa.12639","article-title":"High identification and positive-negative discrimination but limited detailed grading accuracy of ChatGPT-4o in knee osteoarthritis radiographs","volume":"33","author":"Zhu","journal-title":"Knee Surg Sports Traumatol Arthrosc"},{"issue":"19","key":"R13","doi-asserted-by":"publisher","first-page":"1519","DOI":"10.2106\/JBJS.23.00209","article-title":"Assessing ChatGPT responses to common patient questions regarding total hip arthroplasty","volume":"105","author":"Mika","journal-title":"J Bone Joint Surg Am"},{"issue":"8","key":"R14","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial intelligence in radiology","volume":"18","author":"Hosny","journal-title":"Nat Rev Cancer"},{"issue":"10","key":"R15","doi-asserted-by":"publisher","first-page":"2044","DOI":"10.1016\/j.arth.2022.10.003","article-title":"Deep learning approach for diagnosing early osteonecrosis of the femoral head based on magnetic resonance imaging","volume":"38","author":"Shen","journal-title":"J Arthroplasty"},{"issue":"16","key":"R16","doi-asserted-by":"publisher","first-page":"1798","DOI":"10.2106\/JBJS.24.01601","article-title":"A deep learning-based clinical classification system for the differential diagnosis of hip prosthesis failures using radiographs: a multicenter study","volume":"107","author":"Wu","journal-title":"J Bone Joint Surg Am"},{"issue":"1","key":"R17","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.241139","article-title":"Comparing commercial and open-source large language models for labeling chest radiograph reports","volume":"313","author":"Dorfner","journal-title":"Radiology"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232714","article-title":"Potential of GPT-4 for detecting errors in radiology reports: implications for reporting accuracy","volume":"311","author":"Gertz","journal-title":"Radiology"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-46411-8","article-title":"Systematic analysis of ChatGPT, Google search and Llama 2 for clinical decision support tasks","volume":"15","author":"Sandmann","journal-title":"Nat Commun"},{"issue":"15","key":"R20","doi-asserted-by":"publisher","first-page":"1411","DOI":"10.2106\/JBJS.23.01417","article-title":"Large language models in orthopaedics: definitions, uses, and limitations","volume":"106","author":"Yao","journal-title":"J Bone Joint Surg Am"},{"issue":"16","key":"R21","doi-asserted-by":"publisher","first-page":"1833","DOI":"10.2106\/JBJS.24.01429","article-title":"An institutional large language model for musculoskeletal MRI improves protocol adherence and accuracy","volume":"107","author":"Hallinan","journal-title":"J Bone Joint Surg Am"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1007\/s00264-007-0320-2","article-title":"Avascular necrosis of the femoral head: inter- and intraobserver variations of Ficat and ARCO classifications","volume":"32","author":"Schmitt-Sody","journal-title":"Int Orthop"},{"issue":"4","key":"R23","doi-asserted-by":"publisher","first-page":"933","DOI":"10.1016\/j.arth.2019.11.029","article-title":"The 2019 revised version of Association Research Circulation Osseous staging system of osteonecrosis of the femoral head","volume":"35","author":"Yoon","journal-title":"J Arthroplasty"},{"issue":"1","key":"R24","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1302\/0301-620X.77B1.7822393","volume":"77","author":"Steinberg","journal-title":"J Bone Joint Surg Br"},{"key":"R25","unstructured":"Alayrac JB Donahue J Luc P et al. Flamingo: a visual language model for few-shot learning. Presented at: NIPS\u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems; Nov 28 to Dec 9, 2022; New Orleans, LA, USA. URL: http:\/\/www.proceedings.com\/68431.html [Accessed 02-07-2026]"},{"key":"R26","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.256","article-title":"MedCLIP: contrastive learning from unpaired medical images and text","volume":"2022","author":"Wang","journal-title":"Proc Conf Empir Methods Nat Lang Process"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.107522","article-title":"Visual prior-based cross-modal alignment network for radiology report generation","volume":"166","author":"Zhang","journal-title":"Comput Biol Med"},{"issue":"2","key":"R28","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","journal-title":"J Chiropr Med"},{"issue":"2","key":"R29","doi-asserted-by":"publisher","first-page":"1179","DOI":"10.1007\/s00330-023-10104-5","article-title":"Deep learning enables the differentiation between early and late stages of hip avascular necrosis","volume":"34","author":"Klontzas","journal-title":"Eur Radiol"},{"issue":"2","key":"R30","doi-asserted-by":"publisher","first-page":"e224","DOI":"10.4317\/medoral.26824","article-title":"Accuracy of ChatGPT 3.5, 4.0, 4o and Gemini in diagnosing oral potentially malignant lesions based on clinical case reports and image recognition","volume":"30","author":"Pradhan","journal-title":"Med Oral Patol Oral Cir Bucal"},{"issue":"3","key":"R31","doi-asserted-by":"publisher","first-page":"1041","DOI":"10.1007\/s12020-024-04066-x","article-title":"Assessing the feasibility of ChatGPT-4o and Claude 3-Opus in thyroid nodule classification based on ultrasound images","volume":"87","author":"Chen","journal-title":"Endocrine"},{"issue":"4","key":"R32","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1111\/1754-9485.13858","article-title":"Comparative performance of Anthropic Claude and OpenAI GPT models in basic radiological imaging tasks","volume":"69","author":"Nguyen","journal-title":"J Med Imaging Radiat Oncol"},{"issue":"2","key":"R33","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.241191","article-title":"Llama 3 challenges proprietary state-of-the-art large language models in radiology board-style examination questions","volume":"312","author":"Adams","journal-title":"Radiology"},{"issue":"1","key":"R34","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240895","article-title":"Privacy-ensuring open-weights large language models are competitive with closed-weights GPT-4o in extracting chest radiography findings from free-text reports","volume":"314","author":"Nowak","journal-title":"Radiology"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01476-7","article-title":"Implementing large language models in healthcare while balancing control, collaboration, costs and security","volume":"8","author":"Dennst\u00e4dt","journal-title":"NPJ Digit Med"},{"issue":"1","key":"R36","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01010-1","article-title":"Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine","volume":"7","author":"Savage","journal-title":"NPJ Digit Med"},{"key":"R37","doi-asserted-by":"crossref","unstructured":"Liu H Zhang X Xu H et al. MIBench: evaluating multimodal large language models over multiple images. Presented at: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, Florida, USA. [doi: 10.18653\/v1\/2024.emnlp-main.1250]","DOI":"10.18653\/v1\/2024.emnlp-main.1250"},{"key":"R38","doi-asserted-by":"crossref","unstructured":"Li J Wu M Jin Z et al. MIHBench: benchmarking and mitigating multi-image hallucinations in multimodal large language models. Presented at: MM \u201925: Proceedings of the 33rd ACM International Conference on Multimedia; Oct 27-31, 2025:3143-3152; Dublin, Ireland. [doi: 10.1145\/3746027.3754993]","DOI":"10.1145\/3746027.3754993"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01180-y","article-title":"The long but necessary road to responsible use of large language models in healthcare research","volume":"7","author":"Kwong","journal-title":"NPJ Digit Med"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T15:12:13Z","timestamp":1786374733000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e92919"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,10]]},"references-count":39,"URL":"https:\/\/doi.org\/10.2196\/92919","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,10]]},"article-number":"v28i3e92919"}}