{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,19]],"date-time":"2026-10-19T19:58:15Z","timestamp":1792439895231,"version":"4.5.0"},"reference-count":31,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"\n Background<\/jats:title>\n Cystoscopy remains the gold standard for diagnosing bladder lesions; however, its diagnostic accuracy is operator dependent and prone to missing subtle abnormalities such as carcinoma in situ or misinterpreting mimic lesions (tumor, inflammation, or normal variants). Artificial intelligence\u2013based image-analysis systems are emerging, yet conventional models remain limited to single tasks and cannot produce explanatory reports or articulate diagnostic reasoning. Multimodal large language models (MM-LLMs) integrate visual recognition, contextual reasoning, and language generation, offering interpretive capabilities beyond conventional artificial intelligence.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aims to rigorously evaluate state-of-the-art MM-LLMs for cystoscopic image interpretation and lesion classification using clinician-defined stress-test datasets enriched with rare, diverse, and challenging lesions, focusing on diagnostic accuracy, reasoning quality, and clinical relevance.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Four MM-LLMs (OpenAI-o3 and ChatGPT-4o [OpenAI]; Gemini 2.5 Pro and MedGemma-27B [Google]) were evaluated under blinded, randomized procedures across two tasks: (1) free-text image interpretation for anatomic site, findings, lesion reasoning, and final diagnosis (n=401) and (2) seven-class tumor-like lesion classification (n=113) within a multiple-choice framework (cystitis, polyps, papilloma, papillary urothelial carcinoma, carcinoma in situ, non-urothelial carcinoma, and none of the above). Three raters independently scored outputs using a 5-point Likert scale, and classification metrics (accuracy, sensitivity, specificity, Youden J index (Youden J), and Matthews correlation coefficient [MCC]) were calculated for lesion detection, biopsy indication, and malignancy endpoints. For optimization, model performance was compared between zero-shot and text-based in-context learning prompts that were prefixed with brief descriptions of tumor features.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The 401-image test set spanned 40 subcategories, with 322 (80.3%) containing abnormal findings in the image interpretation task. OpenAI-o3 demonstrated strong reasoning, with high satisfaction for anatomy (339\/401, 84.5%) and findings (305\/401, 76%), but lower satisfaction for lesion reasoning (211\/401, 52.5%) and final diagnosis (193\/401, 48.2%), indicating increasing difficulty with higher-order synthesis. Mean Likert score differences (OpenAI-o3 minus Gemini 2.5 Pro) were +0.27 for findings (adjusted P value: q=0.002), +0.24 for lesion reasoning (q=0.047), and +0.19 for final diagnosis. For clinically relevant endpoints in the full set, OpenAI-o3 achieved the most balanced performance, with lesion detection accuracy of 88.3%, sensitivity of 92%, specificity of 73.1%, Youden J of 0.650, and MCC of 0.635. In 7-class tumor-like lesion classification, OpenAI-o3 achieved accuracies of 73.5% for biopsy indication and 62.8% for malignancy, with a balanced sensitivity-specificity trade-off, outperforming other models. Notably, OpenAI-o3 performed best on prevalent malignant lesions. ChatGPT-4o and Gemini 2.5 Pro showed high sensitivity but low specificity, whereas MedGemma-27B underperformed. In-context learning improved OpenAI-o3 microaverage accuracy (40.7%\u219246.0%; MCC 0.311\u21920.370) but yielded only slight specificity gains and minimal accuracy change in other models, likely constrained by the absence of paired image-text context.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n MM-LLMs demonstrate meaningful assistive potential in generating interpretable cystoscopy free-text rationales and supporting biopsy triage and training. However, performance in difficult differential diagnoses remains modest and requires further optimization before safe clinical integration.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/87193","type":"journal-article","created":{"date-parts":[[2026,1,29]],"date-time":"2026-01-29T01:10:08Z","timestamp":1769649008000},"page":"e87193","source":"Crossref","is-referenced-by-count":6,"title":["Multimodal Large Language Models for Cystoscopic Image Interpretation and Bladder Lesion Classification: Comparative Study"],"prefix":"10.2196","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-7099-5838","authenticated-orcid":false,"given":"Yung-Chi","family":"Shih","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-5464-1827","authenticated-orcid":false,"given":"Cheng-Yang","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9777-8561","authenticated-orcid":false,"given":"Shi-Wei","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0527-1830","authenticated-orcid":false,"given":"Chung-You","family":"Tsai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,1,28]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1016\/j.urology.2016.09.018"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.3390\/cancers16010160"},{"key":"ref3","unstructured":"GLOBOCAN 2022: bladder cancer 9th most common worldwideWBCP Coalition2025-11-02https:\/\/tinyurl.com\/y2p57rbx"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1016\/j.euf.2023.08.002"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1245\/s10434-025-17015-3"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1159\/000535652"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.3390\/cancers17010057"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ace499"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1097\/us9.0000000000000057"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1055\/s-0045-1805161"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1136\/gutjnl-2025-335091"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1053\/j.gastro.2020.04.062"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1053\/j.gastro.2022.05.028"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1016\/S2468-1253(19)30411-X"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.2196\/59505"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.3348\/kjr.2025.0599"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1186\/s12894-025-01740-8"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-51465-9"},{"key":"ref19","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-10668-2","author":"Tenny, BC","year":"2022","journal-title":"Diagnostic Cystoscopy"},{"key":"ref20","unstructured":"o3OpenAI Platform20252025-06-15https:\/\/platform.openai.com\/docs\/models\/o3"},{"key":"ref21","unstructured":"Gemini 2.5 ProGoogle Cloud2025-06-15https:\/\/cloud.google.com\/vertex-ai\/generative-ai\/docs\/models\/gemini\/2-5-pro?utm_source=chatgpt.com&hl=zh-tw"},{"key":"ref22","unstructured":"MedGemma 1.5 model cardHealth AI Developer Foundations20252025-07-20https:\/\/developers.google.com\/health-ai-developer-foundations\/medgemma\/model-card"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1186\/s12864-019-6413-7"},{"key":"ref24","author":"Tam, ZR","journal-title":"arXiv"},{"key":"ref25","author":"Ren, J","journal-title":"arXiv"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2102.09690"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-29566-2"},{"key":"ref28","author":"Zhang, Y","journal-title":"arXiv"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-4462333\/v1"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.256"},{"key":"ref31","author":"Xia, P","journal-title":"arXiv"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87193","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:14:41Z","timestamp":1782839681000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2026\/1\/e87193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,28]]},"references-count":31,"URL":"https:\/\/doi.org\/10.2196\/87193","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.87193","asserted-by":"object"}]},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,28]]}}}