{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T07:53:11Z","timestamp":1791186791458,"version":"4.1.0"},"reference-count":27,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T00:00:00Z","timestamp":1791158400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T00:00:00Z","timestamp":1791158400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Abdom Radiol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Rationale and objectives<\/jats:title>\n                    <jats:p>To evaluate the performance of four large language models (LLMs) for automated feature extraction and clear cell likelihood score version 2.0 (ccLS v2.0) assignment from free-text reports using a multi-step prompting strategy.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>This retrospective multicenter study included 1,048 magnetic resonance imaging (MRI) reports of renal masses from three institutions (2020\u20132026). Four LLMs (DeepSeek-V3.2, Qwen 3.5, GPT-5.3, and Gemini 3.0) were evaluated for automated ccLS v2.0 assignment through a three-stage prompting workflow: (1) feature extraction, (2) rule-based criteria matching, and (3) final categorization. Performance was evaluated against expert radiologist consensus-based ccLS v2.0 assignments. Accordingly, these results primarily reflect agreement in task execution rather than diagnostic accuracy for ccRCC.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Gemini 3.0 achieved the highest overall accuracy, correctly assigning ccLS categories in 683 of 802 reports in the internal cohort (85.2%; 95% CI, 82.5%\u201387.5%) and 211 of 246 reports in the external validation cohort (85.8%; 95% CI, 80.9%\u201389.6%). All LLMs achieved accuracy greater than 80% for assigning high-risk categories (ccLS 4\u20135). Gemini 3.0 showed comparatively strong performance in extracting individual imaging features, such as T2-weighted imaging (T2WI) hyperintensity. However, all models showed reduced performance for features requiring multi-step interpretation, particularly the arterial-to-delayed enhancement ratio (ADR), with Qwen 3.5 showing the weakest performance for these complex features. In the pathology subgroup of the internal cohort (\n                      <jats:italic>n<\/jats:italic>\n                      \u2009=\u2009237), expert report-based scores yielded an AUC of 0.799 for identifying ccRCC, with LLM-assigned scores showing AUCs of 0.768\u20130.791 and small AUC differences from the expert standard (\u22120.031 to \u22120.008) with overlapping 95% CIs.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs employing multi-step prompting, particularly Gemini 3.0, demonstrated favorable performance in assigning ccLS v2.0 categories from free-text reports. Despite limitations in complex feature reasoning, these models showed potential for supporting automated ccLS assessment in radiology workflows; however, in their current implementation, they should be considered decision-support tools that require expert verification rather than autonomous, unsupervised scoring.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00261-026-05818-1","type":"journal-article","created":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T07:10:04Z","timestamp":1791184204000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Application of large language models for assigning clear cell likelihood score v2.0 from free\u2013text MRI reports: a feasibility study"],"prefix":"10.1007","author":[{"given":"Yanting","family":"Duan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yilei","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maowei","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tiefeng","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liuqiong","family":"Yang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5192-5340","authenticated-orcid":false,"given":"Zhengyu","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Minhong","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhan","family":"Feng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,10,5]]},"reference":[{"issue":"5","key":"5818_CR1","first-page":"178","volume":"24","author":"S Ferretti","year":"2025","unstructured":"Ferretti S, Pearanda NR, Puliatti S (2025) Advances in Diagnosis, Prognosis, and Treatment of Renal Cell Carcinoma [J]. Technology in Cancer Research & Treatment 24(5):178\u201382","journal-title":"Technology in Cancer Research & Treatment"},{"key":"5818_CR2","doi-asserted-by":"crossref","unstructured":"R Houston T, Jennifer R H, Yuriy B, et al. Metastatic renal cell carcinoma risk according to tumor size [J]. J Urol, 2009, 182(1).","DOI":"10.1016\/j.juro.2009.02.128"},{"key":"5818_CR3","doi-asserted-by":"crossref","unstructured":"Steven C C, Peter E C, Sam S C, et al. Renal Mass and Localized Renal Cancer: Evaluation, Management, and Follow-Up: AUA Guideline: Part I [J]. J Urol, 2021, 206(2).","DOI":"10.1097\/JU.0000000000001911"},{"key":"5818_CR4","doi-asserted-by":"crossref","unstructured":"Brett A J, Sandy K, Ryan L S, et al. Diagnostic performance of prospectively assigned clear cell Likelihood scores (ccLS) in small renal masses at multiparametric magnetic resonance imaging [J]. Urol Oncol, 2019, 37(12).","DOI":"10.1016\/j.urolonc.2019.07.023"},{"key":"5818_CR5","doi-asserted-by":"crossref","unstructured":"Antonio F, Douglas C C, Ashraf A-M, et al. Small Renal Mass Surveillance: Histology-specific Growth Rates in a Biopsy-characterized Cohort [J]. Eur Urol, 2020, 78(3).","DOI":"10.1016\/j.eururo.2020.06.053"},{"key":"5818_CR6","doi-asserted-by":"crossref","unstructured":"Ryan L S, Robert G R, Brett A J, et al. Prospective performance of clear cell likelihood scores (ccLS) in renal masses evaluated with multiparametric magnetic resonance imaging [J]. Eur Radiol, 2020, 31(1).","DOI":"10.1007\/s00330-020-07093-0"},{"key":"5818_CR7","doi-asserted-by":"crossref","unstructured":"Nicola S, Matthew S D, Stuart G S, et al. Multicenter Evaluation of Multiparametric MRI Clear Cell Likelihood Scores in Solid Indeterminate Small Renal Masses [J]. Radiology, 2022, 303(3).","DOI":"10.1148\/radiol.211680"},{"key":"5818_CR8","doi-asserted-by":"crossref","unstructured":"Run W, Fan M, Xiaoxia L, et al. Diagnostic Performance of Multiparametric Clear Cell Likelihood Score for Characterization of Subcentimeter Solid Renal Lesions: A Multi-institutional Study [J]. Acad Radiol, 2025, 32(7).","DOI":"10.1016\/j.acra.2025.01.007"},{"key":"5818_CR9","unstructured":"Aisin I, Vincent P, Maurice A, et al. Performance of clear cell likelihood scores in characterizing solid renal masses at multiparametric MRI: an external validation study [J]. Abdom Radiol (NY), 2023, 48(3)."},{"key":"5818_CR10","doi-asserted-by":"crossref","unstructured":"Mengqiu C, Xueyi N, Huiping G, et al. A simple method based on qualitative MRI features for characterizing clear cell renal cell carcinoma in small renal masses: comparison with the clear cell likelihood score [J]. Abdom Radiol (NY), 2025, 50(9).","DOI":"10.1007\/s00261-025-04844-9"},{"key":"5818_CR11","doi-asserted-by":"crossref","unstructured":"Nicola S, Matthew S D, Stuart G S, et al. Multicenter Evaluation of Multiparametric MRI Clear Cell Likelihood Scores in Solid Indeterminate Small Renal Masses [J]. Radiology, 2023, 306(3).","DOI":"10.1148\/radiol.239001"},{"key":"5818_CR12","doi-asserted-by":"crossref","unstructured":"Eleonora S-S, Elissandra Melo L, Viviane Brand\u00e3o A, et al. Clear cell likelihood score may improve diagnosis and management of renal masses [J]. Abdom Radiol (NY), 2024, 49(12).","DOI":"10.1007\/s00261-024-04415-4"},{"key":"5818_CR13","doi-asserted-by":"crossref","unstructured":"Noah E C, Fernando U K, Yin X, et al. Diagnostic Accuracy of Multiparametric Magnetic Resonance Imaging to Identify Clear Cell Renal Cell Carcinoma in cT1a Renal Masses [J]. J Urol, 2017, 198(4).","DOI":"10.1016\/j.juro.2017.04.089"},{"key":"5818_CR14","doi-asserted-by":"crossref","unstructured":"Matthew S D, Mark A P, Jerry Y, et al. Use of Intravenous Iodinated Contrast Media in Patients with Kidney Disease: Consensus Statements from the American College of Radiology and the National Kidney Foundation [J]. Radiology, 2020, 294(3).","DOI":"10.1148\/radiol.2019192094"},{"key":"5818_CR15","doi-asserted-by":"crossref","unstructured":"Tianjun S, Bing X, Manman L, et al. Development and validation of a fusion model based on multi-phase contrast CT radiomics combined with clinical features for predicting Ki-67 expression in gastric cancer [J]. Biomed Rep, 2025, 23(1).","DOI":"10.3892\/br.2025.1996"},{"key":"5818_CR16","doi-asserted-by":"crossref","unstructured":"Rajesh B, Robert R B, Satheesh K. GPT-4 in Radiology: Improvements in Advanced Reasoning [J]. Radiology, 2023, 307(5).","DOI":"10.1148\/radiol.230987"},{"key":"5818_CR17","doi-asserted-by":"crossref","unstructured":"Donhee L, Seong No Y. Application of Artificial Intelligence-Based Technologies in the Healthcare Industry: Opportunities and Challenges [J]. Int J Environ Res Public Health, 2021, 18(1).","DOI":"10.3390\/ijerph18010271"},{"key":"5818_CR18","doi-asserted-by":"crossref","unstructured":"Usman I, Afifa T, Annisa Ristya R, et al. Impact of large language model (ChatGPT) in healthcare: an umbrella review and evidence synthesis [J]. J Biomed Sci, 2025, 32(1).","DOI":"10.1186\/s12929-025-01131-z"},{"key":"5818_CR19","doi-asserted-by":"crossref","unstructured":"Amir M H, Shiva S, Aryan Z, et al. Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports [J]. Eur Radiol, 2023, 34(6).","DOI":"10.1007\/s00330-023-10384-x"},{"key":"5818_CR20","doi-asserted-by":"crossref","unstructured":"Katharina J, Balthasar S, Jakob D, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports [J]. Eur Radiol, 2023, 34(5).","DOI":"10.1007\/s00330-023-10213-1"},{"key":"5818_CR21","doi-asserted-by":"crossref","unstructured":"Pilar L-\u00da, Teodoro M-N, Alba R-V, et al. The added value of including thyroid nodule features into large language models for automatic ACR TI-RADS classification based on ultrasound reports [J]. Jpn J Radiol, 2024, 43(4).","DOI":"10.1007\/s11604-024-01707-z"},{"key":"5818_CR22","doi-asserted-by":"crossref","unstructured":"Su M, Liang X, Zeng X T, et al. Using a Large Language Model for Breast Imaging Reporting and Data System Classification and Malignancy Prediction to Enhance Breast Ultrasound Diagnosis: Retrospective Study [J]. JMIR Med Inform, 2025, 13(0).","DOI":"10.2196\/70924"},{"key":"5818_CR23","unstructured":"Siying Z, Zhenping W, Mingyang G, et al. Limitations of Large Language Models in Assisting PI-RADS Scoring on Prostate Biparametric MRI Text Reports [J]. Acad Radiol, 2026, (0)."},{"key":"5818_CR24","doi-asserted-by":"crossref","unstructured":"Mario M-G, Anna D, Marc C-C, et al. Utilizing a domain-specific large language model for LI-RADS v2018 categorization of free-text MRI reports: a feasibility study [J]. Insights Imaging, 2024, 15(1).","DOI":"10.1186\/s13244-024-01850-1"},{"key":"5818_CR25","doi-asserted-by":"crossref","unstructured":"Rajesh B, Ankush J, Tanya C, et al. Accuracy of Large Language Model-based Automatic Calculation of Ovarian-Adnexal Reporting and Data System MRI Scores from Pelvic MRI Reports [J]. Radiology, 2025, 315(1).","DOI":"10.1148\/radiol.241554"},{"key":"5818_CR26","doi-asserted-by":"crossref","unstructured":"Betul A D, Muhammed Said B. Evaluation of large language models in assigning PI-RADS v2.1 categories for prostate MRI reports [J]. BMC Urol, 2026, 26(1).","DOI":"10.1186\/s12894-025-02038-5"},{"key":"5818_CR27","unstructured":"Mary L M. Interrater reliability: the kappa statistic [J]. Biochem Med (Zagreb), 2012, 22(3)."}],"container-title":["Abdominal Radiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00261-026-05818-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00261-026-05818-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00261-026-05818-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T07:10:13Z","timestamp":1791184213000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00261-026-05818-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,5]]},"references-count":27,"alternative-id":["5818"],"URL":"https:\/\/doi.org\/10.1007\/s00261-026-05818-1","relation":{},"ISSN":["2366-0058"],"issn-type":[{"value":"2366-0058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,5]]},"assertion":[{"value":"18 June 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 September 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 October 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}]}}