{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T18:13:16Z","timestamp":1791223996488,"version":"4.1.0"},"reference-count":31,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,1,2]],"date-time":"2026-01-02T00:00:00Z","timestamp":1767312000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,2,2]],"date-time":"2026-02-02T00:00:00Z","timestamp":1769990400000},"content-version":"vor","delay-in-days":31,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Urol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>This study aimed to evaluate the performance of large language models (LLMs) in classifying prostate MRI reports according to the Prostate Imaging\u2013Reporting and Data System (PIRADS) version 2.1, and to validate their use in supporting clinical decisions in prostate cancer treatment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This retrospective study included 146 patients. Four LLMs \u2014 GPT-4o, GPT-o1, Google Gemini 1.5 Pro and Google Gemini 2.0 Experimental Advanced \u2014 were tested on standardised, structured prostate MRI reports. A two-radiologist consensus reference standard was used to compare model performance. Agreement was measured using weighted Cohen\u2019s kappa, and accuracy and F1 scores were calculated for three PI-RADS risk groups: low (1\u20132), intermediate (3) and high (4\u20135).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Performance varied by model. GPT-o1 achieved the highest level of agreement with radiologists (\u03ba\u2009=\u20090.867), followed by GPT-4o (\u03ba\u2009=\u20090.743), Gemini 1.5 Pro (\u03ba\u2009=\u20090.728) and Gemini 2.0 Experimental Advanced (\u03ba\u2009=\u20090.664). GPT-o1 achieved the highest F1 scores for the low-risk (0.93) and high-risk (1.00) groups, demonstrating moderate performance for the PI-RADS 3 group (0.75). All models showed weak performance for PI-RADS 3 (F1 range: 0.54\u20130.75). Most importantly, none of the models produced invalid results outside the target PI-RADS 1\u20135 range.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>LLMs show potential for automating PI-RADS classification from MRI reports, with GPT-o1 demonstrating the best overall performance. However, their failure in PI-RADS 3 lesions indicates that multicentre validation, larger datasets and multimodality integration are needed before they can be used clinically for prostate cancer diagnosis and urological decision-making.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Trial registration<\/jats:title>\n                    <jats:p>Not applicable. This retrospective study did not involve a clinical trial.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1186\/s12894-025-02038-5","type":"journal-article","created":{"date-parts":[[2026,1,3]],"date-time":"2026-01-03T05:32:19Z","timestamp":1767418339000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Evaluation of large language models in assigning PI-RADS v2.1 categories for prostate MRI reports"],"prefix":"10.1186","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8556-6278","authenticated-orcid":false,"given":"Betul","family":"Akdal Dolek","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammed Said","family":"Besler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,1,2]]},"reference":[{"key":"2038_CR1","first-page":"17","volume":"73","author":"RL Siegel","year":"2023","unstructured":"Siegel RL, Miller KD, Wagle NS, Jemal A. Cancer statistics, 2023. CA Cancer J Clin. 2023;73:17\u201348.","journal-title":"CA Cancer J Clin"},{"key":"2038_CR2","first-page":"209","volume":"71","author":"H Sung","year":"2021","unstructured":"Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin. 2021;71:209\u201349.","journal-title":"CA Cancer J Clin"},{"key":"2038_CR3","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1016\/j.eururo.2019.02.033","volume":"76","author":"B Turkbey","year":"2019","unstructured":"Turkbey B, Rosenkrantz AB, Haider MA, Padhani AR, Villeirs G, Macura KJ, Tempany CM, Choyke PL, Cornud F, Margolis DJ, Thoeny HC, Verma S, Barentsz J, Weinreb JC. Prostate imaging reporting and data system version 2.1: 2019 update of prostate imaging reporting and data system version 2. Eur Urol. 2019;76:340\u201351.","journal-title":"Eur Urol"},{"key":"2038_CR4","doi-asserted-by":"publisher","first-page":"80","DOI":"10.4274\/dir.2023.232417","volume":"30","author":"D Akinci","year":"2024","unstructured":"Akinci D, Stanzione A, Bluethgen C, Vernuccio F, Ugga L, Klontzas ME, et al. Large language models in radiology: fundamentals, applications, ethical considerations, risks, and future directions. Diagn Interv Radiol. 2024;30:80\u201390.","journal-title":"Diagn Interv Radiol"},{"key":"2038_CR5","doi-asserted-by":"publisher","first-page":"126","DOI":"10.3348\/kjr.2023.0997","volume":"25","author":"S Kim","year":"2024","unstructured":"Kim S, Lee CK, Kim SS. Large language models: a guide for radiologists. Korean J Radiol. 2024;25:126\u201333.","journal-title":"Korean J Radiol"},{"key":"2038_CR6","doi-asserted-by":"crossref","unstructured":"Busch F, Hoffmann L, Dos Santos DP, Makowski MR, Saba L, Prucker P, Hadamitzky M, Navab N, Kather JN, Truhn D, Cuocolo R, Adams LC, Bressem KK. Large Language models for structured reporting in radiology: past, present, and future. Eur Radiol. 2025;35(5):2589\u20132602.","DOI":"10.1007\/s00330-024-11107-6"},{"key":"2038_CR7","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1007\/s11604-024-01552-0","volume":"42","author":"T Nakaura","year":"2024","unstructured":"Nakaura T, Ito R, Ueda D, Nozaki T, Fushimi Y, Matsui Y, et al. The impact of large language models on radiology: a guide for radiologists on the latest innovations in AI. Jpn J Radiol. 2024;42:685\u201396.","journal-title":"Jpn J Radiol"},{"key":"2038_CR8","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.232133","volume":"311","author":"A Cozzi","year":"2024","unstructured":"Cozzi A, Pinker K, Hidber A, Zhang T, Bonomo L, Lo Gullo R, et al. BI-RADS category assignments by GPT-3.5, GPT-4, and Google Bard: a multilanguage study. Radiology. 2024;311:e232133.","journal-title":"Radiology"},{"issue":"11","key":"2038_CR9","doi-asserted-by":"publisher","DOI":"10.2196\/70924","volume":"13","author":"S Miaojiao","year":"2025","unstructured":"Miaojiao S, Xia L, Xian Tao Z, Zhi Liang H, Sheng C, Songsong W. Using a large language model for breast imaging reporting and data system classification and malignancy prediction to enhance breast ultrasound diagnosis: retrospective study. JMIR Med Inform. 2025;13(11):e70924.","journal-title":"JMIR Med Inform"},{"key":"2038_CR10","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinimag.2025.110455","volume":"121","author":"R Singh","year":"2025","unstructured":"Singh R, Hamouda M, Chamberlin JH, T\u00f3th A, Munford J, Silbergleit M, et al. ChatGPT vs. Gemini: comparative accuracy and efficiency in Lung-RADS score assignment from radiology reports. Clin Imaging. 2025;121:110455.","journal-title":"Clin Imaging"},{"issue":"1","key":"2038_CR11","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1186\/s13244-024-01850-1","volume":"22","author":"M Matute-Gonz\u00e1lez","year":"2024","unstructured":"Matute-Gonz\u00e1lez M, Darnell A, Comas-Cuf\u00ed M, Paz\u00f3 J, Soler A, Saborido B, Mauro E, Turnes J, Forner A, Reig M, Rimola J. Utilizing a domain-specific large Language model for LI-RADS v2018 categorization of free-text MRI reports: a feasibility study. Insights Imaging. 2024;22(1):280.","journal-title":"Insights Imaging"},{"key":"2038_CR12","doi-asserted-by":"publisher","first-page":"368","DOI":"10.1093\/bjr\/tqae236","volume":"98","author":"KL Lee","year":"2025","unstructured":"Lee KL, Kessler DA, Caglic I, Kuo YH, Shaida N, Barrett T. Assessing the performance of ChatGPT and Bard\/Gemini against radiologists for PI-RADS classification based on prostate multiparametric MRI text reports. Br J Radiol. 2025;98:368\u201374.","journal-title":"Br J Radiol"},{"key":"2038_CR13","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"ML McHugh","year":"2012","unstructured":"McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22:276\u201382.","journal-title":"Biochem Med"},{"key":"2038_CR14","doi-asserted-by":"publisher","first-page":"1046","DOI":"10.1016\/j.eururo.2013.12.062","volume":"65","author":"S Loeb","year":"2014","unstructured":"Loeb S, Bjurlin MA, Nicholson J, Tammela TL, Penson DF, Carter HB, Carroll P, Etzioni R. Overdiagnosis and overtreatment of prostate cancer. Eur Urol. 2014;65:1046\u201355.","journal-title":"Eur Urol"},{"issue":"3","key":"2038_CR15","doi-asserted-by":"publisher","first-page":"e244258","DOI":"10.1001\/jamanetworkopen.2024.4258","volume":"7","author":"A Haj-Mirzaian","year":"2024","unstructured":"Haj-Mirzaian A, Burk KS, Lacson R, Glazer DI, Saini S, Kibel AS, et al. Magnetic resonance imaging, clinical, and biopsy findings in suspected prostate cancer: a systematic review and meta-analysis. JAMA Netw Open. 2024;7(3):e244258.","journal-title":"JAMA Netw Open"},{"key":"2038_CR16","doi-asserted-by":"crossref","unstructured":"Schr\u00f6der FH, Hugosson J, Roobol MJ, Tammela TL, Ciatto S, Nelen V, Kwiatkowski M, Lujan M, Lilja H, Zappa M, Denis LJ, Recker F, Berenguer A, M\u00e4\u00e4tt\u00e4nen L, Bangma CH, Aus G, Villers A, Rebillard X, van der Kwast T, Blijenberg BG, Moss SM, de Koning HJ, Auvinen A, ERSPC Investigators. (2009) Screening and prostate-cancer mortality in a randomized European study. N Engl J Med. 2009;360:1320\u20131328.","DOI":"10.1056\/NEJMoa0810084"},{"key":"2038_CR17","doi-asserted-by":"crossref","unstructured":"Sidaway P. (2024) MRI-based stratification reduces the risk of overdiagnosis of prostate cancer. Nat Rev Clin Oncol. 2024;21:838.","DOI":"10.1038\/s41571-024-00957-0"},{"issue":"2","key":"2038_CR18","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.euo.2023.08.002","volume":"7","author":"S Wang","year":"2024","unstructured":"Wang S, Kozarek J, Russell R, Drescher M, Khan A, Kundra V, et al. Diagnostic performance of prostate-specific antigen density for detecting clinically significant prostate cancer in the era of magnetic resonance imaging: a systematic review and meta-analysis. Eur Urol Oncol. 2024;7(2):189\u2013203.","journal-title":"Eur Urol Oncol"},{"issue":"4","key":"2038_CR19","doi-asserted-by":"publisher","first-page":"634","DOI":"10.1016\/j.euf.2023.10.006","volume":"10","author":"F Pellegrino","year":"2024","unstructured":"Pellegrino F, Stabile A, Sorce G, Quarta L, Robesti D, Cannoletta D, et al. Added value of prostate-specific antigen density in selecting prostate biopsy candidates among men with elevated prostate-specific antigen and PI-RADS\u2009\u2265\u20093 lesions on multiparametric magnetic resonance imaging of the prostate: a systematic assessment by PI-RADS score. Eur Urol Focus. 2024;10(4):634\u201340.","journal-title":"Eur Urol Focus"},{"key":"2038_CR20","doi-asserted-by":"publisher","DOI":"10.3389\/fonc.2022.1013941","volume":"12","author":"J Wen","year":"2022","unstructured":"Wen J, Ji Y, Han J, Shen X, Qiu Y. Inter-reader agreement of the prostate imaging reporting and data system version v2.1 for detection of prostate cancer: a systematic review and meta-analysis. Front Oncol. 2022;12:1013941.","journal-title":"Front Oncol"},{"key":"2038_CR21","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-021-00438-z","volume":"4","author":"R Aggarwal","year":"2021","unstructured":"Aggarwal R, Sounderajah V, Martin G, Ting DS, Karthikesalingam A, King D, et al. Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis. NPJ Digit Med. 2021;4:65.","journal-title":"NPJ Digit Med"},{"key":"2038_CR22","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231259","volume":"307","author":"Z Sun","year":"2023","unstructured":"Sun Z, Ong H, Kennedy P, Tang L, Chen S, Elias J, et al. Evaluating GPT4 on impressions generation in radiology reports. Radiology. 2023;307:e231259.","journal-title":"Radiology"},{"key":"2038_CR23","doi-asserted-by":"publisher","first-page":"1578","DOI":"10.1111\/liv.15891","volume":"44","author":"K Gu","year":"2024","unstructured":"Gu K, Lee JH, Shin J, Hwang JA, Min JH, Jeong WK, Lee MW, Song KD, Bae SH. Using GPT-4 for LI-RADS feature extraction and categorization with multilingual free-text reports. Liver Int. 2024;44:1578\u201387.","journal-title":"Liver Int"},{"key":"2038_CR24","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1016\/j.jacr.2023.09.011","volume":"21","author":"EB Gordon","year":"2024","unstructured":"Gordon EB, Towbin AJ, Wingrove P, Shafique U, Haas B, Kitts AB, Feldman J, Furlan A. Enhancing patient communication with Chat-GPT in radiology. J Am Coll Radiol. 2024;21:353\u20139.","journal-title":"J Am Coll Radiol"},{"key":"2038_CR25","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.230725","volume":"307","author":"LC Adams","year":"2023","unstructured":"Adams LC, Truhn D, Busch F, Avan K, Stefan MN, Marcus RM, et al. Leveraging GPT-4 for post hoc transformation of free-text radiology reports into structured reporting. Radiology. 2023;307:e230725.","journal-title":"Radiology"},{"key":"2038_CR26","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejrad.2024.111458","volume":"175","author":"H Jiang","year":"2024","unstructured":"Jiang H, Xia S, Yang Y, Xu J, Hua Q, Mei Z, et al. Transforming free-text radiology reports into structured reports using ChatGPT: a study on thyroid ultrasonography. Eur J Radiol. 2024;175:111458.","journal-title":"Eur J Radiol"},{"key":"2038_CR27","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.231167","volume":"308","author":"J Kottlors","year":"2023","unstructured":"Kottlors J, Bratke G, Rauen P, Kabbasch C, Persigehl T, Schlamann M, et al. Feasibility of differential diagnosis based on imaging patterns using a large language model. Radiology. 2023;308:e231167.","journal-title":"Radiology"},{"key":"2038_CR28","doi-asserted-by":"publisher","first-page":"800","DOI":"10.1148\/radiol.2017171920","volume":"286","author":"SH Park","year":"2018","unstructured":"Park SH, Han K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology. 2018;286:800\u20139.","journal-title":"Radiology"},{"key":"2038_CR29","doi-asserted-by":"publisher","first-page":"e232756","DOI":"10.1148\/radiol.232756","volume":"310","author":"R Bhayana","year":"2024","unstructured":"Bhayana R. Chatbots and large language models in radiology: a practical primer for clinical and research applications. Radiology. 2024;310:e232756.","journal-title":"Radiology"},{"issue":"8017","key":"2038_CR30","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1038\/s41586-024-07421-0","volume":"630","author":"S Farquhar","year":"2024","unstructured":"Farquhar S, Kossen J, Kuhn L, Gal Y. Detecting hallucinations in large language models using semantic entropy. Nature. 2024;630(8017):625\u201330.","journal-title":"Nature"},{"key":"2038_CR31","doi-asserted-by":"publisher","first-page":"e59823","DOI":"10.2196\/59823","volume":"14","author":"D Roustan","year":"2025","unstructured":"Roustan D, Bastardot F. The clinicians\u2019 guide to large language models: a general perspective with a focus on hallucinations. Interact J Med Res. 2025;14:e59823.","journal-title":"Interact J Med Res"}],"container-title":["BMC Urology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12894-025-02038-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12894-025-02038-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12894-025-02038-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T16:01:26Z","timestamp":1781625686000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s12894-025-02038-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,2]]},"references-count":31,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["2038"],"URL":"https:\/\/doi.org\/10.1186\/s12894-025-02038-5","relation":{},"ISSN":["1471-2490"],"issn-type":[{"value":"1471-2490","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,2]]},"assertion":[{"value":"3 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 December 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 January 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study protocol was reviewed and approved by the Ankara Bilkent City Hospital Scientific Research Evaluation Board (Approval No: TABED 1-25-1085).","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"27"}}