{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,5]],"date-time":"2026-11-05T02:08:22Z","timestamp":1793844502904,"version":"4.6.1"},"reference-count":41,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,3,1]],"date-time":"2026-03-01T00:00:00Z","timestamp":1772323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,3,1]],"date-time":"2026-03-01T00:00:00Z","timestamp":1772323200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100010148","name":"Asociaci\u00f3n Espa\u00f1ola de Urolog\u00eda","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100010148","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Therapeutic Advances in Urology"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Large language models (LLMs) are increasingly being explored to supporting evidence-based decision-making in urology, but their accuracy in interpreting and applying clinical guidelines remains uncertain.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objectives:<\/jats:title>\n                    <jats:p>We aimed to evaluate the ability of LLMs to interpret and apply clinical guidelines across the full spectrum of major urological cancers.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Design:<\/jats:title>\n                    <jats:p>This expert-validated study evaluated six configurations of three top LLMs (Claude, Gemini, and ChatGPT) using 25 structured questions for each of the seven major urological cancers: prostate cancer, upper tract urothelial carcinoma, muscle-invasive and non-muscle-invasive bladder cancer, renal cell carcinoma, penile cancer, and testicular cancer.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>Both simple and rephrased prompts were used to assess the impact of prompt engineering on response quality. All figures and tables from the English-language EAU guidelines were systematically converted into plain, structured text and peer reviewed by multidisciplinary experts before evaluating the LLM responses. Each response was independently rated by 9\u201311 uro-oncology specialists using a five-point Likert scale (1: incorrect\/unacceptable, 5: optimal), resulting in 10,500 evaluations.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>Claude achieved the highest overall accuracy, with 45.9% of responses rated as optimal (Likert 5) and 87% as optimal\/acceptable (Likert 4\u20135). Tumor-specific performance peaked in muscle-invasive bladder (56.7% optimal, 93% optimal\/acceptable), penile (49.5%, 95%), and testicular cancer (60.9%, 94%). Gemini and ChatGPT showed lower optimal rates but acceptable performance (68%\u201370% optimal\/acceptable). Rephrased prompts did not consistently outperform simple versions. All models showed acceptable accuracy, but the results should be interpreted cautiously due to recency bias and fast LLM tech evolution.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>This study demonstrates the value of rigorous plain language adaptation and expert validation in benchmarking LLMs, supporting their potential as decision-support tools in uro-oncology.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/17562872261436905","type":"journal-article","created":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T07:16:47Z","timestamp":1774682207000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":8,"title":["Accuracy of large language models in interpreting urological clinical guidelines: a comparative study with expert evaluation"],"prefix":"10.1177","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0178-4567","authenticated-orcid":false,"given":"\u00c1ngel","family":"Borque-Fernando","sequence":"first","affiliation":[{"name":"Unidad de Pr\u00f3stata, Servicio de Urolog\u00eda, Hospital Universitario Miguel Servet, Zaragoza, Spain"},{"name":"\u00c1rea de Urolog\u00eda, Departamento de Cirug\u00eda, Facultad de Medicina, Universidad de Zaragoza, Zaragoza, Spain"},{"name":"Grupo de Investigaci\u00f3n \u201cURO-SERVET,\u201d IIS-Arag\u00f3n, Zaragoza, Spain"},{"name":"Grupo de Ingenier\u00eda y Ciencia de Datos Aplicada, Escuela Universitaria Polit\u00e9cnica de La Almunia, Universidad de Zaragoza, Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Denis","family":"Navarro","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering and Communications I3A, Universidad de Zaragoza, Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manuel","family":"Doblare","sequence":"additional","affiliation":[{"name":"Arag\u00f3n Institute of Engineering Research (I3A), Zaragoza, Spain"},{"name":"Arag\u00f3n Institute of Health Research (IIS Arag\u00f3n), Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3007-302X","authenticated-orcid":false,"given":"Luis M.","family":"Esteban","sequence":"additional","affiliation":[{"name":"Escuela Universitaria Polit\u00e9cnica de La Almunia, Instituto de Biocomputacion y F\u00edsica de Sistemas Complejos, Universidad de Zaragoza, Zaragoza, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Perez-Fentes","sequence":"additional","affiliation":[{"name":"Urology Department, Santiago de Compostela University Hospital Complex, Santiago de Compostela, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"\u00c1lvarez-Maestro","sequence":"additional","affiliation":[{"name":"Servicio Urolog\u00eda, Hospital Universitario La Paz, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rafael A.","family":"Medina-L\u00f3pez","sequence":"additional","affiliation":[{"name":"Servicio Urolog\u00eda, Hospital Universitario Virgen del Roc\u00edo, Instituto de Biomedicina de Sevilla (IBiS), Sevilla, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oscar","family":"Rodr\u00edguez Faba","sequence":"additional","affiliation":[{"name":"Servicio de Urolog\u00eda, Fundaci\u00f3 Puigvert Barcelona, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9","family":"Rubio-Briones","sequence":"additional","affiliation":[{"name":"Servicio de Urolog\u00eda, Hospital VITHAS 9 de Octubre, Valencia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergio","family":"Fern\u00e1ndez-Pello","sequence":"additional","affiliation":[{"name":"Servicio de Urolog\u00eda, Hospital Universitario de Cabue\u00f1es, Gij\u00f3n, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jes\u00fas Mar\u00eda","family":"Fern\u00e1ndez-G\u00f3mez","sequence":"additional","affiliation":[{"name":"Hospital Universitario Central de Asturias, Universidad de Oviedo, Oviedo, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom\u00e1s","family":"Fern\u00e1ndez Aparicio","sequence":"additional","affiliation":[{"name":"Servicio de Urologia, Hospital Morales Meseguer, Murcia, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"F\u00e9lix","family":"Guerrero Ramos","sequence":"additional","affiliation":[{"name":"Unidad de UroOncolog\u00eda, Servicio de Urolog\u00eda, Hospital Universitario 12 de Octubre, Madrid, Spain"},{"name":"Unidad de Urooncolog\u00eda, Departamento de Urolog\u00eda, Hospital Universitario HM Sanchinarro, Hospital Universitario HM Montepr\u00edncipe, Hospital Universitario HM Puerta del Sur, Madrid, Espa\u00f1a"},{"name":"Facultad de Medicina, Universidad San Pablo CEU, Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Izquierdo","sequence":"additional","affiliation":[{"name":"Department of Urology, Hospital Clinic, Barcelona, Spain"},{"name":"Genetics and Urologic Tumors, Institut d\u2019Investigacions Biom\u00e8diques August Pi i Sunnyer (IDIBAPS), Barcelona, Spain"},{"name":"Department of Surgery and Medical-Surgical Specialties, Medicine and Health Sciences Faculty, Universitat de Barcelona, Barcelona, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jos\u00e9 Luis","family":"\u00c1lvarez-Ossorio Fern\u00e1ndez","sequence":"additional","affiliation":[{"name":"Director del Patronato de la Fundaci\u00f3n para la Investigaci\u00f3n en Urolog\u00eda (Asociaci\u00f3n Espa\u00f1ola de Urolog\u00eda), Jefe de Servicio Urolog\u00eda Hospital Universitario Puerta del Mar, C\u00e1diz, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,3,28]]},"reference":[{"key":"e_1_3_4_2_2","unstructured":"GPT-4o. https:\/\/openai.com\/es-ES\/index\/hello-gpt-4o\/ (2024 accessed 23 June 2025)."},{"key":"e_1_3_4_3_2","unstructured":"Google Gemini. https:\/\/gemini.google.com (2024 accessed 23 June 2025)."},{"key":"e_1_3_4_4_2","unstructured":"Meet Claude. Anthropic. https:\/\/www.anthropic.com\/claude (2024 accessed 23 June 2025)."},{"key":"e_1_3_4_5_2","doi-asserted-by":"publisher","DOI":"10.1101\/2024.08.11.24311828"},{"key":"e_1_3_4_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01344-w"},{"key":"e_1_3_4_7_2","unstructured":"EAU Guidelines - Uroweb https:\/\/uroweb.org\/guidelines (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2023.07.004"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.2196\/22769"},{"key":"e_1_3_4_10_2","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2506.22405"},{"key":"e_1_3_4_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.identj.2024.09.033"},{"key":"e_1_3_4_12_2","unstructured":"EAU Guidelines on Prostate Cancer - Uroweb https:\/\/uroweb.org\/guidelines\/prostate-cancer (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_13_2","unstructured":"EAU Guidelines on Upper Urinary Tract Urothelial Cell Carcinoma - Uroweb https:\/\/uroweb.org\/guidelines\/upper-urinary-tract-urothelial-cell-carcinoma (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_14_2","unstructured":"EAU Guidelines on Muscle-invasive and Metastatic Bladder Cancer - Uroweb https:\/\/uroweb.org\/guidelines\/muscle-invasive-and-metastatic-bladder-cancer (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_15_2","unstructured":"EAU Guidelines on Non-muscle-invasive Bladder Cancer - Uroweb https:\/\/uroweb.org\/guidelines\/non-muscle-invasive-bladder-cancer (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_16_2","unstructured":"EAU Guidelines on Testicular Cancer \u2013 Uroweb https:\/\/uroweb.org\/guidelines\/testicular-cancer (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_17_2","unstructured":"EAU Guidelines on Renal Cell Carcinoma - Uroweb https:\/\/uroweb.org\/guidelines\/renal-cell-carcinoma (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_18_2","unstructured":"EAU Guidelines on Penile Cancer - Uroweb https:\/\/uroweb.org\/guidelines\/penile-cancer (2023 accessed 19 May 2025)."},{"key":"e_1_3_4_19_2","doi-asserted-by":"publisher","DOI":"10.2196\/50638"},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics13152961"},{"key":"e_1_3_4_21_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0044-1787974"},{"key":"e_1_3_4_22_2","unstructured":"Methodology & Polices - Uroweb https:\/\/uroweb.org\/eau-guidelines\/methodology-policies (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_23_2","unstructured":"Standard Operating Procedures Overview - American Urological Association https:\/\/www.auanet.org\/guidelines-and-quality\/guidelines\/standard-operating-procedures-overview (2025 accessed 19 May 2025)."},{"key":"e_1_3_4_24_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.urpr.2016.10.003"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00432-019-02864-9"},{"key":"e_1_3_4_26_2","doi-asserted-by":"publisher","DOI":"10.1159\/000486344"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.euf.2017.09.002"},{"key":"e_1_3_4_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2022.12.031"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2020.10.011"},{"key":"e_1_3_4_30_2","unstructured":"Large Language Models are Transformers in Artificial Intelligence Industry Education and Society 2024."},{"key":"e_1_3_4_31_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-98483-1"},{"key":"e_1_3_4_32_2","doi-asserted-by":"publisher","DOI":"10.3390\/siuj5060064"},{"key":"e_1_3_4_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00345-024-05146-3"},{"key":"e_1_3_4_34_2","article-title":"Application of large language models in healthcare: a bibliometric analysis","volume":"11","author":"Zhang L","year":"2025","unstructured":"Zhang L, Zhao Q, Zhang D, et al. Application of large language models in healthcare: a bibliometric analysis. Digit Health 2025; 11: 20552076251324444.","journal-title":"Digit Health"},{"key":"e_1_3_4_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.82"},{"key":"e_1_3_4_36_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-024-40763-6"},{"key":"e_1_3_4_37_2","doi-asserted-by":"publisher","DOI":"10.1136\/jech.53.2.105"},{"key":"e_1_3_4_38_2","doi-asserted-by":"publisher","DOI":"10.2196\/63489"},{"key":"e_1_3_4_39_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0044-1782165"},{"key":"e_1_3_4_40_2","first-page":"804","volume-title":"Proceedings of the 8th machine learning for healthcare conference","author":"Wang Y","year":"2023","unstructured":"Wang Y, Zhao Y, Petzold L. Are large language models ready for healthcare? a comparative study on clinical language understanding. In: Proceedings of the 8th machine learning for healthcare conference, PMLR 219, pp. 804\u2013823, 2023."},{"key":"e_1_3_4_41_2","doi-asserted-by":"publisher","DOI":"10.1080\/00015385.2024.2303528"},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","DOI":"10.1089\/end.2024.0860"}],"container-title":["Therapeutic Advances in Urology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/17562872261436905","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/17562872261436905","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/17562872261436905","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T07:16:58Z","timestamp":1774941418000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/17562872261436905"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3]]},"references-count":41,"alternative-id":["10.1177\/17562872261436905"],"URL":"https:\/\/doi.org\/10.1177\/17562872261436905","relation":{},"ISSN":["1756-2872","1756-2880"],"issn-type":[{"value":"1756-2872","type":"print"},{"value":"1756-2880","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3]]},"article-number":"17562872261436905"}}