{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,11]],"date-time":"2026-09-11T01:18:05Z","timestamp":1789089485537,"version":"build-2803163510"},"reference-count":15,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T00:00:00Z","timestamp":1780704000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T00:00:00Z","timestamp":1780704000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Usak University"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Urolithiasis"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Large language models (LLMs) are increasingly investigated for their potential role in guideline-based clinical information support. However, their consistency with subspecialty guidelines, particularly in urolithiasis, remains underexplored. This study aimed to evaluate the performance of four large language models (LLMs); GPT-4, GPT-4-turbo, Claude, and Gemini in generating guideline-concordant responses to clinical questions related to urolithiasis. A total of 105 clinical questions were independently developed by the authors based on urolithiasis management principles. Each LLM generated responses in two separate sessions. Two experienced urologists evaluated the outputs for accuracy and concordance with guideline recommendations. Inter-rater agreement analysis demonstrated fair agreement between evaluators. Differences across models and guideline categories were assessed using appropriate statistical tests. All four LLMs demonstrated high guideline adherence, with mean total scores ranging from 86.5\u2009\u00b1\u20095.2 (Gemini) to 92.8\u2009\u00b1\u20093.1 (Claude). Claude achieved the highest correlation with expert ratings (\n                    <jats:italic>r<\/jats:italic>\n                    \u2009=\u20090.94,\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&lt;\u20090.01). There were no statistically significant differences across models or among the nine clinical categories (\n                    <jats:italic>p<\/jats:italic>\n                    \u2009&gt;\u20090.05). Session-to-session repeatability was also high for all models, with intra-model correlation coefficients exceeding 0.90. LLMs, particularly Claude, can provide reliable, guideline-consistent answers to urolithiasis-related clinical queries. Their consistent performance across themes suggests utility as adjunctive informational tools for guideline-based urological education and support, although further validation in real-world clinical settings remains necessary.\n                  <\/jats:p>","DOI":"10.1007\/s00240-026-02013-1","type":"journal-article","created":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T12:59:22Z","timestamp":1780750762000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Performance of large language models in urological decision support: a guideline-based comparative evaluation in urolithiasis"],"prefix":"10.1007","volume":"54","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9951-2556","authenticated-orcid":false,"given":"Adem","family":"Tun\u00e7ekin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5255-3780","authenticated-orcid":false,"given":"Yasin","family":"Akta\u015f","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,6]]},"reference":[{"key":"2013_CR1","doi-asserted-by":"publisher","first-page":"205520762412972","DOI":"10.1177\/20552076241297237","volume":"8","author":"S Qu","year":"2024","unstructured":"Qu S, Liu L, Zhou M, Zhou C, Campy KS (2024) Factors influencing Chinese doctors to use medical large language models. Digit Health 8:10:20552076241297237. https:\/\/doi.org\/10.1177\/20552076241297237","journal-title":"Digit Health"},{"issue":"4","key":"2013_CR2","doi-asserted-by":"publisher","first-page":"e200250","DOI":"10.1212\/NE9.0000000000200250","volume":"19","author":"C Gao","year":"2025","unstructured":"Gao C, Gheihman G, Kaplan T, McCoy LG, Collins LC, Wenzel T, Paul A, Reda H, Stein LK, Kimbaris G, Sutherland HW, Stacpoole S, Milligan TA, Goh R, Bacchi S (2025) Education Research: Creating Online Interactive Case-Based Learning Experiences From Educational Case Reports With Large Language Models: A Feasibility Study. Neurol Educ Sep 19(4):e200250. https:\/\/doi.org\/10.1212\/NE9.0000000000200250","journal-title":"Neurol Educ Sep"},{"key":"2013_CR3","doi-asserted-by":"publisher","unstructured":"Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepa\u00f1o C, Madriaga M, Aggabao R, Diaz-Candido G, Maningo J, Tseng V (2023) Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health Feb 9;2(2): e0000198. https:\/\/doi.org\/10.1371\/journal.pdig.0000198","DOI":"10.1371\/journal.pdig.0000198"},{"key":"2013_CR4","doi-asserted-by":"publisher","unstructured":"Li\u00e9vin V, Hother CE, Motzfeldt AG, Winther O (2024) Can large language models reason about medical questions? Patterns (N Y) Mar 1. 5(3):100943. https:\/\/doi.org\/10.1016\/j.patter.2024.100943","DOI":"10.1016\/j.patter.2024.100943"},{"key":"2013_CR5","doi-asserted-by":"publisher","first-page":"e66905","DOI":"10.2196\/66905","volume":"2913","author":"HM Serugunda","year":"2025","unstructured":"Serugunda HM, Jianquan O, Kasujja Namatovu H, Ssemaluulu P, Kimbugwe N, Garimoi Orach C, Waiswa P (2025) Using Large Language Models for Chronic Disease Management Tasks: Scoping Review. JMIR Med Inf Sep 2913:e66905. https:\/\/doi.org\/10.2196\/66905","journal-title":"JMIR Med Inf Sep"},{"issue":"2","key":"2013_CR6","doi-asserted-by":"publisher","first-page":"40","DOI":"10.4274\/uob.galenos.2025.2025.1.2","volume":"25","author":"\u00dc Uysal","year":"2025","unstructured":"Uysal \u00dc, U\u00e7ar M, Sa\u011f\u0131r S (2025) ChatGPT-3.5 and ChatGPT-4 Performance in Testicular Cancer: A Comparative Study. Bull Urooncol Jun 25(2):40\u201346. https:\/\/doi.org\/10.4274\/uob.galenos.2025.2025.1.2","journal-title":"Bull Urooncol Jun"},{"issue":"1","key":"2013_CR7","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1007\/s11255-023-03773-0","volume":"56","author":"H Cakir","year":"2024","unstructured":"Cakir H, Caglar U, Yildiz O, Meric A, Ayranci A, Ozgor F (2024) Evaluating the performance of ChatGPT in answering questions related to urolithiasis. Int Urol Nephrol Jan 56(1):17\u201321. https:\/\/doi.org\/10.1007\/s11255-023-03773-0","journal-title":"Int Urol Nephrol Jan"},{"key":"2013_CR8","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.urology.2023.05.010","volume":"177","author":"NA Deebel","year":"2023","unstructured":"Deebel NA, Terlecki R (2023) ChatGPT Performance on the American Urological Association Self-assessment Study Program and the Potential Influence of Artificial Intelligence in Urologic Training. Urol Jul 177:29\u201333. https:\/\/doi.org\/10.1016\/j.urology.2023.05.010","journal-title":"Urol Jul"},{"issue":"6","key":"2013_CR9","doi-asserted-by":"publisher","first-page":"3887","DOI":"10.1245\/s10434-024-15107-0","volume":"31","author":"R Liang","year":"2024","unstructured":"Liang R, Zhao A, Peng L, Xu X, Zhong J, Wu F, Yi F, Zhang S, Wu S, Hou J (2024) Enhanced Artificial Intelligence Strategies in Renal Oncology: Iterative Optimization and Comparative Analysis of GPT 3.5 Versus 4.0. Ann Surg Oncol Jun 31(6):3887\u20133893. https:\/\/doi.org\/10.1245\/s10434-024-15107-0","journal-title":"Ann Surg Oncol Jun"},{"issue":"1","key":"2013_CR10","doi-asserted-by":"publisher","first-page":"64","DOI":"10.1016\/j.eururo.2025.03.011","volume":"88","author":"A Skolarikos","year":"2025","unstructured":"Skolarikos A, Geraghty R, Somani B, Tailly T, Jung H, Neisius A, Pet\u0159\u00edk A, Kamphuis GM, Davis N, Bezuidenhout C, Lardas M, Gambaro G, Sayer JA, Lombardo R, Tzelves L (2025) European Association of Urology Guidelines on the Diagnosis and Treatment of Urolithiasis. Eur Urol Jul 88(1):64\u201375. https:\/\/doi.org\/10.1016\/j.eururo.2025.03.011","journal-title":"Eur Urol Jul"},{"issue":"13","key":"2013_CR11","doi-asserted-by":"publisher","first-page":"1233","DOI":"10.1056\/NEJMsr2214184","volume":"30","author":"P Lee","year":"2023","unstructured":"Lee P, Bubeck S, Petro J (2023) Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine. N Engl J Med Mar 30(13):1233\u20131239. https:\/\/doi.org\/10.1056\/NEJMsr2214184","journal-title":"N Engl J Med Mar"},{"issue":"2","key":"2013_CR12","doi-asserted-by":"publisher","first-page":"130","DOI":"10.1097\/CM9.0000000000003456","volume":"20","author":"X Yang","year":"2025","unstructured":"Yang X, Li T, Su Q, Liu Y, Kang C, Lyu Y, Zhao L, Nie Y, Pan Y (2025) Application of large language models in disease diagnosis and treatment. Chin Med J (Engl) Jan 20(2):130\u2013142. https:\/\/doi.org\/10.1097\/CM9.0000000000003456","journal-title":"Chin Med J (Engl) Jan"},{"issue":"4","key":"2013_CR13","doi-asserted-by":"publisher","first-page":"2294","DOI":"10.1007\/s10278-024-01161-3","volume":"38","author":"L Miller","year":"2025","unstructured":"Miller L, Kamel P, Patel J, Agrawal J, Zhan M, Bumbarger N, Wang K (2025) A Comparative Evaluation of Large Language Model Utility in Neuroimaging Clinical Decision Support. J Imaging Inf Med Aug 38(4):2294\u20132302. https:\/\/doi.org\/10.1007\/s10278-024-01161-3","journal-title":"J Imaging Inf Med Aug"},{"key":"2013_CR14","doi-asserted-by":"publisher","first-page":"e73603","DOI":"10.2196\/73603","volume":"1827","author":"M Toal","year":"2025","unstructured":"Toal M, Hill C, Quinn M, O\u2019Neill C, Maxwell AP (2025) Large Language Models\u2019 Clinical Decision-Making on When to Perform a Kidney Biopsy: Comparative Study. J Med Internet Res Sep 1827:e73603. https:\/\/doi.org\/10.2196\/73603","journal-title":"J Med Internet Res Sep"},{"issue":"1","key":"2013_CR15","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1007\/s10916-025-02152-9","volume":"14","author":"X Wang","year":"2025","unstructured":"Wang X, Ye H, Zhang S, Yang M, Wang X (2025) Evaluation of the Performance of Three Large Language Models in Clinical Decision Support: A Comparative Study Based on Actual Cases. J Med Syst Feb 14(1):23. https:\/\/doi.org\/10.1007\/s10916-025-02152-9","journal-title":"J Med Syst Feb"}],"container-title":["Urolithiasis"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00240-026-02013-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00240-026-02013-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00240-026-02013-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T12:59:23Z","timestamp":1780750763000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00240-026-02013-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,6]]},"references-count":15,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["2013"],"URL":"https:\/\/doi.org\/10.1007\/s00240-026-02013-1","relation":{},"ISSN":["2194-7236"],"issn-type":[{"value":"2194-7236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,6]]},"assertion":[{"value":"28 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 May 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflicts of interest relevant to the content of this manuscript.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This study did not involve human participants, identifiable data, or animal subjects. Therefore, ethical approval was not required in accordance with institutional and international guidelines.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"110"}}