{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T12:58:03Z","timestamp":1790600283572,"version":"4.1.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T00:00:00Z","timestamp":1790553600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T00:00:00Z","timestamp":1790553600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2023\/13960-4"],"award-info":[{"award-number":["2023\/13960-4"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["World J Urol"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>To explore the use of large language models for Clavien-Dindo classification of complications following percutaneous nephrolithotomy by benchmarking their performance against physician grading.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We evaluated ChatGPT, Copilot, DeepSeek, Gemini, Grok and Llama for defining Clavien-Dindo grades, classifying complications according to the Clavien-Dindo classification, and classifying case summaries with and without in-context prompting. Vignettes were obtained from a published consensus categorization of percutaneous nephrolithotomy complications. We measured agreement with Gwet\u2019s AC2 coefficient and compared it with attending urologists and urology residents.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      All models accurately defined the Clavien-Dindo classification and graded with almost perfect agreement complication descriptions and case summaries (0.97 to 0.98 overall agreement). Six attendings and 15 residents classified the same vignettes, demonstrating lower agreement than the chatbots (0.95,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.001 for residents; 0.96,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009=\u20090.03 for attendings; 0.95,\n                      <jats:italic>p<\/jats:italic>\n                      \u2009&lt;\u20090.001 for all participants). Models\u2019 latency ranged from 0.77 to 3.90\u00a0min, while the fastest physician required 17\u00a0min to complete the same tasks. Models\u2019 intrarater reliability was almost perfect: from 0.97 to 0.99. Interrater reliability between chatbots (0.97, 95% confidence interval: 0.97\u20130.98) was significantly higher than among physicians (0.93, 95% confidence interval: 0.92\u20130.94).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Large language models show promise in being faster, more accurate, and precise than physicians in Clavien-Dindo classification in an experimental setting. We support the need for a prospective agreement study using real-world clinical records. We urge discussion of the legal and ethical implications.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1007\/s00345-026-06833-z","type":"journal-article","created":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T12:00:53Z","timestamp":1790596853000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Automating Clavien\u2013Dindo classification with large language models in percutaneous nephrolithotomy: an exploratory study"],"prefix":"10.1007","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-8674-8921","authenticated-orcid":false,"given":"Gustavo","family":"Perrone","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6963-6117","authenticated-orcid":false,"given":"Alexandre","family":"Danilovic","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7240-7956","authenticated-orcid":false,"given":"Carlos","family":"Batagello","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5845-0894","authenticated-orcid":false,"given":"Fabio Cesar Miranda","family":"Torricelli","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3745-341X","authenticated-orcid":false,"given":"Fabio Carvalho","family":"Vicentini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4334-9803","authenticated-orcid":false,"given":"Giovanni Scala","family":"Marchini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9358-2542","authenticated-orcid":false,"given":"Rodrigo","family":"Perrella","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7395-8370","authenticated-orcid":false,"given":"William C.","family":"Nahas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1328-7421","authenticated-orcid":false,"given":"Eduardo","family":"Mazzucchi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,28]]},"reference":[{"key":"6833_CR1","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1097\/SLA.0000000000006471","volume":"280","author":"F Abbassi","year":"2024","unstructured":"Abbassi F, Pfister M, Lucas KL, Domenghino A, Puhan MA, Clavien PA et al (2024) Milestones in Surgical Complication Reporting Clavien-Dindo Classification 20 Years and Comprehensive Complication Index 10 Years. Ann Surg 280:763\u2013771. https:\/\/doi.org\/10.1097\/SLA.0000000000006471","journal-title":"Ann Surg"},{"key":"6833_CR2","doi-asserted-by":"publisher","first-page":"1274","DOI":"10.1111\/iju.12576","volume":"21","author":"S Poletajew","year":"2014","unstructured":"Poletajew S, Zapala L, Piotrowicz S, Wolyniec P, Sochaj M, Buraczy\u0144ski P et al (2014) Interobserver variability of Clavien-Dindo scoring in urology. Int J Urol 21:1274\u20131278. https:\/\/doi.org\/10.1111\/iju.12576","journal-title":"Int J Urol"},{"key":"6833_CR3","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1016\/j.eururo.2011.10.033","volume":"61","author":"D Mitropoulos","year":"2012","unstructured":"Mitropoulos D, Artibani W, Graefen M, Remzi M, Roupr\u00eat M, Truss M (2012) Reporting and grading of complications after urologic surgical procedures: An ad hoc EAU guidelines panel assessment and recommendations. Eur Urol 61:341\u2013349. https:\/\/doi.org\/10.1016\/j.eururo.2011.10.033","journal-title":"Eur Urol"},{"key":"6833_CR4","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1016\/j.urology.2014.05.028","volume":"84","author":"B Peyronnet","year":"2014","unstructured":"Peyronnet B, Pradere B, Brichart N, Bodin T, Bertrand P, Bruy\u00e8re F (2014) Complications associated with photoselective vaporization of the prostate: Categorization by a panel of greenlight users according to Clavien score and report of a single-center experience. Urology 84:657\u2013664. https:\/\/doi.org\/10.1016\/j.urology.2014.05.028","journal-title":"Urology"},{"key":"6833_CR5","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1016\/j.eururo.2012.03.055","volume":"62","author":"JJMCH De La Rosette","year":"2012","unstructured":"De La Rosette JJMCH, Opondo D, Daels FPJ, Giusti G, Serrano \u00c1, Kandasami SV et al (2012) Categorisation of complications and validation of the Clavien score for percutaneous nephrolithotomy. Eur Urol 62:246\u2013255. https:\/\/doi.org\/10.1016\/j.eururo.2012.03.055","journal-title":"Eur Urol"},{"key":"6833_CR6","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1016\/j.juro.2015.09.071","volume":"195","author":"ME Dwyer","year":"2016","unstructured":"Dwyer ME, Dwyer JT, Cannon GM, Stephany HA, Schneck FX, Ost MC (2016) The Clavien-Dindo Classification of Surgical Complications is Not a Statistically Reliable System for Grading Morbidity in Pediatric Urology. J Urol 195:460\u2013464. https:\/\/doi.org\/10.1016\/j.juro.2015.09.071","journal-title":"J Urol"},{"key":"6833_CR7","first-page":"518","volume":"111","author":"PA Clavien","year":"1992","unstructured":"Clavien PA, Sanabria JR, Strasberg SM (1992) Proposed classification of complications of surgery with examples of utility in cholecystectomy. Surgery 111:518\u2013526","journal-title":"Surgery"},{"key":"6833_CR8","doi-asserted-by":"publisher","first-page":"205","DOI":"10.1097\/01.sla.0000133083.54934.ae","volume":"240","author":"D Dindo","year":"2004","unstructured":"Dindo D, Demartines N, Clavien PA (2004) Classification of surgical complications: A new proposal with evaluation in a cohort of 6336 patients and results of a survey. Ann Surg 240:205\u2013213. https:\/\/doi.org\/10.1097\/01.sla.0000133083.54934.ae","journal-title":"Ann Surg"},{"key":"6833_CR9","doi-asserted-by":"publisher","unstructured":"Silveira PSP, Siqueira JO (2023) Better to be in agreement than in bad company: A critical analysis of many kappa-like tests. Behav Res Methods 55. https:\/\/doi.org\/10.3758\/s13428-022-01950-0","DOI":"10.3758\/s13428-022-01950-0"},{"key":"6833_CR10","doi-asserted-by":"publisher","DOI":"10.32614\/CRAN.package.irrCAC","author":"KL Gwet","year":"2019","unstructured":"Gwet KL (2019) irrCAC: Computing Chance-Corrected Agreement Coefficients (CAC). CRAN: Contributed Packages. https:\/\/doi.org\/10.32614\/CRAN.package.irrCAC","journal-title":"CRAN: Contributed Packages"},{"key":"6833_CR11","doi-asserted-by":"publisher","unstructured":"Landis JR, Koch GG The measurement of observer agreement for categorical data. Biometrics 1977;33. https:\/\/doi.org\/10.2307\/2529310","DOI":"10.2307\/2529310"},{"key":"6833_CR12","unstructured":"Kohl M MKinfer: Inferential Statistics 2024"},{"key":"6833_CR13","doi-asserted-by":"publisher","first-page":"175","DOI":"10.32614\/RJ-2018-022","volume":"10","author":"O Dag","year":"2018","unstructured":"Dag O, Dolgun A, Konar NM (2018) onewaytests: An R Package for One-Way Tests in Independent Groups Designs. R J 10:175\u2013199. https:\/\/doi.org\/10.32614\/RJ-2018-022","journal-title":"R J"},{"key":"6833_CR14","doi-asserted-by":"publisher","DOI":"10.32614\/CRAN.package.weights","author":"J Pasek","year":"2025","unstructured":"Pasek J (2025) weights: Weighting and Weighted Statistics. CRAN: Contributed Packages. https:\/\/doi.org\/10.32614\/CRAN.package.weights","journal-title":"CRAN: Contributed Packages"},{"key":"6833_CR15","unstructured":"R Core Team (2025)"},{"key":"6833_CR16","doi-asserted-by":"publisher","first-page":"599","DOI":"10.1007\/s00345-024-05318-1","volume":"42","author":"A Danilovic","year":"2024","unstructured":"Danilovic A, Perrone G, Dias L, Marchini G, Torricelli F, Batagello C et al (2024) Is it worth using the Comprehensive Complication Index over the Clavien\u2013Dindo classification in elderly patients who underwent percutaneous nephrolithotomy? World J Urol 42:599. https:\/\/doi.org\/10.1007\/s00345-024-05318-1","journal-title":"World J Urol"},{"key":"6833_CR17","doi-asserted-by":"publisher","unstructured":"Vallur S, Dutta A, Arjun AP Use of Clavien\u2013Dindo classification system in assessing head and neck surgery complications. Indian J Otolaryngol Head Neck Surg 2020;72. https:\/\/doi.org\/10.1007\/s12070-019-01718-7","DOI":"10.1007\/s12070-019-01718-7"},{"key":"6833_CR18","doi-asserted-by":"publisher","unstructured":"Monteiro E, Sklar MC, Eskander A, de Almeida JR, Shrime M, Gullane P et al Assessment of the Clavien-Dindo classification system for complications in head and neck surgery. Laryngoscope 2014;124. https:\/\/doi.org\/10.1002\/lary.24817","DOI":"10.1002\/lary.24817"},{"key":"6833_CR19","doi-asserted-by":"publisher","unstructured":"Madadi-Sanjani O, Kuebler JF, Brendel J, Costanzo S, Granstr\u00f6m AL, Aydin E et al (2024) Validation of the Clavien-Madadi classification for unexpected events in pediatric surgery: a collaborative ERNICA project. J Pediatr Surg 59. https:\/\/doi.org\/10.1016\/j.jpedsurg.2024.03.022","DOI":"10.1016\/j.jpedsurg.2024.03.022"},{"key":"6833_CR20","doi-asserted-by":"publisher","unstructured":"Garc\u00eda-Garc\u00eda ML, Mart\u00edn-Lorenzo JG, Lir\u00f3n-Ruiz R, Torralba-Mart\u00ednez JA, Garc\u00eda-L\u00f3pez JA, Aguayo-Albasini JL (2017) Perioperative complications following bariatric surgery according to the clavien-dindo classification. Score validation, literature review and results in a single-centre series. Surg Obes Relat Dis 13. https:\/\/doi.org\/10.1016\/j.soard.2017.04.018","DOI":"10.1016\/j.soard.2017.04.018"},{"key":"6833_CR21","doi-asserted-by":"publisher","unstructured":"Agresti A (2010) Analysis of ordinal categorical data. Wiley. https:\/\/doi.org\/10.1002\/9780470594001","DOI":"10.1002\/9780470594001"},{"key":"6833_CR22","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1097\/SLA.0000000000006399","volume":"281","author":"SM Staubli","year":"2025","unstructured":"Staubli SM, Walker HL, Saner F, Salinas CH, Broering DC, Malag\u00f2 M et al (2025) Decoding the Clavien-Dindo Classification. Ann Surg 281:273\u2013279. https:\/\/doi.org\/10.1097\/SLA.0000000000006399","journal-title":"Ann Surg"},{"key":"6833_CR23","doi-asserted-by":"publisher","unstructured":"Hsueh JY, Nethala D, Singh S, Linehan WM, Ball MW (2024) Investigating the clinical reasoning abilities of large language model GPT-4: an analysis of postoperative complications from renal surgeries. Urologic Oncology: Seminars Original Investigations 42. https:\/\/doi.org\/10.1016\/j.urolonc.2024.04.010. :292.e1-292.e7","DOI":"10.1016\/j.urolonc.2024.04.010"},{"key":"6833_CR24","doi-asserted-by":"publisher","unstructured":"Eppler M, Ganjavi C, Ramacciotti LS, Piazza P, Rodler S, Checcucci E et al (2024) Awareness and use of ChatGPT and large language models: a prospective cross-sectional global survey in urology. Eur Urol 85. https:\/\/doi.org\/10.1016\/j.eururo.2023.10.014","DOI":"10.1016\/j.eururo.2023.10.014"},{"key":"6833_CR25","doi-asserted-by":"publisher","unstructured":"Asker OF, Recai MS, Genc YE, Dogan KA, Sener TE, Sahin B (2025) Chatbots in urology: accuracy, calibration, and comprehensibility; is DeepSeek taking over the throne? BJU Int 136. https:\/\/doi.org\/10.1111\/bju.16873","DOI":"10.1111\/bju.16873"},{"key":"6833_CR26","doi-asserted-by":"publisher","unstructured":"Suarez-Ibarrola R, Miernik A (2020) Prospects and challenges of artificial intelligence and computer science for the future of urology. World J Urol 38. https:\/\/doi.org\/10.1007\/s00345-020-03428-0","DOI":"10.1007\/s00345-020-03428-0"},{"key":"6833_CR27","doi-asserted-by":"publisher","unstructured":"Cil G, Dogan K (2024) The efficacy of artificial intelligence in urology: a detailed analysis of kidney stone-related queries. World J Urol 42. https:\/\/doi.org\/10.1007\/s00345-024-04847-z","DOI":"10.1007\/s00345-024-04847-z"},{"key":"6833_CR28","doi-asserted-by":"publisher","unstructured":"Pompili D, Richa Y, Collins P, Richards H, Hennessey DB Using artificial intelligence to generate medical literature for urology patients: a comparison of three different large language models. World J Urol 2024;42. https:\/\/doi.org\/10.1007\/s00345-024-05146-3","DOI":"10.1007\/s00345-024-05146-3"},{"key":"6833_CR29","doi-asserted-by":"publisher","unstructured":"Kollitsch L, Eredics K, Marszalek M, Rauchenwald M, Brookman-May SD, Burger M et al (2024) How does artificial intelligence master urological board examinations? A comparative analysis of different Large Language Models\u2019 accuracy and reliability in the 2022 In-Service Assessment of the European Board of Urology. World J Urol 42. https:\/\/doi.org\/10.1007\/s00345-023-04749-6","DOI":"10.1007\/s00345-023-04749-6"}],"container-title":["World Journal of Urology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00345-026-06833-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00345-026-06833-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00345-026-06833-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T12:00:55Z","timestamp":1790596855000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00345-026-06833-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,28]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["6833"],"URL":"https:\/\/doi.org\/10.1007\/s00345-026-06833-z","relation":{},"ISSN":["1433-8726"],"issn-type":[{"value":"1433-8726","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,28]]},"assertion":[{"value":"13 August 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 September 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","label":"Cnflict of interest","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was performed in line with the principles of the Declaration of Helsinki. Approval was granted by the Research Ethics Committee of\n                      Hospital das Cl\u00ednicas da Faculdade de Medicina da Universidade de S\u00e3o Paulo\n                      (91048325.0.0000.0068).","order":2,"name":"Ethics","label":"Ethical approval","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Informed consent was obtained from all individual participants included in the study.","order":3,"name":"Ethics","label":"Consent to participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"During the preparation of this work the authors used ChatGPT-5.2 (OpenAI) to improve language fluency. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.","order":4,"name":"Ethics","label":"Declaration of generative AI and AI-assisted technologies in the writing process","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"713"}}