{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T16:12:07Z","timestamp":1791303127395,"version":"4.1.0"},"reference-count":23,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T00:00:00Z","timestamp":1762128000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background\/Objectives: Artificial Intelligence (AI)-based chatbots such as ChatGPT are easily available and are quickly becoming a source of information for patients as opposed to traditional Google searches. We assessed the quality of information on bladder cancer, provided by various AI chatbots such as ChatGPT 4o, Google Gemini 2.0 flash, Grok 3, Claude Sonnet 3.7 and DeepSeek R1. Their responses were analysed in terms of Readability Indices, and two consultant urologists rated the quality of information provided using the validated DISCERN tool. Methods: The top 10 most frequently asked questions about bladder cancer were identified using Google Trends. These questions were then provided to five different AI chatbots, and their responses were collected. No prompts were used, reflecting natural language queries that patients would use. The responses were analysed in terms of their readability using five validated indices: Flesch Reading Ease (FRE), the Flesch\u2013Kincaid Reading Grade Level (FKRGL), the Gunning Fog Index, the Coleman\u2013Liau Index and the SMOG index. Two consultant urologists then independently assessed the responses of various AI chatbots using the DISCERN tool, which rates the quality of the health information on a five-point LIKERT scale. Inter-rater agreement was calculated using Cohen\u2019s Kappa and the intraclass correlation coefficient (ICC). Results: ChatGPT 4o was the overall winner in readability scores, with the highest Flesch Reading Ease score (59.4) and the lowest average reading grade level (7.0) required to understand the material. Grok 3 was a close second (FRE 58.3, grade level 8.7). Claude 3.7 Sonnet used the most complex language in its answers and therefore scored the lowest FRE score of 44.9, with the highest grade level (9.5) and also the highest complexity on other indices. In the DISCERN analysis, Grok 3 received the highest average score (52.0), followed closely by ChatGPT 4o (50.5). The inter-rater agreement was highest for ChatGPT 4o (ICC: 0.791; Kappa: 0.437), while it was lowest for Grok 3 (ICC: 0.339, Kappa 0.0, Weighted Kappa 0.335). Conclusions: All AI chatbots can provide generally good-quality answers to questions about bladder cancer with zero hallucinations. ChatGPT 4o was the overall winner, with the best readability metrics, strong DISCERN ratings and highest inter-rater agreement.<\/jats:p>","DOI":"10.3390\/jcm14217804","type":"journal-article","created":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T19:30:27Z","timestamp":1762198227000},"page":"7804","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Evaluating the Readability and Quality of Bladder Cancer Information from AI Chatbots: A Comparative Study Between ChatGPT, Google Gemini, Grok, Claude and DeepSeek"],"prefix":"10.3390","volume":"14","author":[{"given":"Kunjan","family":"Patel","sequence":"first","affiliation":[{"name":"Royal Derby Hospital, Derby DE22 3NE, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1302-5146","authenticated-orcid":false,"given":"Robert","family":"Radcliffe","sequence":"additional","affiliation":[{"name":"Royal Derby Hospital, Derby DE22 3NE, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.euf.2023.06.012","article-title":"Long-Term Recurrence Rates of Low-Risk Non-Muscle-Invasive Bladder Cancer-How Long Is Cystoscopic Surveillance Necessary?","volume":"10","author":"Ma","year":"2024","journal-title":"Eur. Urol. Focus"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Grabe-Heyne, K., Henne, C., Mariappan, P., Geiges, G., P\u00f6hlmann, J., and Pollock, R.F. (2023). Intermediate and high-risk non-muscle-invasive bladder cancer: An overview of epidemiology, burden, and unmet needs. Front. Oncol., 13.","DOI":"10.3389\/fonc.2023.1170124"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e70345","DOI":"10.1002\/cam4.70345","article-title":"Psychological Distress in Bladder Cancer Patients: A Systematic Review","volume":"13","author":"Thomas","year":"2024","journal-title":"Cancer Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1016\/j.eururo.2023.04.034","article-title":"Patient and Carer Experiences with Bladder Cancer: Results from a Global Survey in 45 Countries","volume":"84","author":"Makaroff","year":"2023","journal-title":"Eur. Urol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1016\/j.euo.2019.07.002","article-title":"Consulting \u201cDr. Google\u201d for prostate cancer treatment options: A contemporary worldwide trend analysis","volume":"3","author":"Cacciamani","year":"2020","journal-title":"Eur. Urol. Oncol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1016\/j.euf.2019.10.011","article-title":"Asking \u201cDr. Google\u201d for a second opinion: The devil is in the details","volume":"7","author":"Cacciamani","year":"2021","journal-title":"Eur. Urol. Focus."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1016\/S1470-2045(20)30138-8","article-title":"Web search queries and prostate cancer","volume":"21","author":"Cacciamani","year":"2020","journal-title":"Lancet Oncol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1097\/JU.0000000000003615","article-title":"Evaluating the effectiveness of artificial intelligence-powered large language models application in disseminating appropriate and readable health information in urology","volume":"210","author":"Davis","year":"2023","journal-title":"J. Urol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e54345","DOI":"10.2196\/54345","article-title":"Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study","volume":"12","author":"Aljamaan","year":"2024","journal-title":"JMIR Med. Inform."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e2412767","DOI":"10.1001\/jamanetworkopen.2024.12767","article-title":"Use of Artificial Intelligence Chatbots in Interpretation of Pathology Reports","volume":"7","author":"Steimetz","year":"2024","journal-title":"JAMA Netw. Open."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1111\/bju.16042","article-title":"Potential safety concerns in use of the artificial intelligence chatbot \u2018ChatGPT\u2019 for perioperative patient communication","volume":"132","author":"Tung","year":"2023","journal-title":"BJU Int."},{"key":"ref_12","first-page":"e37281","article-title":"Overview of early ChatGPT\u2019s presence in medical literature: Insights from a hybrid literature review by ChatGPT and human experts","volume":"15","author":"Temsah","year":"2023","journal-title":"Cureus"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1097\/UPJ.0000000000000428","article-title":"Bridging the gap between urological research and patient understanding: The role of large language models in automated generation of layperson\u2019s summaries","volume":"10","author":"Eppler","year":"2023","journal-title":"Urol. Pract."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1038\/s41391-023-00705-y","article-title":"Quality of information and appropriateness of ChatGPT outputs for urology patients","volume":"27","author":"Cocci","year":"2024","journal-title":"Prostate Cancer Prostatic Dis."},{"key":"ref_15","unstructured":"(2025, May 05). Available online: https:\/\/www.ndph.ox.ac.uk\/research\/research-groups\/applied-health-research-unit-ahru\/discern."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1038\/d41586-023-00340-6","article-title":"What ChatGPT and generative AI mean for science","volume":"614","year":"2023","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e48039","DOI":"10.2196\/48039","article-title":"Performance of ChatGPT on the Peruvian National Licensing Medical Examination: Cross-Sectional Study","volume":"9","year":"2023","journal-title":"JMIR Med. Educ."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e45312","DOI":"10.2196\/45312","article-title":"How Does ChatGPT Perform on the United States Medical Licensing Examination? The Implications of Large Language Models for Medical Education and Knowledge Assessment","volume":"9","author":"Gilson","year":"2023","journal-title":"JMIR Med. Educ."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"NP1085","DOI":"10.1093\/asj\/sjad130","article-title":"ChatGPT Is Equivalent to First-Year Plastic Surgery Residents: Evaluation of ChatGPT on the Plastic Surgery In-Service Examination","volume":"43","author":"Humar","year":"2023","journal-title":"Aesthet. Surg. J."},{"key":"ref_20","first-page":"e85","article-title":"The Rapid Development of Artificial Intelligence: GPT-4\u2019s Performance on Orthopedic Surgery Board Questions","volume":"47","author":"Hofmann","year":"2023","journal-title":"Orthopedics"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"101688","DOI":"10.1016\/j.hansur.2024.101688","article-title":"ChatGPT Earns American Board Certification in Hand Surgery","volume":"43","author":"Ghanem","year":"2024","journal-title":"Hand Surg. Rehabil."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1177\/20514158241262081","article-title":"ChatGPT sitting for FRCS Urology examination: Will artificial intelligence get certified?","volume":"18","author":"Desouky","year":"2024","journal-title":"J. Clin. Urol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1186\/s12967-023-04123-5","article-title":"Can the ChatGPT and other large language models with internet-connected database solve the questions and concerns of patient with prostate cancer and help democratize medical knowledge?","volume":"21","author":"Zhu","year":"2023","journal-title":"J. Transl. Med."}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/21\/7804\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T19:55:33Z","timestamp":1762199733000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/21\/7804"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,3]]},"references-count":23,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["jcm14217804"],"URL":"https:\/\/doi.org\/10.3390\/jcm14217804","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,3]]}}}