{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,28]],"date-time":"2026-10-28T02:49:27Z","timestamp":1793155767963,"version":"4.5.0"},"reference-count":31,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T00:00:00Z","timestamp":1760832000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Manfred Wirth grant for international research of the Deutsche Gesellschaft f\u00fcr Urologie","award":["CaN1\/MW-25"],"award-info":[{"award-number":["CaN1\/MW-25"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Current Oncology"],"abstract":"<jats:p>Large language models (LLMs) are increasingly explored as chatbots for patient education, including applications in urooncology. Since only 12% of adults have proficient health literacy and most patient information materials exceed recommended reading levels, improving readability is crucial. Although LLMs could potentially increase the readability of medical information, evidence is mixed, underscoring the need to assess chatbot outputs in clinical settings. Therefore, this study evaluates the measured and perceived readability of chatbot responses in speech-based interactions with urological patients. Urological patients engaged in unscripted conversations with a GPT-4-based chatbot. Transcripts were analyzed using three readability indices: Flesch\u2013Reading-Ease (FRE), Lesbarkeitsindex (LIX) and Wiener-Sachtextformel (WSF). Perceived readability was assessed using a survey covering technical language, clarity and explainability. Associations between measured and perceived readability were analyzed. Knowledge retention was not assessed in this study. A total of 231 conversations were evaluated. The most frequently addressed topics were prostate cancer (22.5%), robotic-assisted prostatectomy (19.9%) and follow-up (18.6%). Objectively, responses were classified as difficult to read (FRE 43.1 \u00b1 9.1; LIX 52.8 \u00b1 6.2; WSF 11.2 \u00b1 1.6). In contrast, perceived readability was rated highly for technical language, clarity and explainability (83\u201390%). Correlation analyses revealed no association between objective and perceived readability. Chatbot responses were objectively written at a difficult reading level, exceeding recommendations for optimized health literacy. Nevertheless, most patients perceived the information as clear and understandable. This discrepancy suggests that perceived comprehensibility is influenced by factors beyond measurable linguistic complexity.<\/jats:p>","DOI":"10.3390\/curroncol32100582","type":"journal-article","created":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T12:27:29Z","timestamp":1760963249000},"page":"582","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Readability of Chatbot Responses in Prostate Cancer and Urological Care: Objective Metrics Versus Patient Perceptions"],"prefix":"10.3390","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5163-3383","authenticated-orcid":false,"given":"Lasse","family":"Maywald","sequence":"first","affiliation":[{"name":"Department of Urology, University Medical Center Mannheim, 68167 Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lisa","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Medical Faculty Mannheim, Ruprecht-Karls University of Heidelberg, 68167 Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3346-9537","authenticated-orcid":false,"given":"Jana Theres","family":"Winterstein","sequence":"additional","affiliation":[{"name":"Division of Digital Prevention, Diagnostics and Therapy Guidance, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany"},{"name":"Medical Faculty, Ruprecht-Karls University of Heidelberg, 69120 Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin Joachim","family":"Hetz","sequence":"additional","affiliation":[{"name":"Division of Digital Prevention, Diagnostics and Therapy Guidance, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany"},{"name":"Medical Faculty, Ruprecht-Karls University of Heidelberg, 69120 Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maurin Helen","family":"Mangold","sequence":"additional","affiliation":[{"name":"Department of Urology, University Medical Center Mannheim, 68167 Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4485-6586","authenticated-orcid":false,"given":"Luisa Vivienne","family":"Renner","sequence":"additional","affiliation":[{"name":"Department of Urology, University Medical Center Mannheim, 68167 Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Titus Josef","family":"Brinker","sequence":"additional","affiliation":[{"name":"Division of Digital Prevention, Diagnostics and Therapy Guidance, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Frederik","family":"Wessels","sequence":"additional","affiliation":[{"name":"Department of Urology, University Medical Center Mannheim, 68167 Mannheim, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0080-3455","authenticated-orcid":false,"given":"Nicolas","family":"Carl","sequence":"additional","affiliation":[{"name":"Department of Urology, University Medical Center Mannheim, 68167 Mannheim, Germany"},{"name":"Division of Digital Prevention, Diagnostics and Therapy Guidance, German Cancer Research Center (DKFZ), 69120 Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,19]]},"reference":[{"key":"ref_1","unstructured":"Kutner, M., Greenberg, E., Jin, Y., and Paulsen, C. (2006). The Health Literacy of America\u2019s Adults: Results from the 2003 National Assessment of Adult Literacy. NCES 2006-483, US Department of Education, National Center for Education Statistics."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1016\/j.euf.2024.06.012","article-title":"Readability Assessment of Patient Education Materials on Uro-oncological Diseases Using Automated Measures","volume":"10","author":"Rodler","year":"2024","journal-title":"Eur. Urol. Focus."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1038\/s41698-024-00733-4","article-title":"Large language model use in clinical oncology","volume":"8","author":"Carl","year":"2024","journal-title":"npj Precis. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","year":"1948","journal-title":"J. Appl. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1016\/j.urology.2014.10.035","article-title":"Readability of American online patient education materials in urologic oncology: A need for simple communication","volume":"85","author":"Pruthi","year":"2015","journal-title":"Urology"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sallam, M. (2023). ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare, 11.","DOI":"10.3390\/healthcare11060887"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1016\/j.euf.2019.10.011","article-title":"Asking \u201cDr. Google\u201d for a Second Opinion: The Devil Is in the Details","volume":"7","author":"Cacciamani","year":"2021","journal-title":"Eur. Urol. Focus."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Swoboda, C.M., Van Hulle, J.M., McAlearney, A.S., and Huerta, T.R. (2018). Odds of talking to healthcare providers as the initial source of healthcare information: Updated cross-sectional results from the Health Information National Trends Survey (HINTS). BMC Fam. Pract., 19.","DOI":"10.1186\/s12875-018-0805-7"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1111\/bju.16676","article-title":"Evaluating interactions of patients with large language models for medical information","volume":"135","author":"Carl","year":"2025","journal-title":"BJU Int."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1097\/JU.0000000000003615","article-title":"Evaluating the Effectiveness of Artificial Intelligence-powered Large Language Models Application in Disseminating Appropriate and Readable Health Information in Urology","volume":"210","author":"Davis","year":"2023","journal-title":"J. Urol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1038\/s41391-024-00826-y","article-title":"Accuracy, readability, and understandability of large language models for prostate cancer information to the public","volume":"28","author":"Hershenhouse","year":"2025","journal-title":"Prostate Cancer Prostatic Dis."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e2500042","DOI":"10.1200\/CCI-25-00042","article-title":"Enhancing Readability of Lay Abstracts and Summaries for Urologic Oncology Literature Using Generative Artificial Intelligence: BRIDGE-AI 6 Randomized Controlled Trial","volume":"9","author":"Ganjavi","year":"2025","journal-title":"JCO Clin. Cancer Inform."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rodler, S., Cei, F., Ganjavi, C., Checcucci, E., De Backer, P., Rivero Belenchon, I., Taratkin, M., Puliatti, S., Veccia, A., and Piazza, P. (2025). GPT-4 generates accurate and readable patient education materials aligned with current oncological guidelines: A randomized assessment. PLoS ONE, 20.","DOI":"10.1371\/journal.pone.0324175"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1007\/s00345-025-05872-2","article-title":"Patient insights into empathy, compassion and self-disclosure in medical large language models: Results from the IPALLM III study","volume":"43","author":"Carl","year":"2025","journal-title":"World J. Urol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.euros.2024.10.009","article-title":"Comparing Patient\u2019s Confidence in Clinical Capabilities in Urology: Large Language Models Versus Urologists","volume":"70","author":"Carl","year":"2024","journal-title":"Eur. Urol. Open Sci."},{"key":"ref_16","unstructured":"Bj\u00f6rnsson, C.H. (1968). L\u00e4sbarhet, Liber; Seelig."},{"key":"ref_17","unstructured":"Bamberger, R., and Vanecek, E. (1984). Lesen-Verstehen-Lernen-Schreiben: Die Schwierigkeitsstufen von Texten in Deutscher Sprache, Jugend und Volk."},{"key":"ref_18","unstructured":"Amstad, T. (1978). Wie Verst\u00e4ndlich Sind Unsere Zeitungen?, Studenten-Schreib-Service."},{"key":"ref_19","first-page":"133","article-title":"Does Reading While Listening to Text Improve Comprehension Compared to Reading Only? A Systematic Review and Meta-Analysis","volume":"34","year":"2023","journal-title":"Educ. Res. Theory Pract."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1037\/0022-3514.77.6.1121","article-title":"Unskilled and unaware of it: How difficulties in recognizing one\u2019s own incompetence lead to inflated self-assessments","volume":"77","author":"Kruger","year":"1999","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"460","DOI":"10.1007\/s10880-022-09895-4","article-title":"Overconfidence in Managing Health Concerns: The Dunning-Kruger Effect and Health Literacy","volume":"30","author":"Canady","year":"2023","journal-title":"J. Clin. Psychol. Med. Settings"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1097\/01.NOR.0000345852.22122.d6","article-title":"Understanding the health literacy of America: Results of the National Assessment of Adult Literacy","volume":"28","author":"Cutilli","year":"2009","journal-title":"Orthop. Nurs."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e002928","DOI":"10.1136\/bmjopen-2013-002928","article-title":"Accuracy in self-reported health literacy screening: A difference between men and women in Taiwan","volume":"3","author":"Lee","year":"2013","journal-title":"BMJ Open"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1097\/00005650-197804000-00005","article-title":"Effects of acquiescent response set on patient satisfaction ratings","volume":"16","author":"Ware","year":"1978","journal-title":"Med. Care"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e000694","DOI":"10.1136\/bmjgh-2017-000694","article-title":"Bias in patient satisfaction surveys: A threat to measuring healthcare quality","volume":"3","author":"Dunsch","year":"2018","journal-title":"BMJ Glob. Health"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"171","DOI":"10.2147\/PROM.S178344","article-title":"Truthfulness in patient-reported outcomes: Factors affecting patients\u2019 responses and impact on data quality","volume":"10","author":"Chang","year":"2019","journal-title":"Patient Relat. Outcome Meas."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Sahin, S., Erkmen, B., Duymaz, Y.K., Bayram, F., Tekin, A.M., and Topsakal, V. (2024). Evaluating ChatGPT-4\u2019s performance as a digital health advisor for otosclerosis surgery. Front. Surg., 11.","DOI":"10.3389\/fsurg.2024.1373843"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e42135","DOI":"10.1097\/MD.0000000000042135","article-title":"Assessing the quality and readability of patient education materials on chemotherapy cardiotoxicity from artificial intelligence chatbots: An observational cross-sectional study","volume":"104","author":"Behers","year":"2025","journal-title":"Medicine"},{"key":"ref_29","first-page":"31","article-title":"ChatGPT: Is This Patient Education Tool for Urological Malignancies Readable for the General Population?","volume":"16","author":"Thia","year":"2024","journal-title":"Res. Rep. Urol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e68817","DOI":"10.2196\/68817","article-title":"Improving the Readability of Institutional Heart Failure-Related Patient Education Materials Using GPT-4: Observational Study","volume":"9","author":"King","year":"2025","journal-title":"JMIR Cardio"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1186\/s13690-024-01428-9","article-title":"The level of electronic health literacy among older adults: A systematic review and meta-analysis","volume":"82","author":"Jiang","year":"2024","journal-title":"Arch. Public Health"}],"container-title":["Current Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1718-7729\/32\/10\/582\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T13:28:49Z","timestamp":1760966929000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1718-7729\/32\/10\/582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,19]]},"references-count":31,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["curroncol32100582"],"URL":"https:\/\/doi.org\/10.3390\/curroncol32100582","relation":{},"ISSN":["1718-7729"],"issn-type":[{"value":"1718-7729","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,19]]}}}