{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T12:26:05Z","timestamp":1784204765131,"version":"3.55.0"},"reference-count":36,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n The increasing volume and complexity of biomedical literature pose challenges for making scientific knowledge accessible to lay audiences. Lay summaries, now widely encouraged or required by journals, aim to bridge this gap by promoting health literacy, patient engagement, and public trust. However, many are written by scientists without formal training in plain-language communication, often resulting in limited clarity, readability, and consistency. Generative large language models such as ChatGPT-4 offer a scalable opportunity to support lay summary creation, though their effectiveness within specific clinical domains has not been systematically evaluated at scale.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to assess ChatGPT-4\u2019s performance in generating lay summaries for prostate cancer studies. A secondary objective was to evaluate how prompt design influences summary quality, aiming to provide practical guidance for the use of generative artificial intelligence (AI) in scientific publishing.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n A total of 204 consecutive articles on prostate cancer were extracted from a high-ranking oncology journal mandating lay summaries. Each abstract was processed with ChatGPT-4 using 2 prompts: a simple prompt based on the journal\u2019s guidelines and an extended prompt refined to improve readability. AI-generated and original summaries were evaluated using 3 criteria: readability (Flesch-Kincaid Reading Ease [FKRE]), factual accuracy (5-point Likert scale, blinded rating by 2 clinical experts), and compliance with word count instructions (120\u2010150 words). Summaries were classified as high-quality as a composite outcome if they met all 3 benchmarks: FKRE >30, accuracy \u22654 from both raters, and word count within range. Statistical comparisons used Wilcoxon signed-rank and paired 2-tailed\n t<\/jats:italic>\n tests (\n P<\/jats:italic>\n <.05).\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n ChatGPT-4-generated lay summaries showed an improvement in readability compared to human-written versions, with the extended prompt achieving higher scores than the simple prompt (median FKRE: extended prompt 47, IQR 42-56; simple prompt 36, IQR 29-43; original 20, IQR 9.5\u201029;\n P<\/jats:italic>\n <.001). Factual accuracy was higher for the AI-generated lay summaries compared to originals (median factual accuracy score: extended prompt 5, IQR 5-5; simple prompt 5, IQR 5-5; original 5, IQR 4-5;\n P<\/jats:italic>\n <.001) in this dataset. Compliance with word count instructions was greater for both AI-generated summaries in comparison to originals (wrong number of words; extended prompt 39 (19%), simple prompt 40 (20%), original 140 (69%)\n ; P<\/jats:italic>\n <.001). Between simple and extended prompts, there were no significant differences in accuracy (\n P<\/jats:italic>\n =.53) and word count compliance (\n P<\/jats:italic>\n =.87). The proportion rated as high-quality was 79.4% for the extended prompt, 54.9% for the simple prompt, and 5.4% for original summaries (\n P<\/jats:italic>\n <.001).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n With optimized prompting, ChatGPT-4 produced lay summaries that, on average, scored higher than author-written versions in readability, factual accuracy, and structural compliance within our dataset. These results support integrating generative AI into editorial workflows to improve science communication for nonexpert audiences. Limitations include focus on a single clinical domain and journal, and absence of layperson evaluation.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/76598","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T20:14:54Z","timestamp":1763583294000},"page":"e76598-e76598","source":"Crossref","is-referenced-by-count":6,"title":["Using ChatGPT-4 for Lay Summarization in Prostate Cancer Research to Advance Patient-Centered Communication: Large-Scale Generative AI Performance Evaluation"],"prefix":"10.2196","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-7283-5237","authenticated-orcid":false,"given":"Emily","family":"Rinderknecht","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3810-3307","authenticated-orcid":false,"given":"Simon U","family":"Engelmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-8827-2840","authenticated-orcid":false,"given":"Veronika","family":"Saberi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8909-4211","authenticated-orcid":false,"given":"Clemens","family":"Kirschner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3365-1563","authenticated-orcid":false,"given":"Anton P","family":"Kravchuk","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1843-8492","authenticated-orcid":false,"given":"Anna","family":"Schmelzer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0674-4150","authenticated-orcid":false,"given":"Johannes","family":"Breyer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6376-7669","authenticated-orcid":false,"given":"Christopher","family":"Go\u00dfler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1796-3260","authenticated-orcid":false,"given":"Roman","family":"Mayr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0211-0636","authenticated-orcid":false,"given":"Christian","family":"Gilfrich","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5013-9435","authenticated-orcid":false,"given":"Maximilian","family":"Burger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9790-3296","authenticated-orcid":false,"given":"Dominik","family":"von Winning","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3955-564X","authenticated-orcid":false,"given":"Hendrik","family":"Borgmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0251-2654","authenticated-orcid":false,"given":"Christian","family":"W\u00fclfing","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6467-3931","authenticated-orcid":false,"given":"Axel S","family":"Merseburger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3132-0130","authenticated-orcid":false,"given":"Maximilian","family":"Haas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3896-0405","authenticated-orcid":false,"given":"Matthias","family":"May","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,11,19]]},"reference":[{"issue":"4","key":"R1","doi-asserted-by":"publisher","first-page":"474","DOI":"10.1177\/2168479017738723","article-title":"Value of developing plain language summaries of scientific and clinical articles: a survey of patients and physicians","volume":"52","author":"Pushparajah","journal-title":"Ther Innov Regul Sci"},{"issue":"4","key":"R2","doi-asserted-by":"publisher","first-page":"558","DOI":"10.1007\/s11606-017-4269-6","article-title":"Twelve lessons learned for effective research partnerships between patients, caregivers, clinicians, academic researchers, and other stakeholders","volume":"33","author":"Witteman","journal-title":"J Gen Intern Med"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","DOI":"10.1186\/s40900-019-0139-1","article-title":"Patient and public engagement in integrated knowledge translation research: Are we there yet?","volume":"5","author":"Banner","journal-title":"Res Involv Engagem"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1370\/afm.2042","article-title":"The PCORI Engagement Rubric: Promising practices for partnering in research","volume":"15","author":"Sheridan","journal-title":"Ann Fam Med"},{"issue":"5","key":"R5","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1111\/j.1369-7625.2012.00795.x","article-title":"Mapping the impact of patient and public involvement on health and social care research: a systematic review","volume":"17","author":"Brett","journal-title":"Health Expect"},{"issue":"4","key":"R6","doi-asserted-by":"publisher","first-page":"317","DOI":"10.4415\/ANN_14_04_04","article-title":"Regulation (EU) No 536\/2014 on clinical trials on medicinal products for human use: an overview","volume":"50","author":"Petrini","journal-title":"Ann Ist Super Sanita"},{"key":"R7","unstructured":"Summaries of clinical trial results for laypersons. recommendations of the expert group on clinical trials for the implementation of regulation (EU) no 536\/2014 on clinical trials on medicinal products for human use. European Commission; 2021. URL: https:\/\/health.ec.europa.eu\/system\/files\/2020-02\/2017_01_26_summaries_of_ct_results_for_laypersons_0.pdf [Accessed 06-11-2025]"},{"key":"R8","unstructured":"Cancers: instructions for authors. Multidisciplinary Digital Publishing Institute. URL: https:\/\/www.mdpi.com\/journal\/cancers\/instructions [Accessed 19-02-2025]"},{"key":"R9","unstructured":"Guide for authors. European Urology. URL: https:\/\/www.europeanurology.com\/guide-for-authors [Accessed 19-02-2025]"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1186\/s40900-017-0064-0","article-title":"Understanding Plain English summaries. A comparison of two approaches to improve the quality of Plain English summaries in research reports","volume":"3","author":"Kirkpatrick","journal-title":"Res Involv Engagem"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1007\/s40271-022-00606-7","article-title":"What author instructions do health journals provide for writing plain language summaries? A scoping review","volume":"16","author":"Gainey","journal-title":"Patient"},{"issue":"1","key":"R12","doi-asserted-by":"publisher","DOI":"10.1186\/s12910-016-0126-0","article-title":"Readability of patient information and consent documents in rheumatological studies","volume":"17","author":"Hamnes","journal-title":"BMC Med Ethics"},{"issue":"6","key":"R13","doi-asserted-by":"publisher","first-page":"1068","DOI":"10.1016\/j.euf.2023.06.003","article-title":"Clinical patient summaries not fit for purpose: a study in urology","volume":"9","author":"Ganjavi","journal-title":"Eur Urol Focus"},{"issue":"105-112","key":"R14","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.jclinepi.2023.02.023","article-title":"Trial lay summaries were not fit for purpose","volume":"156","author":"Shiely","journal-title":"J Clin Epidemiol"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","first-page":"67","DOI":"10.7812\/TPP\/07-144","article-title":"Do patients understand?","volume":"12","author":"Graham","journal-title":"Perm J"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Goldsack T Scarton C Shardlow M Lin C . Overview of the biolaysumm 2024 shared task on the lay summarization of biomedical research articles. Presented at: Proceedings of the 23rd Workshop on Biomedical Natural Language Processing; Aug 16, 2024; Bangkok, Thailand. URL: https:\/\/aclanthology.org\/2024.bionlp-1 [Accessed 06-11-2025] [doi: 10.18653\/v1\/2024.bionlp-1.10]","DOI":"10.18653\/v1\/2024.bionlp-1.10"},{"issue":"10","key":"R17","doi-asserted-by":"publisher","first-page":"2294","DOI":"10.1093\/jamia\/ocae186","article-title":"Leveraging artificial intelligence to summarize abstracts in lay language for increasing research accessibility and transparency","volume":"31","author":"Shyr","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"R18","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1097\/UPJ.0000000000000428","article-title":"Bridging the gap between urological research and patient understanding: the role of large language models in automated generation of layperson\u2019s summaries","volume":"10","author":"Eppler","journal-title":"Urol Pract"},{"issue":"2","key":"R19","doi-asserted-by":"publisher","DOI":"10.3390\/curroncol32020102","article-title":"Leveraging large language models for high-quality lay summaries: efficacy of ChatGPT-4 with custom prompts in a consecutive series of prostate cancer manuscripts","volume":"32","author":"Rinderknecht","journal-title":"Curr Oncol"},{"issue":"1","key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-024-02838-z","article-title":"Enhancing doctor-patient communication using large language models for pathology report interpretation","volume":"25","author":"Yang","journal-title":"BMC Med Inform Decis Mak"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.2196\/63786","article-title":"Enhancing physician-patient communication in oncology using GPT-4 through simplified radiology reports: multicenter quantitative study","volume":"27","author":"Yang","journal-title":"J Med Internet Res"},{"issue":"9","key":"R22","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae387","article-title":"From complexity to clarity: How AI enhances perceptions of scientists and the public\u2019s understanding of science","volume":"3","author":"Markowitz","journal-title":"PNAS Nexus"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.2196\/63347","article-title":"Using ChatGPT to improve the presentation of plain language summaries of Cochrane systematic reviews about oncology interventions: cross-sectional study","volume":"11","author":"\u0160uto Pavi\u010di\u0107","journal-title":"JMIR Cancer"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.1136\/bmj-2022-070904","article-title":"Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI","volume":"377","author":"Vasey","journal-title":"BMJ"},{"key":"R25","unstructured":"Readability test. WebFX. 2025. URL: https:\/\/www.webfx.com\/tools\/read-able\/ [Accessed 31-03-2025]"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","journal-title":"J Appl Psychol"},{"key":"R27","unstructured":"DuBay WH . The principles of readability. Education Resources Information Center. 2004. URL: https:\/\/eric.ed.gov\/?id=ed490073 [Accessed 06-11-2025]"},{"key":"R28","unstructured":"Brega AG Barnard J Mabachi NM et al. AHRQ Health Literacy Universal Precautions Toolkit. 2nd ed. Agency for Healthcare Research and Quality; 2015. URL: https:\/\/www.ahrq.gov\/sites\/default\/files\/publications\/files\/healthlittoolkit2_3.pdf [Accessed 06-11-2025]"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"World Health Organization. Global Strategy on Digital Health 2020-2025. 1st ed. World Health Organization; 2021. [doi: 10.1007\/978-3-030-05325-3_125-1] ISBN: 9789240020924","DOI":"10.1007\/978-3-030-05325-3_125-1"},{"issue":"277-279","key":"R30","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1016\/j.ijcard.2019.05.070","article-title":"Best practices in digital health literacy","volume":"292","author":"Conard","journal-title":"Int J Cardiol"},{"key":"R31","doi-asserted-by":"crossref","unstructured":"Clark E August T Serrano S Haduong N Gururangan S Smith NA . All that\u2019s \u2018human\u2019 is not gold: evaluating human evaluation of generated text. Presented at: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1. 2021.URL: https:\/\/aclanthology.org\/2021.acl-long [doi: 10.18653\/v1\/2021.acl-long.565]","DOI":"10.18653\/v1\/2021.acl-long.565"},{"issue":"11","key":"R32","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2208839120","article-title":"Human heuristics for AI-generated language are flawed","volume":"120","author":"Jakesch","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1162\/coli_a_00549","article-title":"A survey on LLM-generated text detection: necessity, methods, and future directions","volume":"51","author":"Wu","journal-title":"Comput Linguist Assoc Comput Linguist"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.21428\/e4baedd9.0319e3a6","article-title":"Labeling AI-generated content: Promises, perils, and future directions","author":"Wittenberg","journal-title":"MIT Explor Gener AI"},{"issue":"6","key":"R35","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgaf170","article-title":"Labeling AI-generated media online","volume":"4","author":"Wittenberg","journal-title":"PNAS Nexus"},{"issue":"10","key":"R36","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae403","article-title":"People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automation","volume":"3","author":"Altay","journal-title":"PNAS Nexus"}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2025\/1\/e76598","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:08:14Z","timestamp":1782842894000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2025\/1\/e76598"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,19]]},"references-count":36,"URL":"https:\/\/doi.org\/10.2196\/76598","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,19]]},"article-number":"v27i4e76598"}}