{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T18:03:49Z","timestamp":1789668229960,"version":"4.0.1"},"reference-count":40,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T00:00:00Z","timestamp":1789603200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ontario Graduate Scholarship"}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>\n                    Radiology reports are often filled with medical jargon that limits patient understanding. Lay summaries can improve understanding but are time-consuming for healthcare providers to create. The objective of this study is to explore the use of tailored prompts for five Large Language Models (LLMs) in generating lay summaries from radiology reports. Using 100 reports from the publicly available \u201cBioNLP 2023 report summarization\u201d dataset, lay summaries were generated by each LLM, under select prompting styles [\n                    <jats:italic>Few-Shot<\/jats:italic>\n                    (GPT-4),\n                    <jats:italic>Generated Knowledge<\/jats:italic>\n                    (GPT-4o mini, Gemini 1.5 \u2013 Pro, Gemini 1.5 \u2013 Flash), and\n                    <jats:italic>Zero-Shot<\/jats:italic>\n                    (Llama 3.1)] informed by a pilot work. The summaries were evaluated using a mixed-method framework: subjective assessment (Likert statements) by blinded experts (n\u2009=\u20092 radiology fellows) and Large Reasoning Models (LRMs) [(Gemini 2.5 \u2013 Pro (LRM 1); GPT-oss-120b (LRM 2)], and readability metrics (Flesch-Kincaid Grade Level and Flesch Reading Ease). Using percentage agreement of Likert statements, the LLM-prompt combinations\u2019 performances were ranked, and Friedman and post-hoc Nemenyi tests were conducted. Gemini 1.5 - Flash and - Pro\n                    <jats:italic>(generated knowledge)<\/jats:italic>\n                    were rated highest by human experts and LRMs for generating actionable lay summaries that require minimal supervision [P\u2009&lt;\u20094.97\u2009\u00d7\u200910\n                    <jats:sup>-2<\/jats:sup>\n                    (Rater 1); P\u2009&lt;\u20099.03\u2009\u00d7\u200910\n                    <jats:sup>-21<\/jats:sup>\n                    (Rater 2), P\u2009&lt;\u20096.90\u2009\u00d7\u200910\n                    <jats:sup>-15<\/jats:sup>\n                    (LRM 1),\n                    <jats:italic>P<\/jats:italic>\n                    \u2009&lt;\u20092.760\u2009\u00d7\u200910\n                    <jats:sup>-5<\/jats:sup>\n                    (LRM 2). GPT-4 (\n                    <jats:italic>few-shot<\/jats:italic>\n                    ) achieved the highest human-rated accuracy (98%), while Gemini 1.5 \u2013 Flash (LRM 1-rated: 95%) and Gemini 1.5 \u2013 Pro (LRM 2-rated: 91%) ranked first in LRM-rated accuracy. Gemini 1.5 - Pro produced the most accessible summaries (Flesch-Kincaid Grade Level: 7.55\u2009\u00b1\u20091.38, Flesch Reading Ease: 67.84\u2009\u00b1\u20097.78). Strong agreement was observed between experts and LRMs [0.96% (LRM 1) and 3.4% (LRM 2) complete disagreement]. Overall, this study highlights Gemini-models with\n                    <jats:italic>generated knowledge<\/jats:italic>\n                    prompts and the potential of LRM evaluators in assessing LLM-generated lay summaries.\n                  <\/jats:p>","DOI":"10.1371\/journal.pdig.0001672","type":"journal-article","created":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:21:06Z","timestamp":1789665666000},"page":"e0001672","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating large language models for lay summaries of radiology reports using tailored prompting strategies and mixed-method assessment"],"prefix":"10.1371","volume":"5","author":[{"given":"Nanziba","family":"Tasneem","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian B.","family":"van der Pol","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ambreen","family":"Zahoor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nitin","family":"Juggath","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-4794-3959","authenticated-orcid":true,"given":"Kyle","family":"McGowan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cynthia","family":"Lokker","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7650-1720","authenticated-orcid":true,"given":"Ashirbani","family":"Saha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,17]]},"reference":[{"issue":"11","key":"pdig.0001672.ref001","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2022.42972","article-title":"Accuracy in Patient Understanding of Common Medical Phrases","volume":"5","author":"R Gotlieb","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"3","key":"pdig.0001672.ref002","doi-asserted-by":"crossref","first-page":"407","DOI":"10.59249\/NKOY5498","article-title":"Artificial Intelligence to Improve Patient Understanding of Radiology Reports","volume":"96","author":"K Amin","year":"2023","journal-title":"Yale J Biol Med"},{"issue":"3","key":"pdig.0001672.ref003","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2024.0357","article-title":"Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format","volume":"7","author":"J Zaretsky","year":"2024","journal-title":"JAMA Netw Open"},{"key":"pdig.0001672.ref004","doi-asserted-by":"crossref","first-page":"100530","DOI":"10.1016\/j.ejro.2023.100530","article-title":"Incidence and factors associated with burnout in radiologists: A systematic review","volume":"11","author":"N Ashraf","year":"2023","journal-title":"Eur J Radiol Open"},{"issue":"1","key":"pdig.0001672.ref005","doi-asserted-by":"crossref","DOI":"10.1148\/radiol.240844","article-title":"Large Language Models as an Inexpensive and Effective Extra Set of Eyes in Radiology Reporting","volume":"311","author":"HP Forman","year":"2024","journal-title":"Radiology"},{"key":"pdig.0001672.ref006","author":"Google Cloud"},{"key":"pdig.0001672.ref007","doi-asserted-by":"crossref","unstructured":"Kojima T, Gu S (Shane), Reid M, Matsuo Y, Iwasawa Y. Large Language Models Are Zero-Shot Reasoners. In: Advances in Neural Information Processing Systems 35, 2022. 22199\u2013213. https:\/\/doi.org\/10.52202\/068431-1613","DOI":"10.52202\/068431-1613"},{"issue":"6","key":"pdig.0001672.ref008","doi-asserted-by":"crossref","first-page":"101260","DOI":"10.1016\/j.patter.2025.101260","article-title":"Unleashing the potential of prompt engineering for large language models","volume":"6","author":"B Chen","year":"2025","journal-title":"Patterns (N Y)"},{"issue":"2","key":"pdig.0001672.ref009","article-title":"Accuracy of ChatGPT, Google Bard, and Microsoft Bing for Simplifying Radiology Reports","volume":"309","author":"KS Amin","year":"2023","journal-title":"Radiology"},{"issue":"2","key":"pdig.0001672.ref010","doi-asserted-by":"crossref","first-page":"888","DOI":"10.1016\/j.acra.2024.09.041","article-title":"Large Language Models for Simplified Interventional Radiology Reports: A Comparative Analysis","volume":"32","author":"E Can","year":"2025","journal-title":"Acad Radiol"},{"issue":"3","key":"pdig.0001672.ref011","doi-asserted-by":"crossref","DOI":"10.1148\/radiol.231593","article-title":"Quantitative Evaluation of Large Language Models to Streamline Radiology Report Impressions: A Multimodal Retrospective Analysis","volume":"310","author":"R Doshi","year":"2024","journal-title":"Radiology"},{"issue":"5","key":"pdig.0001672.ref012","doi-asserted-by":"crossref","first-page":"2817","DOI":"10.1007\/s00330-023-10213-1","article-title":"ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports","volume":"34","author":"K Jeblick","year":"2024","journal-title":"Eur Radiol"},{"key":"pdig.0001672.ref013","doi-asserted-by":"crossref","DOI":"10.1177\/23743735241259477","article-title":"Patient-Readable Radiology Report Summaries Generated via Large Language Model: Safety and Quality","volume":"11","author":"NW Sterling","year":"2024","journal-title":"Journal of Patient Experience"},{"key":"pdig.0001672.ref014","article-title":"Prompt Engineering in Clinical Practice: Tutorial for Clinicians","volume":"27","author":"J Liu","year":"2025","journal-title":"J Med Internet Res"},{"key":"pdig.0001672.ref015","doi-asserted-by":"crossref","DOI":"10.2196\/55318","article-title":"An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study","volume":"12","author":"S Sivarajkumar","year":"2024","journal-title":"JMIR Med Inform"},{"key":"pdig.0001672.ref016","article-title":"On the worst prompt performance of large language models","author":"B Cao","year":"2024","journal-title":"arXiv"},{"key":"pdig.0001672.ref017","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1162\/tacl_a_00681","article-title":"State of what art? A call for multi-prompt LLM evaluation","volume":"12","author":"M Mizrahi","year":"2024","journal-title":"Trans Assoc Comput Linguist"},{"issue":"10","key":"pdig.0001672.ref018","doi-asserted-by":"crossref","first-page":"101370","DOI":"10.1016\/j.patter.2025.101370","article-title":"Toward large reasoning models: A survey of reinforced reasoning with large language models","volume":"6","author":"F Xu","year":"2025","journal-title":"Patterns (N Y)"},{"issue":"1","key":"pdig.0001672.ref019","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1038\/s41597-019-0322-0","article-title":"MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports","volume":"6","author":"AEW Johnson","year":"2019","journal-title":"Sci Data"},{"key":"pdig.0001672.ref020","author":"OpenAI"},{"key":"pdig.0001672.ref021","author":"Google Cloud","year":"2025"},{"key":"pdig.0001672.ref022","author":"Hugging F, Face"},{"key":"pdig.0001672.ref023"},{"key":"pdig.0001672.ref024","author":"DeepMind","year":"2025"},{"key":"pdig.0001672.ref025","author":"AI Vals","year":"2025"},{"key":"pdig.0001672.ref026","year":"2025"},{"key":"pdig.0001672.ref027","article-title":"GPT-OSS-120B & GPT-OSS-20B model card","author":"S Agarwal","year":"2025","journal-title":"arXiv"},{"issue":"2","key":"pdig.0001672.ref028","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.imed.2025.11.003","article-title":"From radiology findings to artificial intelligence-powered impressions: A retrospective study on the comparative performance of recent large language models","volume":"6","author":"N Tasneem","year":"2026","journal-title":"Intell Med"},{"issue":"12","key":"pdig.0001672.ref029","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2022-062981","article-title":"Supporting patient and public partners in writing lay summaries of scientific evidence in healthcare: a scoping review protocol","volume":"12","author":"S Zarshenas","year":"2022","journal-title":"BMJ Open"},{"key":"pdig.0001672.ref030","article-title":"Prompt engineering techniques","volume-title":"IBM Think","author":"V Gadesha"},{"issue":"2","key":"pdig.0001672.ref031","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1016\/j.jbi.2008.08.010","article-title":"Research electronic data capture (REDCap)\u2014A metadata-driven methodology and workflow process for providing translational research informatics support","volume":"42","author":"PA Harris","year":"2008","journal-title":"Journal of Biomedical Informatics"},{"key":"pdig.0001672.ref032","doi-asserted-by":"crossref","first-page":"103208","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: Building an international community of software platform partners","volume":"95","author":"PA Harris","year":"2019","journal-title":"Journal of Biomedical Informatics"},{"key":"pdig.0001672.ref033","author":"Google C"},{"key":"pdig.0001672.ref034","author":"OpenAI","year":"2025"},{"key":"pdig.0001672.ref035","article-title":"Self-preference bias in LLM-as-a-judge","author":"K Wataoka","year":"2024","journal-title":"arXiv"},{"issue":"4","key":"pdig.0001672.ref036","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"D Van Veen","year":"2024","journal-title":"Nat Med"},{"key":"pdig.0001672.ref037","article-title":"ReadCtrl: Personalizing text generation with readability-controlled instruction learning","author":"H Tran","year":"2024","journal-title":"arXiv"},{"key":"pdig.0001672.ref038","doi-asserted-by":"crossref","first-page":"9780317","DOI":"10.1155\/2017\/9780317","article-title":"Readability Assessment of Online Patient Education Material on Congestive Heart Failure","volume":"2017","author":"A Kher","year":"2017","journal-title":"Adv Prev Med"},{"key":"pdig.0001672.ref039","volume-title":"The Canadian Medical Imaging Inventory 2019\u20132020","author":"YS Chao","year":"2021"},{"issue":"2","key":"pdig.0001672.ref040","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.carj.2019.03.001","article-title":"Canadian Association of Radiologists White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology","volume":"70","author":"JL Jaremko","year":"2019","journal-title":"Can Assoc Radiol J"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001672","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,17]],"date-time":"2026-09-17T17:21:13Z","timestamp":1789665673000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001672"}},"subtitle":[],"editor":[{"given":"Phat Kim","family":"Huynh","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,17]]},"references-count":40,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,17]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001672","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,17]]}}}