{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T18:05:32Z","timestamp":1790100332653,"version":"4.0.1"},"reference-count":35,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:00:00Z","timestamp":1790035200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>Outpatient clinic letters are a cornerstone of clinical communication but frequently exceed recommended reading levels, limiting patient comprehension and engagement. Large Language Models (LLMs) and Artificial Intelligence (AI) have been proposed as scalable tools for generating and simplifying these documents, but the evidence regarding their clinical accuracy, safety, and effectiveness remains fragmented. We conducted a systematic review of studies evaluating AI or LLMs for generating, simplifying, or enhancing outpatient clinic letters. Five databases (PubMed, EMBASE, Web of Science, CENTRAL, and CINAHL) were searched from inception to 1 November 2025. Two independent reviewers screened studies, extracted data, assessed quality using the Mixed Methods Appraisal Tool (MMAT), and certainty using the GRADE appraisal tool. Seven studies were included, comprising two studies using real-world clinic data and five using synthetic or hypothetical scenarios. Findings regarding readability were mixed; while some AI models improved readability scores compared to human-authored letters, most AI-generated content still failed to meet the recommended US Grade 6 reading level. The two studies that measured patient-reported outcomes reported high satisfaction and comprehension with AI-simplified letters, but these studies relied on subjective measures of understanding or lacked a human-authored control group. Clinical accuracy varied substantially, with information fidelity ranging from 10% to 100%. Risks of \u2018hallucinations\u2019, inappropriate medical advice, and tactless patient descriptions were documented. One study reported a ten-fold reduction in drafting time compared to human dictation. Overall certainty of evidence was rated very low across all primary outcomes due to methodological limitations, small sample sizes, and reliance on hypothetical scenarios. The current evidence supporting AI and LLMs use for outpatient clinic letters is preliminary and limited, providing insufficient evidence to support routine clinical implementation beyond supervised experimental and quality-improvement settings. Concerns regarding accuracy, safety, and generalisability remain. Further real-world patient-centred research is required before widespread adoption. PROSPERO registration: CRD420251181303.<\/jats:p>","DOI":"10.1371\/journal.pdig.0001745","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T17:25:15Z","timestamp":1790097915000},"page":"e0001745","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Large language models and artificial intelligence for generating, simplifying, and enhancing outpatient clinic letters: A systematic review"],"prefix":"10.1371","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-8848-0812","authenticated-orcid":true,"given":"Rocco","family":"Sheldon","sequence":"first","affiliation":[{"name":"University of Birmingham","place":["Birmingham, United Kingdom"],"department":["College of Medicine and Health"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Morya","family":"Wadodkar","sequence":"additional","affiliation":[{"name":"University of Bristol","place":["Bristol, United Kingdom"],"department":["Bristol Medical School"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Gan","sequence":"additional","affiliation":[{"name":"University of Birmingham","place":["Birmingham, United Kingdom"],"department":["College of Medicine and Health"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachael","family":"Yip","sequence":"additional","affiliation":[{"name":"University of Birmingham","place":["Birmingham, United Kingdom"],"department":["College of Medicine and Health"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yimeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"University Hospitals Birmingham NHS Foundation Trust","place":["Birmingham, United Kingdom"],"department":["Department of Renal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jyoti","family":"Baharani","sequence":"additional","affiliation":[{"name":"University Hospitals Birmingham NHS Foundation Trust","place":["Birmingham, United Kingdom"],"department":["Department of Renal Medicine"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"pdig.0001745.ref001","article-title":"Writing outpatient letters to patients","volume":"368","author":"H Rayner","year":"2020","journal-title":"BMJ"},{"key":"pdig.0001745.ref002","volume-title":"Copying letters to patients: good practice guidelines","year":"2003"},{"key":"pdig.0001745.ref003","volume-title":"Please, write to me. Writing outpatient clinic letters to patients. Guidance. Online","year":"2018"},{"key":"pdig.0001745.ref004","unstructured":"ROAN information sheet 23: Quality improvement: best practice for clinical letters Online: NHS England. 2023. Available from: https:\/\/www.england.nhs.uk\/professional-standards\/medical-revalidation\/ro\/info-docs\/roan-information-sheets\/quality-improvement-best-practice-for-clinical-letters\/"},{"issue":"12","key":"pdig.0001745.ref005","doi-asserted-by":"crossref","first-page":"2065","DOI":"10.1016\/j.pec.2018.06.014","article-title":"Should patients still be copied into their letters? A rapid review","volume":"101","author":"E Harris","year":"2018","journal-title":"Patient Educ Couns"},{"key":"pdig.0001745.ref006","volume-title":"National Action Plan to Improve Health Literacy. Online","year":"2010"},{"issue":"5","key":"pdig.0001745.ref007","doi-asserted-by":"crossref","DOI":"10.1111\/hex.70067","article-title":"How readable is the information the United Kingdom\u2019s statutory health and social care professional regulators provide for the public to engage with fitness to practise processes?","volume":"27","author":"S Haider","year":"2024","journal-title":"Health Expect"},{"issue":"3","key":"pdig.0001745.ref008","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0897-1897(96)80254-0","article-title":"Readability of patient education materials: implications for clinical practice","volume":"9","author":"J Albright","year":"1996","journal-title":"Appl Nurs Res"},{"key":"pdig.0001745.ref009","doi-asserted-by":"crossref","first-page":"108656","DOI":"10.1016\/j.pec.2025.108656","article-title":"Readability of written information for patients across 30 years: A systematic review of systematic reviews","volume":"135","author":"T Okuhara","year":"2025","journal-title":"Patient Educ Couns"},{"issue":"199","key":"pdig.0001745.ref010","first-page":"1","article-title":"Health literacy interventions and outcomes: an updated systematic review","author":"ND Berkman","year":"2011","journal-title":"Evid Rep Technol Assess (Full Rep)"},{"key":"pdig.0001745.ref011","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1080\/10810730.2011.604391","article-title":"Interventions for individuals with low health literacy: a systematic review","author":"SL Sheridan","year":"2011","journal-title":"J Health Commun"},{"issue":"7","key":"pdig.0001745.ref012","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1093\/rheumatology\/keae093","article-title":"Writing directly to patients, the time is now-but how?","volume":"63","author":"CA Sharp","year":"2024","journal-title":"Rheumatology (Oxford)"},{"issue":"11","key":"pdig.0001745.ref013","doi-asserted-by":"crossref","first-page":"1257","DOI":"10.1111\/ijcp.12686","article-title":"Communication in healthcare: a narrative review of the literature and practical recommendations","volume":"69","author":"P Vermeir","year":"2015","journal-title":"Int J Clin Pract"},{"issue":"3","key":"pdig.0001745.ref014","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1017\/S0022215109991770","article-title":"Readability of out-patient letters copied to patients: can patients understand what is written about them?","volume":"124","author":"SL Todhunter","year":"2010","journal-title":"J Laryngol Otol"},{"issue":"3","key":"pdig.0001745.ref015","doi-asserted-by":"crossref","DOI":"10.1136\/bmjoq-2019-000721","article-title":"Write2me: using patient feedback to improve postconsultation urology clinic letters","volume":"8","author":"PE Lonergan","year":"2019","journal-title":"BMJ Open Qual"},{"key":"pdig.0001745.ref016","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1186\/1741-7015-4-2","article-title":"How useful are post consultation letters to patients?","volume":"4","author":"NJ Roberts","year":"2006","journal-title":"BMC Med"},{"key":"pdig.0001745.ref017","doi-asserted-by":"crossref","DOI":"10.1192\/bjo.2024.484","article-title":"A service evaluation of clinicians writing clinic letters to patients","volume":"10","author":"E Junaid","year":"2024","journal-title":"BJPsych open"},{"issue":"4","key":"pdig.0001745.ref018","article-title":"Using ChatGPT to write patient clinic letters","volume":"5","author":"SR Ali","year":"2023","journal-title":"Lancet Digit Health"},{"issue":"3","key":"pdig.0001745.ref019","article-title":"Evaluating ChatGPT for converting clinic letters into patient-friendly language: a quantitative study","volume":"9","author":"SC Cork","year":"2025","journal-title":"BJGP Open"},{"key":"pdig.0001745.ref020","article-title":"The PRISMA 2020 statement: An updated guideline for reporting systematic reviews","volume":"372","author":"MJ Page","year":"2021","journal-title":"BMJ"},{"issue":"4","key":"pdig.0001745.ref021","doi-asserted-by":"crossref","first-page":"285","DOI":"10.3233\/EFI-180221","article-title":"The Mixed Methods Appraisal Tool (MMAT) version 2018 for information professionals and researchers","volume":"34","author":"QN Hong","year":"2018","journal-title":"EFI"},{"key":"pdig.0001745.ref022","doi-asserted-by":"crossref","first-page":"101484","DOI":"10.1016\/j.cegh.2023.101484","article-title":"Introduction to the GRADE tool for rating certainty in evidence and recommendations","volume":"25","author":"M Prasad","year":"2024","journal-title":"Clin Epidemiol Glob Health"},{"issue":"3","key":"pdig.0001745.ref023","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2023-076484","article-title":"Application of generative language models to orthopaedic practice","volume":"14","author":"J Caterson","year":"2024","journal-title":"BMJ Open"},{"issue":"12","key":"pdig.0001745.ref024","doi-asserted-by":"crossref","first-page":"2425","DOI":"10.1093\/ced\/llaf323","article-title":"Artificial intelligence compared with traditional methods of generating dermatology consultation letters: a pilot study comparing accuracy, readability and efficiency","volume":"50","author":"F Farooq","year":"2025","journal-title":"Clin Exp Dermatol"},{"key":"pdig.0001745.ref025","first-page":"6","article-title":"Can large language models generate outpatient clinic letters at first consultation that incorporate complication profiles from UK and USA Aesthetic Plastic Surgery Associations?","author":"RRH Roberts","year":"2024","journal-title":"Aesthetic Surg J Open Forum"},{"issue":"10","key":"pdig.0001745.ref026","doi-asserted-by":"crossref","first-page":"1269","DOI":"10.1177\/17531934241246479","article-title":"Can artificial intelligence make elective hand clinic letters easier for patients to understand?","volume":"49","author":"AC Stoneham","year":"2024","journal-title":"J Hand Surg Eur Vol"},{"key":"pdig.0001745.ref027","article-title":"Application of generative artificial intelligence for physician and patient oncology letters-AI-OncLetters","volume":"9","author":"A Shahnam","year":"2025","journal-title":"JCO Clin Cancer Inform"},{"issue":"3","key":"pdig.0001745.ref028","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"SJ Shoemaker","year":"2014","journal-title":"Patient Educ Couns"},{"issue":"1","key":"pdig.0001745.ref029","article-title":"Artificial Intelligence Scribe and Large Language Model Technology in Healthcare Documentation: Advantages, Limitations, and Recommendations","volume":"13","author":"SA Mess","year":"2025","journal-title":"Plast Reconstr Surg Glob Open"},{"issue":"4","key":"pdig.0001745.ref030","doi-asserted-by":"crossref","DOI":"10.1136\/bmjopen-2025-115562","article-title":"Use of ambient AI scribe in physicians\u2019 clinical documentation: a protocol for a systematic review on effectiveness, efficiency, and satisfaction","volume":"16","author":"C Garcia Sanchez","year":"2026","journal-title":"BMJ Open"},{"issue":"5","key":"pdig.0001745.ref031","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1111\/j.1365-2214.1996.tb00432.x","article-title":"Letter writing to parents following paediatric outpatient consultation: a survey of parent and GP views","volume":"22","author":"DM Cowper","year":"1996","journal-title":"Child Care Health Dev"},{"key":"pdig.0001745.ref032","doi-asserted-by":"crossref","first-page":"9780317","DOI":"10.1155\/2017\/9780317","article-title":"Readability assessment of online patient education material on congestive heart failure","volume":"2017","author":"A Kher","year":"2017","journal-title":"Adv Prev Med"},{"key":"pdig.0001745.ref033","article-title":"Readability of patient education materials from high-impact medical journals: A 20-year analysis","volume":"8","author":"MK Rooney","year":"2021","journal-title":"J Patient Exp"},{"issue":"1","key":"pdig.0001745.ref034","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1186\/s12889-020-08498-8","article-title":"Associations of health literacy with socioeconomic position, health risk behavior, and health status: a large national population-based survey among Danish adults","volume":"20","author":"MT Svendsen","year":"2020","journal-title":"BMC Public Health"},{"key":"pdig.0001745.ref035","doi-asserted-by":"crossref","unstructured":"Sheldon R, Aldons F, Khan H, Zhang Y, Baharani J. Outpatient letters in renal care: preferences, barriers, and recommendations from a diverse patient population [preprint]. 2025.","DOI":"10.2139\/ssrn.5553184"}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001745","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T17:25:22Z","timestamp":1790097922000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0001745"}},"subtitle":[],"editor":[{"given":"Milit","family":"Patel","sequence":"first","affiliation":[{"name":"The University of Texas at Austin","place":["UNITED STATES OF AMERICA"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":35,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,22]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0001745","relation":{},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]}}}