{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T19:54:56Z","timestamp":1791230096570,"version":"4.1.0"},"reference-count":37,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T00:00:00Z","timestamp":1773964800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Autoimmune hepatitis (AIH) is a chronic immune-mediated liver disease that requires long-term management, in which effective patient education plays a critical role. With the rapid development of large language models (LLMs), AI-generated health information is increasingly accessed by patients; however, the readability, quality, and educational suitability of LLM-generated AIH-related content remain insufficiently evaluated.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Five widely used LLMs\u2014ChatGPT, Doubao, DeepSeek, Wenxin Yiyan, and Tongyi Qianwen\u2014were assessed based on their responses to 20 frequently asked AIH patient education questions covering five thematic categories. Text readability was evaluated using multiple indices, including the Automated Readability Index, Flesch Reading Ease Score, Gunning Fog Index, Flesch\u2013Kincaid Grade Level, Coleman\u2013Liau Index, SMOG, and Linsear Write formula. Information quality and educational suitability were assessed using the Global Quality Score (GQS) and the Chinese version of the Patient Education Materials Assessment Tool (C-PEMAT). Clinical Intent Alignment (CIA) was used to evaluate the coverage of guideline-defined medical key points based on the 2025 EASL Clinical Practice Guidelines. Inter-rater reliability was analyzed using Cohen\u2019s kappa, and comparative and correlation analyses were performed.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Significant differences were observed among the LLMs in readability, information quality, and educational suitability (all\n                      <jats:italic>p<\/jats:italic>\n                      &amp;lt;\u202f0.05). ChatGPT achieved the highest GQS and C-PEMAT scores, followed by Doubao and DeepSeek, whereas Wenxin Yiyan and Tongyi Qianwen showed lower performance and greater variability. CIA analysis indicated comparable coverage of guideline-defined clinical intent across models. Readability varied significantly across content themes, with texts related to disease mechanisms and diagnostic processes exhibiting higher linguistic complexity. Correlation analysis demonstrated moderate associations between GQS and grade-level readability indices, whereas C-PEMAT and CIA showed weak correlations with traditional readability metrics.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Substantial variability exists among LLMs in generating AIH patient education materials. Model selection critically influences information quality and educational suitability, whereas content theme primarily affects linguistic complexity. Although most models produced moderate-to-good quality information, relatively high readability levels suggest that further simplification may be needed for general patient populations. A multidimensional evaluation framework integrating readability, quality, educational suitability, and clinical intent alignment is essential for the responsible use of LLMs in AIH patient education.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1805848","type":"journal-article","created":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T06:37:48Z","timestamp":1773988668000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Performance evaluation of mainstream large language models in autoimmune hepatitis patient education: a comparative study of readability, quality, and reliability"],"prefix":"10.3389","volume":"14","author":[{"given":"Hanlu","family":"Li","sequence":"first","affiliation":[{"name":"Department of Infectious Diseases, Beibei Affiliated Hospital of Chongqing Medical University","place":["Chongqing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuru","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Infectious Diseases, Beibei Affiliated Hospital of Chongqing Medical University","place":["Chongqing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Lv","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology, The Second Affiliated Hospital of Chongqing Medical University","place":["Chongqing, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,3,20]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"e070201","DOI":"10.1136\/bmj-2022-070201","article-title":"Diagnosis and management of autoimmune hepatitis","volume":"380","author":"Muratori","year":"2023","journal-title":"BMJ (Clinical research ed)"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"671","DOI":"10.1002\/hep.31065","article-title":"Diagnosis and Management of Autoimmune Hepatitis in adults and children: 2019 practice guidance and guidelines from the American Association for the Study of Liver Diseases","volume":"72","author":"Mack","year":"2020","journal-title":"Hepatology"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"971","DOI":"10.1016\/j.jhep.2015.06.030","article-title":"EASL clinical practice guidelines: autoimmune hepatitis","volume":"63","year":"2015","journal-title":"J Hepatol"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"102570","DOI":"10.1016\/j.clinre.2025.102570","article-title":"Advancements in autoimmune hepatitis epidemiology, treatment and complication - a 15-year retrospective study","volume":"49","author":"Tatour","year":"2025","journal-title":"Clin Res Hepatol Gastroenterol"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"453","DOI":"10.1016\/j.jhep.2025.03.017","article-title":"EASL clinical practice guidelines on the management of autoimmune hepatitis","volume":"83","year":"2025","journal-title":"J Hepatol"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"653","DOI":"10.1016\/j.bpg.2011.09.009","article-title":"Pathogenesis of autoimmune hepatitis","volume":"25","author":"Liberal","year":"2011","journal-title":"Best Pract Res Clin Gastroenterol"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1016\/j.jaut.2012.11.002","article-title":"Autoimmune hepatitis: a comprehensive review","volume":"41","author":"Liberal","year":"2013","journal-title":"J Autoimmun"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1016\/s0140-6736(12)62163-1","article-title":"Autoimmune hepatitis","volume":"382","author":"Heneghan","year":"2013","journal-title":"Lancet"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.1097\/hc9.0000000000000458","article-title":"Autoimmune hepatitis: current and future therapies","volume":"8","author":"Reau","year":"2024","journal-title":"Hepatol Commun"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"1642","DOI":"10.4254\/wjh.v13.i11.1642","article-title":"Health-related quality of life in autoimmune hepatitis","volume":"13","author":"Snijders","year":"2021","journal-title":"World J Hepatol"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"618","DOI":"10.1016\/j.jhep.2013.10.035","article-title":"Health-related quality of life, depression, and anxiety in patients with autoimmune hepatitis","volume":"60","author":"Schramm","year":"2014","journal-title":"J Hepatol"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e9","DOI":"10.2196\/jmir.5729","article-title":"Internet health information seeking and the patient-physician relationship: a systematic review","volume":"19","author":"Tan","year":"2017","journal-title":"J Med Internet Res"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"2417","DOI":"10.1001\/jama.2018.16865","article-title":"Addressing health-related misinformation on social media","volume":"320","author":"Chou","year":"2018","journal-title":"JAMA"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1884","DOI":"10.1007\/s11606-019-05109-0","article-title":"Can patients trust online health information? A Meta-narrative systematic review addressing the quality of health information on the internet","volume":"34","author":"Daraz","year":"2019","journal-title":"J Gen Intern Med"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digital Health"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"e51319","DOI":"10.2196\/51319","article-title":"Assessing the current limitations of large language models in advancing health care education","volume":"9","author":"Kim","year":"2025","journal-title":"JMIR Form Res"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3390\/informatics11030057","article-title":"Large language models in healthcare and medical domain: a review","volume":"11","author":"Nazi","year":"2024","journal-title":"Informatics"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"914","DOI":"10.14309\/ajg.0000000000003179","article-title":"Is ChatGPT-4 a reliable tool in autoimmune hepatitis?","volume":"120","author":"Colapietro","year":"2025","journal-title":"Am J Gastroenterol"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"101537","DOI":"10.1016\/j.aohep.2024.101537","article-title":"Evaluation of four chatbots in autoimmune liver disease: a comparative analysis","volume":"30","author":"Daza","year":"2025","journal-title":"Ann Hepatol"},{"key":"ref21","article-title":"Automated rubrics for reliable evaluation of medical dialogue systems","author":"Chen","year":"2026"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"NP839","DOI":"10.1093\/asj\/sjae140","article-title":"Future perspective of risk prediction in aesthetic surgery: is artificial intelligence reliable?","volume":"44","author":"Duran","year":"2024","journal-title":"Aesthet Surg J"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"1559265","DOI":"10.3389\/fendo.2025.1559265","article-title":"Comparison of artificial intelligence-generated and physician-generated patient education materials on early diabetic kidney disease","volume":"16","author":"Cheng","year":"2025","journal-title":"Front Endocrinol"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Van Veen","year":"2024","journal-title":"Nat Med"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"e40895","DOI":"10.7759\/cureus.40895","article-title":"ChatDoctor: a medical chat model fine-tuned on a large language model Meta-AI (LLaMA) using medical domain knowledge","volume":"15","author":"Li","year":"2023","journal-title":"Cureus"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1056\/aioa2300068","article-title":"Almanac - retrieval-augmented language models for clinical medicine","volume":"1","author":"Zakka","year":"2024","journal-title":"Nejm ai"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"1","DOI":"10.48550\/arXiv.2101.03961","article-title":"Switch transformers: scaling to trillion parameter models with simple and efficient sparsity","volume":"23","author":"Fedus","year":"2022","journal-title":"J Machine Learn Res."},{"key":"ref29","article-title":"Training language models to follow instructions with human feedback","author":"Ouyang","year":"2022"},{"key":"ref30","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Advanc Neural Inform Process Syst"},{"key":"ref31","article-title":"Language models don't always say what they think: unfaithful explanations in chain-of-thought prompting","author":"Turpin","year":"2023"},{"key":"ref32","first-page":"1532","article-title":"Online and ChatGPT-generated patient education materials regarding brain tumor prognosis fail to meet readability standards","author":"Shukla","year":"2025","journal-title":"J Clin Neurosci"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"e16670","DOI":"10.2196\/16670","article-title":"Patient perception of plain-language medical notes generated using artificial intelligence software: pilot mixed-methods study","volume":"4","author":"Bala","year":"2020","journal-title":"JMIR Form Res"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: a 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J Patient Exp"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1093\/ajhp\/zxy021","article-title":"Understandability, actionability, and readability of online patient education materials about diabetes mellitus","volume":"76","author":"Lipari","year":"2019","journal-title":"Am J Health Syst Pharm"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1177\/0194599815626146","article-title":"Readability and understandability of online vocal cord paralysis materials","volume":"154","author":"Balakrishnan","year":"2016","journal-title":"Otolaryngol Head Neck Surg"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"1750225","DOI":"10.3389\/fbioe.2026.1750225","article-title":"Benchmarking readability, reliability, and scientific quality of large language models in communicating organoid science","volume":"14","author":"Sun","year":"2026","journal-title":"Front Bioeng Biotechnol"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1805848\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T06:37:50Z","timestamp":1773988670000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1805848\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,20]]},"references-count":37,"alternative-id":["10.3389\/fpubh.2026.1805848"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1805848","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,20]]},"article-number":"1805848"}}