{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T17:56:29Z","timestamp":1793382989521,"version":"4.5.0"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T00:00:00Z","timestamp":1778457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Pertussis (whooping cough) is a highly contagious respiratory infection that continues to cause substantial morbidity and mortality, particularly among infants, and has re-emerged globally among adolescents and adults. Large language models (LLMs) are increasingly used for health communication and science popularization; however, evidence regarding their readability, quality, and educational suitability for disease-specific patient education remains limited. To date, no systematic evaluation has focused on LLM-generated pertussis health education materials.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aimed to systematically evaluate and compare the performance of five mainstream LLMs in generating pertussis-related science popularization content, with particular attention to readability, informational quality, and educational suitability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>A cross-sectional simulation study was conducted using 20 frequently asked pertussis-related questions covering five domains: basic knowledge, symptom presentation, diagnostic methods, treatment and management, and prevention and prognosis. On October 28, 2025, all questions were identically input into five publicly accessible LLMs. Text readability was assessed using seven classical indices. Two independent pharmacists performed blinded evaluations using the Chinese version of the Patient Education Materials Assessment Tool for print materials (C-PEMAT-P) and the Global Quality Score (GQS). Additionally, two independent clinical experts assessed the factual accuracy and guideline concordance of each LLM-generated response against the Chinese Pertussis Diagnosis and Treatment Guidelines (2024) using a 4-point scale. Inter-rater agreement was evaluated using Cohen\u2019s kappa coefficient.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      ChatGPT, DeepSeek, and Doubao achieved significantly higher C-PEMAT and GQS scores than Wenxin Yiyan and Gemini (\n                      <jats:italic>p<\/jats:italic>\n                      \u202f&amp;lt;\u202f0.001), indicating superior understandability, actionability, and overall quality. Median C-PEMAT scores across all models suggested generally acceptable accessibility for patient education. Regarding factual accuracy and guideline concordance, ChatGPT achieved the highest mean score. No harmful advice or direct guideline contradictions were identified in any model output. Correlation analyses showed weak associations between traditional readability metrics and GQS, whereas C-PEMAT demonstrated a moderate positive correlation with GQS (r\u202f=\u202f0.34).\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Mainstream LLMs show preliminary capability in generating pertussis-related health education materials, although substantial inter-model variability persists. Domain-specific patient education assessment tools better capture perceived informational quality than generic readability metrics. These findings support the cautious, assistive use of LLMs in health communication within a human-AI collaborative framework.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1799204","type":"journal-article","created":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T05:29:49Z","timestamp":1778477389000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Benchmarking the readability, quality, and educational suitability of large language models in communicating pertussis"],"prefix":"10.3389","volume":"14","author":[{"given":"Xue Feng","family":"Zhong","sequence":"first","affiliation":[{"name":"Department of Clinical Trials of Drugs, Affiliated Jinhua Hospital, Zhejiang University School of Medicine, Jinhua Municipal Central Hospital","place":["Jinhua, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rong","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Pharmacy, The Affiliated Changsha Central Hospital, Hengyang Medical School, University of South China","place":["Changsha, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,5,11]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"4","DOI":"10.5455\/aim.2024.33.4-10","article-title":"The use of large language models in generating patient education materials: a scoping review","volume":"33","author":"AlSammarraie","year":"2025","journal-title":"Acta Inform Med"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1128\/cmr.00083-15","article-title":"Pertussis: microbiology, disease, treatment, and prevention","volume":"29","author":"Kilgore","year":"2016","journal-title":"Clin Microbiol Rev"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"2513729","DOI":"10.1080\/21645515.2025.2513729","article-title":"Resurgence of pertussis: epidemiological trends, contributing factors, challenges, and recommendations for vaccination and surveillance","volume":"21","author":"Wang","year":"2025","journal-title":"Hum Vaccin Immunother"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"974","DOI":"10.1016\/s1473-3099(17)30390-0","article-title":"An update of the global burden of pertussis in children younger than 5 years: a modelling study","volume":"17","author":"Yeung","year":"2017","journal-title":"Lancet Infect Dis"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"1181","DOI":"10.15537\/1658-3175.6051","article-title":"Pertussis. A reemerging and an underreported infectious disease","volume":"35","author":"Syed","year":"2014","journal-title":"Saudi Med J"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"1346","DOI":"10.3390\/vaccines12121346","article-title":"Epidemiology of pertussis after the COVID-19 pandemic: analysis of the factors involved in the resurgence of the disease in high-, middle-, and low-income countries","volume":"12","author":"Bricks","year":"2024","journal-title":"Vaccines (Basel)"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"2324547","DOI":"10.1080\/21645515.2024.2324547","article-title":"Understanding the impact of adult pertussis and current approaches to vaccination: a narrative review and expert panel recommendations","volume":"20","author":"Kardos","year":"2024","journal-title":"Hum Vaccin Immunother"},{"key":"ref8","article-title":"Health literacy","year":"2025"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"1257","DOI":"10.1186\/s12889-021-11344-0","article-title":"How to increase awareness of additional vaccinations; the case of maternal pertussis vaccination","volume":"21","author":"van Zoonen","year":"2021","journal-title":"BMC Public Health"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"114","DOI":"10.4258\/hir.2025.31.2.114","article-title":"Large language models in medicine: clinical applications, technical challenges, and ethical considerations","volume":"31","author":"Jung","year":"2025","journal-title":"Healthc Inform Res"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1111\/imj.16607","article-title":"Can artificial intelligence improve patient educational material readability? A systematic review and narrative synthesis","volume":"55","author":"Nasra","year":"2025","journal-title":"Intern Med J"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1186\/s12911-025-02954-4","article-title":"A systematic review of large language model (LLM) evaluations in clinical medicine","volume":"25","author":"Shool","year":"2025","journal-title":"BMC Med Inform Decis Mak"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1111\/apt.18058","article-title":"Systematic review: the use of large language models as medical chatbots in digestive diseases","volume":"60","author":"Giuffr\u00e8","year":"2024","journal-title":"Aliment Pharmacol Ther"},{"key":"ref14","article-title":"Centers for Disease Control and Prevention (CDC)","year":"2025"},{"key":"ref15","first-page":"1258","article-title":"Guidelines for diagnosis and management and prevention of pertussis of China (2024 edition)","volume":"104","year":"2024","journal-title":"Zhonghua Yi Xue Za Zhi"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"e47","DOI":"10.1097\/ccm.0000000000006121","article-title":"Assessment of the readability of the online patient education materials of intensive and critical care societies","volume":"52","author":"Hanci","year":"2024","journal-title":"Crit Care Med"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"e2246051","DOI":"10.1001\/jamanetworkopen.2022.46051","article-title":"Comparison of readability scores for written health information across formulas using automated vs manual measures","volume":"5","author":"Mac","year":"2022","journal-title":"JAMA Netw Open"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"e1366","DOI":"10.1016\/j.crad.2024.08.019","article-title":"Can generative AI improve the readability of patient education materials at a radiology practice?","volume":"79","author":"Gupta","year":"2024","journal-title":"Clin Radiol"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"104727","DOI":"10.1016\/j.jbi.2024.104727","article-title":"Health text simplification: an annotated corpus for digestive cancer education and novel strategies for reinforcement learning","volume":"158","author":"Rahman","year":"2024","journal-title":"J Biomed Inform"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"e39808","DOI":"10.2196\/39808","article-title":"The Chinese version of the patient education materials assessment tool for printable materials: translation, adaptation, and validation study","volume":"25","author":"Shan","year":"2023","journal-title":"J Med Internet Res"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"1751","DOI":"10.1002\/ohn.927","article-title":"Enhancing health literacy: evaluating the readability of patient handouts revised by ChatGPT's large language model","volume":"171","author":"Swisher","year":"2024","journal-title":"Otolaryngol Head Neck Surg"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"e69955","DOI":"10.2196\/69955","article-title":"Enhancing the readability of online patient education materials using large language models: cross-sectional study","volume":"27","author":"Will","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"1015","DOI":"10.1093\/jamia\/ocaf045","article-title":"Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks","volume":"32","author":"Dorfner","year":"2025","journal-title":"J Am Med Inform Assoc"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"2114","DOI":"10.1093\/jamia\/ocae074","article-title":"Large language models for biomedicine: foundations, opportunities, challenges, and best practices","volume":"31","author":"Sahoo","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"5450","DOI":"10.21037\/qims-2024-2655","article-title":"Fine-tuning medical language models for enhanced long-contextual understanding and domain expertise","volume":"15","author":"Yang","year":"2025","journal-title":"Quant Imaging Med Surg"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"640.e1","DOI":"10.1016\/j.ajog.2025.06.047","article-title":"Can artificial intelligence improve the readability of patient education information in gynecology?","volume":"233","author":"Daram","year":"2025","journal-title":"Am J Obstet Gynecol"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1186\/s13018-025-05955-1","article-title":"The role of large language models in improving the readability of orthopaedic spine patient educational material","volume":"20","author":"Romoff","year":"2025","journal-title":"J Orthop Surg Res"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"e69996","DOI":"10.7759\/cureus.69996","article-title":"Assessing the quality of patient education materials on cardiac catheterization from artificial intelligence Chatbots: an observational cross-sectional study","volume":"16","author":"Behers","year":"2024","journal-title":"Cureus"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"2260","DOI":"10.1097\/corr.0000000000002668","article-title":"Can artificial intelligence improve the readability of patient education materials?","volume":"481","author":"Kirchner","year":"2023","journal-title":"Clin Orthop Relat Res"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1111\/1744-9987.70058","article-title":"A comparative assessment of large language models in pediatric Dialysis: reliability, quality and readability","volume":"29","author":"Ensari","year":"2025","journal-title":"Ther Apher Dial"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med (Lausanne)"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"e71916","DOI":"10.2196\/71916","article-title":"Implementing large language models in health care: clinician-focused review with interactive guideline","volume":"27","author":"Li","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"e67883","DOI":"10.2196\/67883","article-title":"Evaluating the effectiveness of large language models in providing patient education for Chinese patients with ocular myasthenia gravis: mixed methods study","volume":"27","author":"Wei","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"e68817","DOI":"10.2196\/68817","article-title":"Improving the readability of institutional heart failure-related patient education materials using GPT-4: observational study","volume":"9","author":"King","year":"2025","journal-title":"JMIR Cardio"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"1624","DOI":"10.1186\/s12879-025-11621-y","article-title":"Readability and information quality of LLM-generated HIV content: a methodological content evaluation","volume":"25","author":"Chen","year":"2025","journal-title":"BMC Infect Dis"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"35454","DOI":"10.1038\/s41598-025-19370-3","article-title":"Evaluating the readability and quality of AI-generated scoliosis education materials: a comparative analysis of five language models","volume":"15","author":"Zhao","year":"2025","journal-title":"Sci Rep"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1799204\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T05:29:49Z","timestamp":1778477389000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1799204\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,11]]},"references-count":36,"alternative-id":["10.3389\/fpubh.2026.1799204"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1799204","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,11]]},"article-number":"1799204"}}