{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T01:22:47Z","timestamp":1791249767318,"version":"4.1.0"},"reference-count":32,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T00:00:00Z","timestamp":1786924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Patients increasingly rely on large language models (LLMs) for health information, yet their suitability for decision-critical conditions such as acute pancreatitis remains unclear. Given that acute pancreatitis requires timely symptom recognition, severity assessment, treatment decision-making, recurrence prevention, and follow-up management, LLM-generated information should demonstrate reliability, transparency, and readability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To evaluate the informational quality, visible transparency-related features, and readability of English-language responses generated by five publicly accessible LLMs to standardized patient-facing questions on acute pancreatitis.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>\n                      This cross-sectional benchmark study developed 24 English-language, single-intent questions on acute pancreatitis across six clinical domains using public search intents and guideline-derived decision-critical content. The analysis focused exclusively on English-language patient-facing responses. Each question was submitted once to GPT-5.4 Thinking, DeepSeek-V3.2, Gemini 3.1 Pro, Grok 4.3, and Qwen3.6-Max-Preview, generating 120 responses. Anonymized responses were independently evaluated by two blinded gastroenterologists using DISCERN, Ensuring Quality Information for Patients (EQIP), Global Quality Scale (GQS), and Journal of the American Medical Association (JAMA) benchmark criteria. Readability was assessed using six established formulas. A structured response-level safety analysis was added to evaluate factual inaccuracies, clinical hallucinations, and clinical safety-risk severity. Between-model differences were analyzed with Friedman tests followed by\n                      <jats:italic>post hoc<\/jats:italic>\n                      paired Wilcoxon signed-rank tests with Holm correction.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Significant between-model differences were observed across all quality, transparency-related, and readability outcomes. Grok 4.3 achieved the highest mean DISCERN, GQS, EQIP, and JAMA scores, reflecting the strongest informational quality profile according to the predefined quality instruments, although visible transparency cues remained limited across all models. In the added safety analysis, factual inaccuracies and clinical hallucinations were each identified in 16 of 120 responses (13.3%), whereas clinical safety-risk signals were identified in 7 responses (5.8%), all of which were adjudicated as score 1 (low risk) under the predefined clinician-rated rubric; no moderate- or high-risk clinical safety event was adjudicated. DeepSeek-V3.2 demonstrated the most favorable readability profile, with the highest Flesch Reading Ease score (44.42\u202f\u00b1\u202f12.27), which nevertheless remained substantially below the recommended threshold of \u2265 80. None of the 120 responses satisfied all six predefined readability targets. All model-level readability distributions differed significantly from recommended thresholds in the direction of poorer readability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Publicly accessible LLMs generated English-language responses on acute pancreatitis with variable informational quality, limited visible transparency cues, and consistently inadequate readability under standardized default public-interface conditions. Because the primary analysis used a single-generation cross-sectional design, model rankings should be interpreted as performance snapshots rather than definitive or temporally stable hierarchies. Higher quality scores did not necessarily establish factual accuracy, clinical safety, patient-education-level readability, or stronger response-level transparency. Current public-interface LLM outputs may serve as clinician-reviewed drafts for patient education but should not function as standalone patient resources. Future AI-based health information systems should strengthen clinical completeness, plain-language communication, visible evidence support, actionability, and clinician-supervised safeguards.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1880639","type":"journal-article","created":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T12:55:29Z","timestamp":1786971329000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Benchmarking publicly accessible large language models for English-language patient-facing acute pancreatitis information: a cross-sectional study of quality, transparency, and readability"],"prefix":"10.3389","volume":"14","author":[{"given":"Biao","family":"Jiang","sequence":"first","affiliation":[{"name":"Ward I, Department of Gastroenterology, Suining Central Hospital","place":["Suining, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongxi","family":"Sun","sequence":"additional","affiliation":[{"name":"Stomatology Center, Suining Central Hospital","place":["Suining, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Linlin","family":"Chen","sequence":"additional","affiliation":[{"name":"Ward I, Department of Gastroenterology, Suining Central Hospital","place":["Suining, Sichuan, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,17]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"419","DOI":"10.14309\/ajg.0000000000002645","article-title":"American College of Gastroenterology guidelines: management of acute pancreatitis","volume":"119","author":"Tenner","year":"2024","journal-title":"Am J Gastroenterol"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1053\/j.gastro.2024.02.052","article-title":"Diagnosis and Management of Acute Pancreatitis","volume":"167","author":"Trikudanathan","year":"2024","journal-title":"Gastroenterology"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"770","DOI":"10.1016\/j.pan.2025.04.020","article-title":"IAP\/APA\/EPC\/IPC\/JPS Working Group, P\u00e1rniczky A, Mik\u00f3 A, Uc A, Singh AN, Elhence A, et al. International Association of Pancreatology Revised Guidelines on Acute Pancreatitis 2025: Supported and Endorsed by the American Pancreatic Association, European Pancreatic Club, Indian Pancreas Club, and Japan Pancreas Society","volume":"25","year":"2025","journal-title":"Pancreatology"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1002\/ueg2.12743","article-title":"Acute pancreatitis: an update of evidence-based management and recent trends in treatment strategies","volume":"13","author":"Beij","year":"2025","journal-title":"United Eur Gastroenterol J"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"507","DOI":"10.2147\/PPA.S444955","article-title":"Health literacy needs of acute pancreatitis patients during the diagnosis and treatment process under the Lens of the timing it right theory: a qualitative study","volume":"18","author":"Su","year":"2024","journal-title":"Patient Prefer Adher"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"e68560","DOI":"10.2196\/68560","article-title":"Online health information-seeking in the era of large language models: cross-sectional web-based survey study","volume":"27","author":"Yun","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"e2457879","DOI":"10.1001\/jamanetworkopen.2024.57879","article-title":"Large language models for Chatbot health advice studies: a systematic review","volume":"8","author":"Huo","year":"2025","journal-title":"JAMA Netw Open"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1038\/s43856-024-00717-2","article-title":"Current applications and challenges in large language models for patient care: a systematic review","volume":"5","author":"Busch","year":"2025","journal-title":"Commun Med (Lond)"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence Chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med (Lausanne)"},{"key":"ref11","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e6541","DOI":"10.1097\/GOX.0000000000006541","article-title":"Readability of patient education materials in plastic surgery: assessing 14 years of progress","volume":"13","author":"Wirth","year":"2025","journal-title":"Plast Reconstr Surg Glob Open"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"e41780","DOI":"10.1097\/MD.0000000000041780","article-title":"Readability, reliability and quality of responses generated by ChatGPT, gemini, and perplexity for the most frequently asked questions about pain","volume":"104","author":"Ozduran","year":"2025","journal-title":"Medicine (Baltimore)"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"e18847","DOI":"10.7717\/peerj.18847","article-title":"Assessing the readability, quality and reliability of responses produced by ChatGPT, Gemini, and perplexity regarding most frequently asked keywords about low back pain","volume":"13","author":"Ozduran","year":"2025","journal-title":"PeerJ"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e69955","DOI":"10.2196\/69955","article-title":"Enhancing the readability of online patient education materials using large language models: cross-sectional study","volume":"27","author":"Will","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"578","DOI":"10.3760\/cma.j.cn112139-20210416-00172","article-title":"Guidelines for diagnosis and treatment of acute pancreatitis in China (2021)","volume":"59","year":"2021","journal-title":"Zhonghua Wai Ke Za Zhi"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the world wide web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"2572","DOI":"10.1007\/s11999-010-1380-y","article-title":"Assessing readability of patient education materials: current role in orthopaedics","volume":"468","author":"Badarudeen","year":"2010","journal-title":"Clin Orthop Relat Res"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: a 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J Patient Exp"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"108656","DOI":"10.1016\/j.pec.2025.108656","article-title":"Readability of written information for patients across 30 years: a systematic review of systematic reviews","volume":"135","author":"Okuhara","year":"2025","journal-title":"Patient Educ Couns"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1016\/j.asmr.2022.10.004","article-title":"Reducing the use of complex words and reducing sentence length to <15 words improves readability of patient education materials regarding sports medicine knee injuries","volume":"5","author":"Hanish","year":"2023","journal-title":"Arthrosc Sports Med Rehabil"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"17562848241255303","DOI":"10.1177\/17562848241255303","article-title":"Incidence of recurrent and chronic pancreatitis after acute pancreatitis: a systematic review and meta-analysis","volume":"17","author":"Gagyi","year":"2024","journal-title":"Ther Adv Gastroenterol"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the patient education materials assessment tool (PEMAT): a new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"Shoemaker","year":"2014","journal-title":"Patient Educ Couns"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"1810358","DOI":"10.3389\/fpubh.2026.1810358","article-title":"Benchmarking public large language model responses to patient-facing inflammatory bowel disease questions: informational quality, transparency proxies, and readability","volume":"14","author":"Wang","year":"2026","journal-title":"Front Public Health"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"1760871","DOI":"10.3389\/fpubh.2026.1760871","article-title":"Evaluation of validity, reliability, and readability of AI chatbots for gestational diabetes mellitus: a multi-model comparative study","volume":"14","author":"Wang","year":"2026","journal-title":"Front Public Health"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1038\/s41746-025-01752-6","article-title":"Evaluating evidence-based health information from generative AI using a cross-sectional study with laypeople seeking screening information","volume":"8","author":"Rebitschek","year":"2025","journal-title":"NPJ Digit Med"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"502","DOI":"10.1186\/s12876-025-04095-4","article-title":"Long-term quality of life after acute pancreatitis: a systematic review and meta-analysis","volume":"25","author":"Xu","year":"2025","journal-title":"BMC Gastroenterol"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"103222","DOI":"10.1016\/j.artmed.2025.103222","article-title":"Reporting guideline for chatbot health advice studies: the CHART statement","volume":"168","year":"2025","journal-title":"Artif Intell Med"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"e0000877","DOI":"10.1371\/journal.pdig.0000877","article-title":"Retrieval augmented generation for large language models in healthcare: a systematic review","volume":"4","author":"Amugongo","year":"2025","journal-title":"PLOS Digit Health"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3348\/kjr.2025.1522","article-title":"Minimum reporting items for clear evaluation of accuracy reports of large language models in healthcare (MI-CLEAR-LLM): 2025 updates","volume":"26","author":"Park","year":"2025","journal-title":"Korean J Radiol"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1880639\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T12:55:32Z","timestamp":1786971332000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1880639\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,17]]},"references-count":32,"alternative-id":["10.3389\/fpubh.2026.1880639"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1880639","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,17]]},"article-number":"1880639"}}