{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T06:04:25Z","timestamp":1789452265646,"version":"build-2803163510"},"reference-count":30,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T00:00:00Z","timestamp":1789430400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012151","name":"Sanming Project of Medicine in Shenzhen","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012151","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Large language models (LLMs) are increasingly used by the public to obtain health information, but their ability to provide reliable information for achalasia remains unclear. We therefore compared six LLMs in answering public questions about achalasia, an uncommon esophageal motility disorder, focusing on safety, accuracy, empathy, information quality and reliability, and readability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In this cross-sectional comparative study, 40 patient-oriented questions about achalasia were submitted once to each of six LLMs (ChatGPT 5.5, Claude Opus 4.7, DeepSeek V4 Pro, Gemini 3.1 Pro Thinking, Grok-4.3, and Qwen3-Max), between May 7 and May 12, 2026. Three gastroenterologists independently assessed 240 responses for safety, accuracy, empathy, information quality, and readability using DISCERN, EQIP, the JAMA benchmark criteria assessing authorship, attribution, disclosure, and currency, and the Global Quality Score (GQS). Readability was evaluated using six established readability indices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Overall, 27 of 240 responses (11.3%) were classified as potentially unsafe, with proportions ranging from 7.5 to 15.0% across models; the overall between-model difference in safety was not statistically significant. Accuracy differed significantly among models, although the effect size was small. Larger differences were observed in empathy, information reliability and quality, and readability. ChatGPT 5.5 and Gemini 3.1 Pro Thinking achieved higher empathy scores, whereas Claude Opus 4.7 showed greater reading difficulty. JAMA benchmark scores were low overall, indicating limited source transparency.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Current LLMs provided generally accurate and often useful answers to public questions about achalasia. However, some differences were found in safety, empathy, transparency, information quality, and readability. Public-facing LLM responses to public questions about achalasia should be consistent with guidelines, clearly cite sources, communicate with empathy, and use plain language.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1962232","type":"journal-article","created":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T05:33:54Z","timestamp":1789450434000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Public-facing health information on achalasia generated by large language models: a multidimensional comparative study"],"prefix":"10.3389","volume":"14","author":[{"given":"Junzhen","family":"Li","sequence":"first","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pan","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Man","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kailing","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lihua","family":"Ren","sequence":"additional","affiliation":[{"name":"Department of Respiratory and Critical Care Medicine, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Qi","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Ouyang","sequence":"additional","affiliation":[{"name":"Department of Gastroenterology and Hepatology, Digestive Medicine Center, The Seventh Affiliated Hospital, Sun Yat-sen University","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,9,15]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"504","DOI":"10.1002\/ueg2.12555","article-title":"Global trends in incidence and prevalence of achalasia, 1925-2021: a systematic review and meta-analysis","volume":"12","author":"Lee","year":"2024","journal-title":"United Eur Gastroenterol J"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"1393","DOI":"10.14309\/ajg.0000000000000731","article-title":"ACG clinical guidelines: diagnosis and Management of Achalasia","volume":"115","author":"Vaezi","year":"2020","journal-title":"Am J Gastroenterol"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1177\/2050640620903213","article-title":"European guidelines on achalasia: united European gastroenterology and European Society of Neurogastroenterology and Motility recommendations","volume":"8","author":"Oude Nijhuis","year":"2020","journal-title":"United Eur Gastroenterol J"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"znag046","DOI":"10.1093\/bjs\/znag046","article-title":"Modern treatment for achalasia: endoscopic and surgical therapies","volume":"113","author":"Conner","year":"2026","journal-title":"Br J Surg"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"4027","DOI":"10.1007\/s00464-025-11789-z","article-title":"2024 update to SAGES guidelines for the use of peroral endoscopic myotomy (POEM) in the treatment of achalasia","volume":"39","author":"Calabrese","year":"2025","journal-title":"Surg Endosc"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence Chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med (Lausanne)"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digit Health"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1038\/s41746-024-01258-7","article-title":"A framework for human evaluation of large language models in healthcare derived from literature review","volume":"7","author":"Tam","year":"2024","journal-title":"NPJ Digit Med"},{"key":"ref11","first-page":"e2530220","article-title":"Reporting guideline for chatbot health advice studies: the CHART statement","volume-title":"JAMA Netw Open","author":"Huo","year":"2025"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"e14058","DOI":"10.1111\/nmo.14058","article-title":"Esophageal motility disorders on high-resolution manometry: Chicago classification version 4.0\u00a9","volume":"33","author":"Yadlapati","year":"2021","journal-title":"Neurogastroenterol Motil."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"ref15","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the world wide web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting Intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1007\/s10620-019-05784-3","article-title":"Updated systematic review of achalasia, with a focus on POEM therapy","volume":"65","author":"Cappell","year":"2020","journal-title":"Dig Dis Sci"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1111\/apt.18058","article-title":"Systematic review: the use of large language models as medical chatbots in digestive diseases","volume":"60","author":"Giuffr\u00e8","year":"2024","journal-title":"Aliment Pharmacol Ther"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"2276","DOI":"10.14309\/ajg.0000000000002397","article-title":"Evaluation of the potential utility of an artificial intelligence Chatbot in gastroesophageal reflux disease management","volume":"118","author":"Henson","year":"2023","journal-title":"Am J Gastroenterol"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1007\/s11695-025-08418-y","article-title":"Performance of large language models in metabolic bariatric surgery: a comparative study","volume":"36","author":"El-Masry","year":"2026","journal-title":"Obes Surg"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"e64486","DOI":"10.2196\/64486","article-title":"Accuracy of large language models when answering clinical research questions: systematic review and network Meta-analysis","volume":"27","author":"Wang","year":"2025","journal-title":"J Med Internet Res"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"e52597","DOI":"10.2196\/52597","article-title":"Large language models and empathy: systematic review","volume":"26","author":"Sorin","year":"2024","journal-title":"J Med Internet Res"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"1807","DOI":"10.1056\/NEJMoa1010502","article-title":"Pneumatic dilation versus laparoscopic Heller's myotomy for idiopathic achalasia","volume":"364","author":"Boeckxstaens","year":"2011","journal-title":"N Engl J Med"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"2219","DOI":"10.1056\/NEJMoa1905380","article-title":"Endoscopic or surgical Myotomy in patients with idiopathic achalasia","volume":"381","author":"Werner","year":"2019","journal-title":"N Engl J Med"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1001\/jama.2019.8859","article-title":"Effect of Peroral endoscopic Myotomy vs pneumatic dilation on symptom severity and treatment outcomes among treatment-naive patients with achalasia: a randomized clinical trial","volume":"322","author":"Ponds","year":"2019","journal-title":"JAMA"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"2572","DOI":"10.1007\/s11999-010-1380-y","article-title":"Assessing readability of patient education materials: current role in orthopaedics","volume":"468","author":"Badarudeen","year":"2010","journal-title":"Clin Orthop Relat Res"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"247","DOI":"10.5056\/jnm21081","article-title":"Adverse events associated with peroral endoscopic myotomy affecting extended hospital stay: a multi-center retrospective study in South Korea","volume":"28","author":"Lee","year":"2022","journal-title":"J Neurogastroenterol Motil"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.gie.2017.09.029","article-title":"Adverse events during and after per-oral endoscopic myotomy: prevention, diagnosis, and management","volume":"87","author":"Nabi","year":"2018","journal-title":"Gastrointest Endosc"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1962232\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T05:33:55Z","timestamp":1789450435000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1962232\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,15]]},"references-count":30,"alternative-id":["10.3389\/fpubh.2026.1962232"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1962232","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,15]]},"article-number":"1962232"}}