{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T05:20:21Z","timestamp":1787721621974,"version":"build-2784847793"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T00:00:00Z","timestamp":1787702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Search-enabled large language model interfaces are increasingly used by the public for health information, but their performance in mpox-related public health consultation remains unclear. This study evaluated their safety, accuracy, empathy, reliability\/information quality, and readability.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a single-query comparative cross-sectional evaluation using 52 predefined mpox-related public consultation questions. Each question was submitted once to each of six search-enabled LLM interfaces, yielding 312 first responses. Responses were assessed against a guideline-based reference framework. Safety was coded as a binary outcome, while accuracy and empathy were rated on 5-point scales. Reliability\/information quality was evaluated using DISCERN, EQIP, JAMA benchmark criteria, and GQS. Readability was assessed using six established readability indices. Five trained raters independently evaluated the human-scored outcomes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Unsafe responses were relatively infrequent but occurred in all six interfaces, with safe-response rates ranging from 86.5 to 92.3%. No pairwise difference in Safety remained statistically significant after Benjamini\u2013Hochberg correction. Overall differences across interfaces were statistically significant for Accuracy, Empathy, all four reliability\/information quality measures, and all six readability indices. Benjamini\u2013Hochberg-adjusted\n                      <jats:italic>post hoc<\/jats:italic>\n                      analyses identified outcome-specific pairwise differences, although the pairwise patterns varied across measures.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>The evaluated search-enabled LLM interfaces showed heterogeneous performance across safety, accuracy, empathy, reliability\/information quality, and readability. Although unsafe responses were relatively uncommon, potentially harmful outputs occurred in every interface. These findings support the need for guideline-based evaluation, source transparency, readability optimization, and robust safety safeguards when such interfaces are evaluated or considered for mpox-related public health consultation. The results represent a time- and configuration-specific interface-level snapshot; they should not be attributed to the underlying base models in isolation or interpreted as establishing reproducible performance or a stable hierarchy across sessions, versions, or settings.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1913260","type":"journal-article","created":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T04:47:52Z","timestamp":1787719672000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating search-enabled large language model interfaces for mpox public health consultation: a guideline-based comparative study"],"prefix":"10.3389","volume":"14","author":[{"given":"Qiqi","family":"Zheng","sequence":"first","affiliation":[{"name":"Nursing Unit, Department of Infectious Diseases and Hepatology Center, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ru","family":"Chen","sequence":"additional","affiliation":[{"name":"Nursing Unit, Department of Infectious Diseases and Hepatology Center, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingming","family":"Cai","sequence":"additional","affiliation":[{"name":"Nursing Unit, Department of Infectious Diseases and Hepatology Center, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanyan","family":"Chen","sequence":"additional","affiliation":[{"name":"Nursing Unit, Department of Infectious Diseases and Hepatology Center, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiuli","family":"Lin","sequence":"additional","affiliation":[{"name":"Nursing Unit, Department of Infectious Diseases and Hepatology Center, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tingting","family":"Wang","sequence":"additional","affiliation":[{"name":"Nursing Unit, Department of Gastroenterology, The First Affiliated Hospital of Wenzhou Medical University","place":["Wenzhou, Zhejiang, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,26]]},"reference":[{"key":"ref1","volume-title":"Mpox","year":"2024"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"e1012","DOI":"10.1016\/S2214-109X(23)00198-5","article-title":"Description of the first global outbreak of mpox: an analysis of global surveillance data","volume":"11","author":"Laurenson-Schafer","year":"2023","journal-title":"Lancet Glob Health"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1056\/nejmoa2207323","article-title":"Monkeypox virus infection in humans across 16 countries - April-June 2022","volume":"387","author":"Thornhill","year":"2022","journal-title":"N Engl J Med"},{"key":"ref4","volume-title":"Monkeypox in the United States and around the World","year":"2026"},{"key":"ref5","volume-title":"Mpox Global Strategic Preparedness and Response Plan","year":"2024"},{"key":"ref6","volume-title":"WHO Recommends New Name for Monkeypox Disease","year":"2022"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"e009838","DOI":"10.1136\/bmjgh-2022-009838","article-title":"Availability, scope and quality of monkeypox clinical management guidelines globally: a systematic review","volume":"7","author":"Webb","year":"2022","journal-title":"BMJ Glob Health"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"385","DOI":"10.15585\/mmwr.mm7422a3","article-title":"Use of JYNNEOS (smallpox and mpox vaccine, live, nonreplicating) for persons aged \u226518 years at risk for mpox during an mpox outbreak: recommendations of the advisory committee on immunization practices - United States, 2023","volume":"74","author":"Rao","year":"2025","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"734","DOI":"10.15585\/mmwr.mm7122e1","article-title":"Use of JYNNEOS (smallpox and monkeypox vaccine, live, nonreplicating) for preexposure vaccination of persons at risk for occupational exposure to Orthopoxviruses: recommendations of the advisory committee on immunization practices - United States, 2022","volume":"71","author":"Rao","year":"2022","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"232","DOI":"10.15585\/mmwr.mm7209a4","article-title":"Interim clinical treatment considerations for severe manifestations of mpox - United States, February 2023","volume":"72","author":"Rao","year":"2023","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1136\/bmjgh-2024-016937","article-title":"Risk factors associated with human mpox infection: a systematic review and meta-analysis","volume":"10","author":"Ugwu","year":"2025","journal-title":"BMJ Glob Health"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"1477898","DOI":"10.3389\/fmed.2024.1477898","article-title":"Large language models in patient education: a scoping review of applications in medicine","volume":"11","author":"Aydin","year":"2024","journal-title":"Front Med"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1016\/j.cmi.2023.05.017","article-title":"A study on the performance of ChatGPT in infectious diseases clinical consultation","volume":"29","author":"Sarink","year":"2023","journal-title":"Clin Microbiol Infect"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"104620","DOI":"10.1016\/j.jbi.2024.104620","article-title":"Evaluation of ChatGPT-generated medical responses: a systematic review and meta-analysis","volume":"151","author":"Wei","year":"2024","journal-title":"J Biomed Inform"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1370\/afm.250386","article-title":"The CHART Collaborative. Reporting guideline for chatbot health advice studies: chatbot assessment reporting tool (CHART) statement","volume":"23","year":"2025","journal-title":"Ann Fam Med"},{"key":"ref18","volume-title":"CDC Clear Communication Index","year":"2024"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"2374373521998847","DOI":"10.1177\/2374373521998847","article-title":"Readability of patient education materials from high-impact medical journals: a 20-year analysis","volume":"8","author":"Rooney","year":"2021","journal-title":"J Patient Exp"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"699","DOI":"10.1093\/fampra\/cmh621","article-title":"The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure","volume":"21","author":"Mercer","year":"2004","journal-title":"Fam Pract"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"ref23","doi-asserted-by":"crossref","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1007\/s10916-025-02178-z","article-title":"AI chatbots as sources of STD information: a study on reliability and readability","volume":"49","author":"Y\u0131ld\u0131z","year":"2025","journal-title":"J Med Syst"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"1776697","DOI":"10.3389\/fpubh.2026.1776697","article-title":"The information challenge in public health crises: a study on the reliability and readability of information provided by large language model for thunderstorm asthma","volume":"14","author":"Zhu","year":"2026","journal-title":"Front Public Health"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the world wide web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am J Gastroenterol"},{"key":"ref27","first-page":"1","volume-title":"Automated Readability Index","author":"Smith","year":"1967"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1037\/h0076540","article-title":"A computer readability formula designed for machine scoring","volume":"60","author":"Coleman","year":"1975","journal-title":"J Appl Psychol"},{"key":"ref29","first-page":"8","volume-title":"Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel","author":"Kincaid","year":"1975"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/h0057532","article-title":"A new readability yardstick","volume":"32","author":"Flesch","year":"1948","journal-title":"J Appl Psychol"},{"key":"ref31","first-page":"639","article-title":"SMOG grading: a new readability formula","volume":"12","author":"McLaughlin","year":"1969","journal-title":"J Reading"},{"key":"ref32","volume-title":"The Technique of Clear Writing","author":"Gunning","year":"1952"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"774","DOI":"10.21105\/joss.00774","article-title":"Quanteda: an R package for the quantitative analysis of textual data","volume":"3","author":"Benoit","year":"2018","journal-title":"J Open Source Softw"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J R Stat Soc Ser B Stat Methodol"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"5","DOI":"10.11613\/bm.2026.010101","article-title":"Effect sizes for nonparametric tests","volume":"36","author":"Fiel Peres","year":"2026","journal-title":"Biochem Med"},{"key":"ref36","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1007\/978-1-4899-4541-9","volume-title":"An Introduction to the Bootstrap","author":"Efron","year":"1993"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol Bull"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","article-title":"Intraclass correlations: uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol Bull"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1080\/19312450709336664","article-title":"Answering the call for a standard reliability measure for coding data","volume":"1","author":"Hayes","year":"2007","journal-title":"Commun Methods Meas"},{"key":"ref40","volume-title":"WHO Publishes public Health Advice on Preventing and Addressing stigma and Discrimination Related to mpox","year":"2022"},{"key":"ref41","volume-title":"Risk Assessment and Monitoring in Community Settings","year":"2026"},{"key":"ref42","volume-title":"Vaccine for Monkeypox Prevention in the United States","year":"2026"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-026-02428-5","article-title":"Large language models provide unsafe answers to patient-posed medical questions","volume":"9","author":"Draelos","year":"2026","journal-title":"NPJ Digit Med"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"1698596","DOI":"10.3389\/fpubh.2025.1698596","article-title":"Evaluation of accuracy, quality, and readability of information on hypothyroidism provided by different artificial intelligence chatbot models","volume":"13","author":"Ruan","year":"2025","journal-title":"Front Public Health"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"e59960","DOI":"10.7759\/cureus.59960","article-title":"Assessing the responses of large language models (ChatGPT-4, Gemini, and Microsoft copilot) to frequently asked questions in breast imaging: a study on readability and accuracy","volume":"16","author":"Tepe","year":"2024","journal-title":"Cureus"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"e39305","DOI":"10.1097\/MD.0000000000039305","article-title":"Assessment of readability, reliability, and quality of ChatGPT\u00ae, BARD\u00ae, Gemini\u00ae, copilot\u00ae, perplexity\u00ae responses on palliative care","volume":"103","author":"Hanc\u0131","year":"2024","journal-title":"Medicine"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1913260\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T04:48:08Z","timestamp":1787719688000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1913260\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,26]]},"references-count":46,"alternative-id":["10.3389\/fpubh.2026.1913260"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1913260","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,26]]},"article-number":"1913260"}}