{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T15:44:45Z","timestamp":1789055085043,"version":"build-2803163510"},"reference-count":30,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T00:00:00Z","timestamp":1770163200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Gestational diabetes mellitus (GDM) is increasingly prevalent worldwide and is associated with substantial short- and long-term risks for mothers and offspring, making high-quality, accessible health information essential. At the same time, artificial intelligence (AI) chatbots based on large language models are being widely used for health queries, yet their accuracy, reliability and readability in the context of GDM remain unclear.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We first evaluated six AI chatbots (ChatGPT-5, ChatGPT-4o, DeepSeek-V3.2, DeepSeek-R1, Gemini 2.5 Pro and Claude Sonnet 4.5) using 200 single-best-answer multiple-choice questions (MCQs) on GDM drawn from MedQA, MedMCQA and the Chinese National Medical Examination item bank, covering four domains: epidemiology and risk factors, clinical manifestations and diagnosis, maternal and neonatal outcomes, and management and treatment. Each item was posed three times to every model under a standardized prompting protocol, and accuracy was defined as the proportion of correctly answered questions. For public-facing information, we identified 15 core GDM education questions using Google Trends and expert review, and queried four chatbots (ChatGPT-5, DeepSeek-V3.2, Claude Sonnet 4.5 and Gemini 2.5 Pro). Two obstetricians independently assessed reliability using DISCERN, EQIP, GQS and JAMA benchmarks, and readability was quantified using ARI, CL, FKGL, FRES, GFI and SMOG indices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>\n                      Overall MCQ accuracy differed significantly across the six chatbots (\n                      <jats:italic>p<\/jats:italic>\n                      \u202f&amp;lt;\u202f0.0001), with ChatGPT-5 achieving the highest mean accuracy (92.17%) and DeepSeek-V3.2 and Gemini 2.5 Pro performing comparably well, while ChatGPT-4o, DeepSeek-R1 and Claude Sonnet 4.5 scored lower. Newer model generations (ChatGPT-5 vs. ChatGPT-4o; DeepSeek-V3.2 vs. DeepSeek-R1) consistently outperformed their predecessors across all four domains. Among the four models evaluated on public-education questions, ChatGPT-5 achieved the highest reliability scores (DISCERN 42.53\u202f\u00b1\u202f7.20; EQIP 71.67\u202f\u00b1\u202f6.17), whereas Claude Sonnet 4.5, DeepSeek-V3.2 and Gemini 2.5 Pro scored lower. JAMA scores were uniformly low (0\u20130.07\/4), reflecting poor transparency. All models produced text above the recommended sixth-grade reading level; ChatGPT-5 showed the most favorable readability profile (for example, FKGL 7.43\u202f\u00b1\u202f2.42, FRES 62.47\u202f\u00b1\u202f13.51) but still did not meet guideline targets.\n                    <\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Contemporary AI chatbots can generate generally accurate and moderately reliable GDM-related information, with newer model generations showing clear gains in diagnostic validity. However, limited transparency and systematically high reading levels indicate that these tools are not yet suitable as stand-alone resources for GDM patient education and should be used as adjuncts to clinician counseling and professionally curated materials.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2026.1760871","type":"journal-article","created":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T06:48:42Z","timestamp":1770187722000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Evaluation of validity, reliability, and readability of AI chatbots for gestational diabetes mellitus: a multi-model comparative study"],"prefix":"10.3389","volume":"14","author":[{"given":"Xinxin","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuyan","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chuanqing","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Obstetrics, Shenzhen Nanshan Maternity and Child Healthcare Hospital","place":["Shenzhen, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rongkang","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Urology, South China Hospital, Medical School, Shenzhen University","place":["Shenzhen, China"]},{"name":"Department of Urology, Lanzhou University Second Hospital, Lanzhou University","place":["Lanzhou, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,2,4]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"109050","DOI":"10.1016\/j.diabres.2021.109050","article-title":"IDF diabetes atlas: estimation of global and regional gestational diabetes mellitus prevalence for 2021 by International Association of Diabetes in pregnancy study group's criteria","volume":"183","author":"Wang","year":"2022","journal-title":"Diabetes Res Clin Pract"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"108642","DOI":"10.1016\/j.diabres.2020.108642","article-title":"Increasing prevalence of gestational diabetes mellitus when implementing the IADPSG criteria: a systematic review and meta-analysis","volume":"172","author":"Saeedi","year":"2021","journal-title":"Diabetes Res Clin Pract"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"e13122","DOI":"10.1111\/obr.13122","article-title":"Effects of lifestyle intervention on long-term risk of diabetes in women with prior gestational diabetes: a systematic review and meta-analysis of randomized controlled trials","volume":"22","author":"Li","year":"2021","journal-title":"Obes Rev"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1111\/obr.12645","article-title":"Long-term risk of diabetes in women at varying durations after gestational diabetes: a systematic review and meta-analysis with more than 2 million women","volume":"19","author":"Song","year":"2018","journal-title":"Obes Rev"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1186\/s12884-016-0856-5","article-title":"Internet use by pregnant women seeking pregnancy-related information: a systematic review","volume":"16","author":"Sayakhot","year":"2016","journal-title":"BMC Pregnancy Childbirth"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"79","DOI":"10.4103\/ijnmr.IJNMR_82_17","article-title":"Internet usage among pregnant women for seeking health information: a review article","volume":"23","author":"Javanmardi","year":"2018","journal-title":"Iran J Nurs Midwifery Res"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"392","DOI":"10.1186\/s12884-024-06594-w","article-title":"Current status of electronic health literacy among pregnant women with gestational diabetes mellitus and their perceptions of online health information: a mixed-methods study","volume":"24","author":"Xu","year":"2024","journal-title":"BMC Pregnancy Childbirth"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"e41780","DOI":"10.1097\/md.0000000000041780","article-title":"Readability, reliability and quality of responses generated by ChatGPT, gemini, and perplexity for the most frequently asked questions about pain","volume":"104","author":"Ozduran","year":"2025","journal-title":"Medicine"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1038\/s41598-023-50884-w","article-title":"Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy","volume":"14","author":"Onder","year":"2024","journal-title":"Sci Rep"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"100594","DOI":"10.1016\/j.ajpc.2023.100594","article-title":"Readability and reliability of online patient education materials about statins","volume":"16","author":"Ngo","year":"2023","journal-title":"Am J Prev Cardiol"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"e13439","DOI":"10.2196\/13439","article-title":"Google trends in infodemiology and infoveillance: methodology framework","volume":"5","author":"Mavragani","year":"2019","journal-title":"JMIR Public Health Surveill"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"2054","DOI":"10.1093\/jamia\/ocae079","article-title":"Large language models leverage external knowledge to extend clinical insight beyond language boundaries","volume":"31","author":"Wu","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","year":"1999","journal-title":"J Epidemiol Community Health"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e31","DOI":"10.1016\/j.wneu.2012.11.003","article-title":"Evaluation of the quality of information on the internet available to patients undergoing cervical spine surgery","volume":"82","author":"Weil","year":"2014","journal-title":"World Neurosurg"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1111\/j.1369-7625.2004.00273.x","article-title":"Ensuring quality information for patients: development and preliminary validation of a new instrument to improve the quality of written health care information","volume":"7","author":"Moult","year":"2004","journal-title":"Health Expect"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1046\/j.1369-6513.2002.00189.x","article-title":"Improving the quality of health information: the contribution of C-H-i-Q","volume":"5","author":"Hain","year":"2002","journal-title":"Health Expect"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1111\/iej.14014","article-title":"Validity and reliability of artificial intelligence chatbots as public sources of information on endodontics","volume":"57","author":"Mohammad-Rahimi","year":"2024","journal-title":"Int Endod J"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"1244","DOI":"10.1001\/jama.1997.03540390074039","article-title":"Assessing, controlling, and assuring the quality of medical information on the internet: caveant lector et viewor--let the reader and viewer beware","volume":"277","author":"Silberg","year":"1997","journal-title":"JAMA"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting Intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1037\/1082-989x.1.1.30","article-title":"Forming inferences about some intraclass correlation coefficients","volume":"1","author":"McGraw","year":"1996","journal-title":"Psychol Methods"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1037\/\/0033-2909.86.2.420","article-title":"Intraclass correlations: uses in assessing rater reliability","volume":"86","author":"Shrout","year":"1979","journal-title":"Psychol Bull"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"691033","DOI":"10.3389\/fendo.2021.691033","article-title":"Gestational diabetes mellitus in Europe: a systematic review and Meta-analysis of prevalence studies","volume":"12","author":"Paulo","year":"2021","journal-title":"Front Endocrinol"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1991","DOI":"10.1056\/NEJMoa0707943","article-title":"Hyperglycemia and adverse pregnancy outcomes","volume":"358","author":"Metzger","year":"2008","journal-title":"N Engl J Med"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"e29020","DOI":"10.1016\/j.heliyon.2024.e29020","article-title":"Quality assessment of videos on social media platforms related to gestational diabetes mellitus in China: a cross-section study","volume":"10","author":"Cai","year":"2024","journal-title":"Heliyon"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"e211","DOI":"10.2196\/jmir.5999","article-title":"Making quality health websites a National Public Health Priority: toward quality standards","volume":"18","author":"Devine","year":"2016","journal-title":"J Med Internet Res"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"556","DOI":"10.1001\/jamaoto.2023.0704","article-title":"Comparison between ChatGPT and Google search as sources of postoperative patient instructions","volume":"149","author":"Ayoub","year":"2023","journal-title":"JAMA Otolaryngol Head Neck Surg"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"1392","DOI":"10.1016\/j.arth.2023.01.032","article-title":"Readability of online patient education materials for Total joint arthroplasty: a systematic review","volume":"38","author":"Karimi","year":"2023","journal-title":"J Arthroplast"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1038\/s41443-025-01056-z","article-title":"Comparative analysis of the effectiveness of Microsoft copilot artificial intelligence chatbot and Google search in answering patient inquiries about infertility: evaluating readability, understandability, and actionability","volume":"37","author":"Bahceci","year":"2025","journal-title":"Int J Impot Res"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"e109583","DOI":"10.1371\/journal.pone.0109583","article-title":"The use of google trends in health care research: a systematic review","volume":"9","author":"Nuti","year":"2014","journal-title":"PLoS One"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1760871\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T06:48:42Z","timestamp":1770187722000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2026.1760871\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,4]]},"references-count":30,"alternative-id":["10.3389\/fpubh.2026.1760871"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2026.1760871","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,4]]},"article-number":"1760871"}}