{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T19:38:50Z","timestamp":1790624330643,"version":"4.1.0"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T00:00:00Z","timestamp":1765324800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Public Health"],"abstract":"<jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study assessed the accuracy, quality, and readability of responses from three leading AI chatbots\u2014ChatGPT-3.5, DeepSeek-V3, and Google Gemini-2.5\u2014on the diagnosis, treatment, and long-term risks of adult hypothyroidism, comparing their outputs with current clinical guidelines.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>Two thyroid specialists developed 27 questions based on the Guideline for the Diagnosis and Management of Hypothyroidism in Adults (2017 edition), covering three categories: diagnosis, treatment, and long-term health risks. Responses from each AI model were independently evaluated by two reviewers. Accuracy was rated using a six-point Likert scale, quality using the DISCERN tool and the five-point Likert scale, and readability was assessed by the Flesch Reading Ease (FRE), Flesch\u2013Kincaid Grade Level (FKGL), Gunning Fog Index (GFI),and Simple Measure of Gobbledygook(SMOG).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>All three AI models demonstrated excellent performance in accuracy (mean score &amp;gt; 4.5) and quality (high-quality rate &amp;gt; 94%). According to the DISCERN tool, no significant difference was observed in the overall information quality among the models. However, Gemini-2.5 generated responses of significantly lower quality for treatment-related questions than for diagnostic inquiries. The content generated by all models was relatively difficult to comprehend (low FRE scores and high FKGL\/GFI scores), generally requiring a college-level or higher education for adequate understanding.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>All three AI chatbots were capable of producing highly accurate and high-quality medical information regarding hypothyroidism, with their responses showing strong consistency with clinical guidelines. This underscores the substantial potential of AI in supporting medical information delivery. However, the consistently high reading difficulty of their outputs may limit their practical utility in patient education. Future research should focus on improving the readability and patient-friendliness of AI outputs\u2014through prompt engineering and multi-round dialogue optimization\u2014while maintaining professional accuracy, to enable broader application of AI in health education.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpubh.2025.1698596","type":"journal-article","created":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T06:40:57Z","timestamp":1765348857000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Evaluation of accuracy, quality, and readability of information on hypothyroidism provided by different artificial intelligence chatbot models"],"prefix":"10.3389","volume":"13","author":[{"given":"Ting","family":"Ruan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xinran","family":"Shao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yihan","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingai","family":"Ju","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianchun","family":"Cui","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,12,10]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1038\/nrendo.2018.18","article-title":"Global epidemiology of hyperthyroidism and hypothyroidism","volume":"14","author":"Taylor","year":"2018","journal-title":"Nat Rev Endocrinol"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1016\/j.eprac.2022.05.004","article-title":"Management of Hypothyroidism and Hypothyroxinemia during Pregnancy","volume":"28","author":"Pearce","year":"2022","journal-title":"Endocr Pract"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"2618","DOI":"10.1001\/archinte.165.22.2618","article-title":"Trust and sources of health information: the impact of the internet and its implications for health care providers: findings from the first health information National Trends Survey","volume":"165","author":"Hesse","year":"2005","journal-title":"Arch Intern Med"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"e30940","DOI":"10.2196\/30940","article-title":"Accelerating the appropriate adoption of artificial intelligence in health care: protocol for a multistepped approach","volume":"10","author":"Wiljer","year":"2021","journal-title":"JMIR Res Protoc"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"527","DOI":"10.1007\/s00417-024-06625-4","article-title":"Gemini AI vs. ChatGPT: a comprehensive examination alongside ophthalmology residents in medical knowledge","volume":"263","author":"Bahir","year":"2025","journal-title":"Graefes Arch Clin Exp Ophthalmol"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"104667","DOI":"10.1016\/j.amjoto.2025.104667","article-title":"Evaluating advanced AI reasoning models: ChatGPT-4.0 and DeepSeek-R1 diagnostic performance in otolaryngology: a comparative analysis","volume":"46","author":"Prasad","year":"2025","journal-title":"Am J Otolaryngol"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"e22618","DOI":"10.2196\/22618","article-title":"Using the ensuring quality information for patients tool to assess patient information on appendicitis websites: systematic search and evaluation","volume":"23","author":"Ghani","year":"2021","journal-title":"J Med Internet Res"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"105593","DOI":"10.1016\/j.ijmedinf.2024.105593","article-title":"Assessing the accuracy and reliability of ChatGPT's medical responses about thyroid cancer","volume":"191","author":"Cavnar Helvaci","year":"2024","journal-title":"Int J Med Inform"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"13","DOI":"10.3760\/cma.j.issn.1000-6699.2017.02.018","article-title":"Guideline for the diagnosis and management of hypothyroidism in adults","volume":"33","year":"2017","journal-title":"Chin J Endocrinol Metab"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"56","DOI":"10.3390\/ai6030056","article-title":"Emotion-aware embedding fusion in large language models (flan-T5, llama 2, DeepSeek-R1, and ChatGPT 4) for intelligent response generation","volume":"6","author":"Rasool","year":"2025","journal-title":"AI"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1007\/s10916-025-02178-z","article-title":"AI Chatbots as sources of STD information: a study on reliability and readability","volume":"49","author":"Y\u0131ld\u0131z","year":"2025","journal-title":"J Med Syst"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"e39305","DOI":"10.1097\/MD.0000000000039305","article-title":"Assessment of readability, reliability, and quality of ChatGPT\u00ae, BARD\u00ae, Gemini\u00ae, copilot\u00ae, perplexity\u00ae responses on palliative care","volume":"103","author":"Hanc\u0131","year":"2024","journal-title":"Medicine (Baltimore)"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1038\/s41598-023-50884-w","article-title":"Evaluation of the reliability and readability of ChatGPT-4 responses regarding hypothyroidism during pregnancy","volume":"14","author":"Onder","year":"2024","journal-title":"Sci Rep"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"1041","DOI":"10.1007\/s12020-024-04066-x","article-title":"Assessing the feasibility of ChatGPT-4o and Claude 3-opus in thyroid nodule classification based on ultrasound images","volume":"87","author":"Chen","year":"2025","journal-title":"Endocrine"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"e47576","DOI":"10.7759\/cureus.47576","article-title":"A novel approach: evaluating ChatGPT's utility for the management of thyroid nodules","volume":"15","author":"K\u00f6ro\u011flu","year":"2023","journal-title":"Cureus"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"e47479","DOI":"10.2196\/47479","article-title":"Reliability of medical information provided by ChatGPT: assessment against clinical guidelines and patient information quality instrument","volume":"25","author":"Walker","year":"2023","journal-title":"J Med Internet Res"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"113684","DOI":"10.1016\/j.bbr.2021.113684","article-title":"Methodological aspects of studying the mechanisms of consciousness","volume":"419","author":"Tsytsarev","year":"2022","journal-title":"Behav Brain Res"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"e37432","DOI":"10.7759\/cureus.37432","article-title":"Exploring the boundaries of reality: investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references","volume":"15","author":"Athaluri","year":"2023","journal-title":"Cureus."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"5129","DOI":"10.1007\/s00405-023-08205-4","article-title":"Assessing the accuracy of ChatGPT references in head and neck and ENT disciplines","volume":"280","author":"Frosolini","year":"2023","journal-title":"Eur Arch Otorrinolaringol"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"2120","DOI":"10.1007\/s10439-023-03248-4","article-title":"In reference to \"role of chat GPT in public health\", to highlight the AI'S incorrect reference generation","volume":"51","author":"Frosolini","year":"2023","journal-title":"Ann Biomed Eng"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"301","DOI":"10.3390\/info16040301","article-title":"nBERT: harnessing NLP for emotion recognition in psychotherapy to transform mental health care","volume":"16","author":"Rasool","year":"2025","journal-title":"Information"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"105871","DOI":"10.1016\/j.ijmedinf.2025.105871","article-title":"Evaluating AI-generated patient education materials for spinal surgeries: comparative analysis of readability and DISCERN quality across ChatGPT and deepseek models","volume":"198","author":"Zhou","year":"2025","journal-title":"Int J Med Inform"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"487","DOI":"10.1177\/1062860617751639","article-title":"Readability of online health information: a Meta-narrative systematic review","volume":"33","author":"Daraz","year":"2018","journal-title":"Am J Med Qual"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"e38009","DOI":"10.1097\/MD.0000000000038009","article-title":"How artificial intelligence can provide information about subdural hematoma: assessment of readability, reliability, and quality of ChatGPT, BARD, and perplexity responses","volume":"103","author":"G\u00fcl","year":"2024","journal-title":"Medicine (Baltimore)"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1136\/ard-2024-226202","article-title":"Assessing the performance of AI chatbots in answering patients' common questions about low back pain","volume":"84","author":"Scaff","year":"2025","journal-title":"Ann Rheum Dis"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"862","DOI":"10.1016\/j.oret.2023.05.022","article-title":"Appropriateness and readability of ChatGPT-4-generated responses for surgical treatment of retinal diseases","volume":"7","author":"Momenaei","year":"2023","journal-title":"Ophthalmol Retina"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"105948","DOI":"10.1016\/j.ijmedinf.2025.105948","article-title":"Readability, accuracy and appropriateness and quality of AI chatbot responses as a patient information source on root canal retreatment: a comparative assessment","volume":"201","author":"B\u00fcker","year":"2025","journal-title":"Int J Med Inform"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1007\/s10916-024-02129-0","article-title":"Assessing the efficacy of ChatGPT prompting strategies in enhancing thyroid Cancer patient education: a prospective study","volume":"49","author":"Xu","year":"2025","journal-title":"J Med Syst"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1038\/s41391-024-00826-y","article-title":"Accuracy, readability, and understandability of large language models for prostate cancer information to the public","volume":"28","author":"Hershenhouse","year":"2025","journal-title":"Prostate Cancer Prostatic Dis"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"1077","DOI":"10.1002\/ksa.12133","article-title":"From technical to understandable: artificial intelligence large language models improve the readability of knee radiology reports","volume":"32","author":"Butler","year":"2024","journal-title":"Knee Surg Sports Traumatol Arthrosc"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"1115","DOI":"10.1038\/s41433-024-03476-5","article-title":"Leveraging large language models to improve patient education on dry eye disease","volume":"39","author":"Dihan","year":"2025","journal-title":"Eye (Lond)"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"102090","DOI":"10.3748\/wjg.v31.i6.102090","article-title":"Evaluating large language models as patient education tools for inflammatory bowel disease: a comparative study","volume":"31","author":"Zhang","year":"2025","journal-title":"World J Gastroenterol"}],"container-title":["Frontiers in Public Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2025.1698596\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,10]],"date-time":"2025-12-10T06:40:58Z","timestamp":1765348858000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpubh.2025.1698596\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,10]]},"references-count":33,"alternative-id":["10.3389\/fpubh.2025.1698596"],"URL":"https:\/\/doi.org\/10.3389\/fpubh.2025.1698596","relation":{},"ISSN":["2296-2565"],"issn-type":[{"value":"2296-2565","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,10]]},"article-number":"1698596"}}