{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:43:41Z","timestamp":1785545021109,"version":"3.56.0"},"reference-count":43,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Diabetes"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Type 2 diabetes mellitus (T2DM) affects approximately 590 million people worldwide, and its management relies heavily on patient education. With the emergence of online health information and artificial intelligence (AI) large language models, patients are increasingly sourcing medical information independently.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study compared the quality, readability, and transparency of websites and AI-generated leaflets (AIGLs) related to T2DM.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n Four predefined search terms (\u201ctype 2 diabetes,\u201d \u201ctype 2 diabetes mellitus,\u201d \u201cT2DM,\u201d and \u201cadult diabetes\u201d) were entered into 3 major search engines (Google, Yahoo, and Bing), and the top 20 search results were retrieved. AIGLs with patient information about T2DM were produced using a standardized prompt in 4 AI large language models (ChatGPT, Gemini, DeepSeek, and Grok). Information quality was assessed using the DISCERN score, calculated by 3 independent raters and ChatGPT. The\n Journal of the American Medical Association (JAMA)<\/jats:italic>\n benchmarks were used to measure reliability and transparency. The Flesch-Kincaid Grade Level was used to determine readability.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Seventy-five websites and 4 AIGLs were evaluated. Mean author-rated DISCERN scores were 42.6 (SD 11.3) for websites and 43.9 (SD 1.74) for AIGLs, corresponding to fair quality (DISCERN 41\u201051). In contrast, ChatGPT-rated mean DISCERN scores were higher, with 58.5 (SD 11.5) for websites and 61.0 (SD 2.94) for AIGLs, corresponding to good quality (DISCERN 52\u201063). Mean\n JAMA<\/jats:italic>\n benchmark scores were 2.74 (SD 0.965) for websites, indicating moderate reliability (2-3 out of 4 points), whereas all AIGLs scored 0 out of 4 points. Mean Flesch-Kincaid Grade Level scores for websites were 8.67 (SD 2.23) and 8.30 (SD 1.92) for AIGLs, corresponding to an eighth- to ninth-grade comprehension level. Spearman rank correlation demonstrated minimal variability among the 3 independent raters but showed a significant difference between ChatGPT and the 3 independent raters.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Given the high prevalence of T2DM, both websites and AIGLs demonstrated suboptimal quality, readability, and transparency. Increasing patient reliance on digital health information calls for improved readability standards and stronger safeguards for AI-generated content. Both websites and AIGLs require an eighth- to ninth-grade comprehension level, far above the average reading age of 9 years in the United Kingdom (fourth- to fifth-grade level). This reduces the accessibility of online health information. The landscape of medical consultations is evolving, with patients increasingly presenting with preconceived notions based on online health information; hence, health care professionals should adapt to this shift.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/93822","type":"journal-article","created":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T08:20:06Z","timestamp":1780647606000},"page":"e93822-e93822","source":"Crossref","is-referenced-by-count":0,"title":["Assessing Quality, Readability, and Transparency of Online and AI-Generated Information on Type 2 Diabetes Mellitus: Cross-Sectional Exploratory Study"],"prefix":"10.2196","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6176-1853","authenticated-orcid":false,"given":"Sripradha","family":"Srinivasan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8012-0322","authenticated-orcid":false,"given":"Ravanth","family":"Baskaran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4419-8808","authenticated-orcid":false,"given":"Srinjay","family":"Mukhopadhyay","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6609-5394","authenticated-orcid":false,"given":"Mayank","family":"Patel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,15]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.mpmed.2018.10.010","article-title":"What is type 2 diabetes?","volume":"47","author":"Hurtado","journal-title":"Medicine"},{"key":"R2","unstructured":"Type 2 diabetes. International Diabetes Federation. URL: https:\/\/idf.org\/about-diabetes\/types-of-diabetes\/type-2\/ [Accessed 18-12-2025]"},{"issue":"9933","key":"R3","doi-asserted-by":"publisher","first-page":"1999","DOI":"10.1016\/S0140-6736(14)60613-9","article-title":"Prevention and management of type 2 diabetes: dietary components and nutritional strategies","volume":"383","author":"Ley","journal-title":"Lancet"},{"key":"R4","unstructured":"Type 2 diabetes in adults: management. National Institute for Health and Care Excellence. URL: https:\/\/www.nice.org.uk\/guidance\/ng28\/chapter\/Recommendations#individualised-care [Accessed 08-03-2023]"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s00125-021-05585-2","article-title":"Interpreting global trends in type 2 diabetes complications and mortality","volume":"65","author":"Ali","journal-title":"Diabetologia"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1016\/j.diabres.2019.04.004","article-title":"The role of education in type 2 diabetes treatment","volume":"151","author":"\u015awi\u0105toniowska","journal-title":"Diabetes Res Clin Pract"},{"issue":"12","key":"R7","doi-asserted-by":"publisher","first-page":"1740","DOI":"10.3390\/healthcare9121740","article-title":"Online health information seeking behavior: a systematic review","volume":"9","author":"Jia","journal-title":"Healthcare (Basel)"},{"issue":"5","key":"R8","doi-asserted-by":"publisher","DOI":"10.2196\/32723","article-title":"Disparity and factors associated with internet health information seeking among US adults living with diabetes mellitus: cross-sectional study","volume":"24","author":"Eke","journal-title":"J Med Internet Res"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2023.1211285","article-title":"Editorial: digital information for patient education","volume":"11","author":"Guo","journal-title":"Front Public Health"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","first-page":"1302785","DOI":"10.1080\/23311886.2017.1302785","article-title":"Health information seeking in the digital age: an analysis of health information seeking behavior among US adults","volume":"3","author":"Jacobs","journal-title":"Cogent Soc Sci"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1038\/s44401-025-00032-5","article-title":"A systematic review of early evidence on generative AI for drafting responses to patient messages","volume":"2","author":"Hu","journal-title":"NPJ Health Syst"},{"key":"R12","doi-asserted-by":"publisher","DOI":"10.2196\/72553","article-title":"Generative AI chatbot for diabetes management: formative 2-part qualitative study using DTalksBot involving patients and clinicians","volume":"9","author":"Jeon","journal-title":"JMIR Form Res"},{"key":"R13","unstructured":"Kim Y Jeong H Chen S et al. Medical hallucinations in foundation models and their impact on healthcare. arXiv. Preprint posted online on Feb 26, 2025. [doi: 10.48550\/arXiv.2503.05777]"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01413-0","article-title":"Quality and readability of web-based Arabic health information on periodontal disease","volume":"21","author":"Al-Ak\u2019hali","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","first-page":"9","DOI":"10.3390\/informatics12010009","article-title":"Evaluating the reliability of ChatGPT for health-related questions: a systematic review","volume":"12","author":"Beheshti","journal-title":"Informatics"},{"issue":"2","key":"R16","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1136\/jech.53.2.105","article-title":"DISCERN: an instrument for judging the quality of written consumer health information on treatment choices","volume":"53","author":"Charnock","journal-title":"J Epidemiol Community Health"},{"issue":"9","key":"R17","doi-asserted-by":"publisher","first-page":"3214","DOI":"10.3390\/app10093214","article-title":"Evaluation of quality and readability of online health information on high blood pressure using DISCERN and Flesch-Kincaid tools","volume":"10","author":"Tahir","journal-title":"Appl Sci"},{"key":"R18","doi-asserted-by":"publisher","DOI":"10.2196\/22769","article-title":"Applications and concerns of ChatGPT and other conversational large language models in health care: systematic review","volume":"26","author":"Wang","journal-title":"J Med Internet Res"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"84","DOI":"10.4103\/1357-6283.210517","article-title":"Assessing reading levels of health information: uses and limitations of Flesch formula","volume":"30","author":"Jindal","journal-title":"Educ Health (Abingdon)"},{"key":"R20","unstructured":"Flesch Kincaid calculator. Good Calculators. URL: https:\/\/goodcalculators.com\/flesch-kincaid-calculator\/ [Accessed 20-12-2025]"},{"key":"R21","unstructured":"Journal of American Medical Association (JAMA) benchmark criteria. ResearchGate. URL: https:\/\/www.researchgate.net\/figure\/Journal-of-American-Medical-Association-JAMA-benchmark-criteria_tbl1_339976668 [Accessed 20-12-2025]"},{"key":"R22","unstructured":"Liu J He J . The decoy dilemma in online medical information evaluation: a comparative study of credibility assessments by LLM and human judges. arXiv. Preprint posted online on Nov 23, 2024. [doi: 10.48550\/arXiv.2411.15396]"},{"key":"R23","doi-asserted-by":"crossref","unstructured":"Ahmad MA Yaramic I Dutta Roy T . Creating trustworthy LLMs: dealing with hallucinations in healthcare AI. arXiv. Preprint posted online on Sep 26, 2023. [doi: 10.48550\/arXiv.2311.01463]","DOI":"10.20944\/preprints202310.1662.v1"},{"key":"R24","unstructured":"AI hallucinates more frequently as it gets more advanced \u2014 is there any way to stop it from happening, and should we even try? Live Science. 2025. URL: https:\/\/tinyurl.com\/ycyt3zv7 [Accessed 04-01-2026]"},{"key":"R25","unstructured":"OpenAI o3 and o4-mini system card. OpenAI. 2025. URL: https:\/\/cdn.openai.com\/pdf\/2221c875-02dc-4789-800b-e7758f3722c1\/o3-and-o4-mini-system-card.pdf [Accessed 04-01-2026]"},{"key":"R26","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-025-11454-w","article-title":"Hallucination to truth: a review of fact-checking and factuality evaluation in large language models","volume":"59","author":"Rahman","journal-title":"Artif Intell Rev"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/66917","article-title":"Benchmarking the confidence of large language models in answering clinical questions: cross-sectional evaluation study","volume":"13","author":"Omar","journal-title":"JMIR Med Inform"},{"key":"R28","doi-asserted-by":"crossref","unstructured":"Lim KH Kang U Li X et al. Susceptibility of large language models to user-driven factors in medical queries. arXiv. Preprint posted online on Mar 26, 2025. [doi: 10.48550\/arXiv.2503.22746]","DOI":"10.2139\/ssrn.5181657"},{"issue":"2","key":"R29","doi-asserted-by":"publisher","DOI":"10.1093\/jncics\/pkad010","article-title":"Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift","volume":"7","author":"Hopkins","journal-title":"JNCI Cancer Spectr"},{"issue":"10","key":"R30","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.46662","article-title":"How efficient is ChatGPT in accessing accurate and quality health-related information?","volume":"15","author":"Ulusoy","journal-title":"Cureus"},{"issue":"6","key":"R31","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.86543","article-title":"The reliability gap: how traditional search engines outperform artificial intelligence (AI) chatbots in rosacea public health information quality","volume":"17","author":"Nelson","journal-title":"Cureus"},{"issue":"6","key":"R32","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000877","article-title":"Retrieval augmented generation for large language models in healthcare: a systematic review","volume":"4","author":"Amugongo","journal-title":"PLOS Digit Health"},{"issue":"3","key":"R33","doi-asserted-by":"publisher","DOI":"10.3390\/medicina60030445","article-title":"Integrating retrieval-augmented generation with large language models in nephrology: advancing practical applications","volume":"60","author":"Miao","journal-title":"Medicina (Kaunas)"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.2196\/75262","article-title":"Development and evaluation of a retrieval-augmented generation chatbot for orthopedic and trauma surgery patient education: mixed-methods study","volume":"4","author":"Baur","journal-title":"JMIR AI"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.2196\/58041","article-title":"Enhancement of the performance of large language models in diabetes education through retrieval-augmented generation: comparative study","volume":"26","author":"Wang","journal-title":"J Med Internet Res"},{"key":"R36","unstructured":"Reading level for web pages. Surrey County Council. URL: https:\/\/www.surreycc.gov.uk\/website\/writing-guide\/reading-levels-for-web-pages [Accessed 08-01-2026]"},{"key":"R37","unstructured":"The importance of adult literacy levels for elearning. What You Need to Know. URL: https:\/\/whatyouneedtoknow.co.uk\/adult-literacy-and-elearning\/ [Accessed 08-01-2026]"},{"key":"R38","doi-asserted-by":"publisher","DOI":"10.1016\/j.wneu.2021.06.140","article-title":"Current status of websites offering information to patients with traumatic brain injury and caregivers: time for reform?","volume":"153","author":"Manivannan","journal-title":"World Neurosurg"},{"key":"R39","unstructured":"Tackling Inequality Commission. Diabetes UK. URL: https:\/\/www.diabetes.org.uk\/for-professionals\/improving-care\/tackling-inequality-commission [Accessed 27-06-2026]"},{"issue":"2","key":"R40","doi-asserted-by":"publisher","DOI":"10.1093\/cdn\/nzab002","article-title":"Using the Google\u2122 search engine for health information: is there a problem? Case study: supplements for cancer","volume":"5","author":"Cai","journal-title":"Curr Dev Nutr"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1080\/20905998.2025.2602411","article-title":"Patient information on ureteric stents; online resources versus artificial intelligence generated content: a benchmarking study from EAU endourology","author":"Baskaran","journal-title":"Arab J Urol"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","DOI":"10.1186\/s12913-016-1691-0","article-title":"Social media use in healthcare: a systematic review of effects on patients and on their relationship with healthcare professionals","volume":"16","author":"Smailhodzic","journal-title":"BMC Health Serv Res"},{"issue":"12","key":"R43","doi-asserted-by":"publisher","first-page":"1294","DOI":"10.15537\/smj.2019.12.24682","article-title":"Health information on social media. Perceptions, attitudes, and practices of patients and their companions","volume":"40","author":"Sumayyia","journal-title":"Saudi Med J"}],"container-title":["JMIR Diabetes"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/diabetes.jmir.org\/2026\/1\/e93822","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:23:37Z","timestamp":1785536617000},"score":1,"resource":{"primary":{"URL":"https:\/\/diabetes.jmir.org\/2026\/1\/e93822"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,15]]},"references-count":43,"URL":"https:\/\/doi.org\/10.2196\/93822","relation":{},"ISSN":["2371-4379"],"issn-type":[{"value":"2371-4379","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,15]]},"article-number":"v11i1e93822"}}