{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T17:13:07Z","timestamp":1788455587584,"version":"build-2803163510"},"reference-count":16,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T00:00:00Z","timestamp":1787875200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100020912","name":"King Faisal University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100020912","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"\n Background<\/jats:title>\n Patients with diabetes increasingly consult artificial intelligence (AI) chatbots for medical advice, including guidance on antidiabetic medication management during Ramadan fasting, because AI can simplify and summarize long, complex guidelines. Also, in hospital settings, these tools are being used in hospitals much faster than it takes to establish formal regulations and guidelines for their use. Evaluations of the accuracy, completeness, and reproducibility of such advice across languages are still lacking. Therefore, the study aims to evaluate and compare the accuracy, completeness, safety, and reproducibility of three widely used AI chatbots\u2014ChatGPT, Google Gemini, and Microsoft Copilot\u2014when providing antidiabetic medication adjustment advice during Ramadan in both English and Arabic.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Twenty-three standardized clinical scenarios covering common antidiabetic regimens were presented to each chatbot in both English and Arabic. Each query was repeated to evaluate reproducibility, resulting in 276 responses scored. Responses were assessed against the International Diabetes Federation\u2013Diabetes and Ramadan (IDF-DAR) Guidelines using a 0\u20132 accuracy scale, a 0\u20134 completeness scale, and a 0\u20133 safety scale.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Overall, 77% of responses were fully consistent with the guideline, 12% were partially consistent, and 11% (30\/276) contained clinically harmful or contradictory advice; harmful responses were about twice as common in Arabic as in English (14% vs. 8%). Completeness and safety were high, with medians at the observed ceiling. In the generalized linear mixed models, chatbots did not differ significantly in accuracy, completeness, or safety, and there was no significant main effect of language or chatbot \u00d7 language interaction; the strongest signals were a chatbot effect on completeness (\n p<\/jats:italic>\n = 0.068) and a language effect on safety (\n p<\/jats:italic>\n = 0.064), both non-significant. Two-week reproducibility was fair for accuracy (weighted \u03ba = 0.20,\n p<\/jats:italic>\n = 0.009) and completeness (\u03ba = 0.29,\n p<\/jats:italic>\n = 0.001) and showed a very low \u03ba in the safety scale (\u03ba = 0.02,\n p<\/jats:italic>\n = 0.81).\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n AI chatbots demonstrated comparable performance in delivering guideline-based advice for diabetes management during Ramadan, with no significant differences in accuracy, completeness, or safety. While most responses aligned with the IDF-DAR guideline, some harmful recommendations persisted, and response consistency fluctuated over time. These results suggest that AI chatbots should serve as a supplementary resource rather than a substitute for professional medical advice.<\/jats:p>\n <\/jats:sec>","DOI":"10.3389\/fmed.2026.1934432","type":"journal-article","created":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T12:53:41Z","timestamp":1787921621000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["AI Chatbots as a source of Ramadan medication management advice for patients with diabetes: a multilingual comparative evaluation"],"prefix":"10.3389","volume":"13","author":[{"given":"Sufyan","family":"Alomair","sequence":"first","affiliation":[{"name":"Department of Pharmacy Practice, College of Clinical Pharmacy, King Faisal University","place":["Al-Ahsaa, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maryam","family":"Alsuwayq","sequence":"additional","affiliation":[{"name":"Department of Pharmacy Practice, College of Clinical Pharmacy, King Faisal University","place":["Al-Ahsaa, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Walaa","family":"Alabbad","sequence":"additional","affiliation":[{"name":"Department of Pharmacy Practice, College of Clinical Pharmacy, King Faisal University","place":["Al-Ahsaa, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khawlah","family":"Albeladi","sequence":"additional","affiliation":[{"name":"Department of Pharmacy Practice, College of Clinical Pharmacy, King Faisal University","place":["Al-Ahsaa, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zainab","family":"Alhassan","sequence":"additional","affiliation":[{"name":"Department of Pharmacy Practice, College of Clinical Pharmacy, King Faisal University","place":["Al-Ahsaa, Saudi Arabia"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,28]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"2306","DOI":"10.2337\/diacare.27.10.2306","article-title":"A population-based study of diabetes and its characteristics during the fasting month of ramadan in 13 countries","volume":"27","author":"Salti","year":"2004","journal-title":"Diabetes Care"},{"key":"B2","unstructured":"Hassanein\n MAAMHO\n \n \n Afandi\n B\n \n \n Ahmedani\n MY\n \n \n Alamoudi\n RM\n \n \n Alawadi\n F\n \n \n Bajaj\n HS\n \n \n Diabetes and Ramadan Diabetes and Ramadan International Diabetes Federation (IDF), in collaboration with the Diabetes and Ramadan (DAR) International Alliance\n \n 2021"},{"key":"B3","doi-asserted-by":"publisher","first-page":"9","DOI":"10.3389\/fnut.2022.846600","article-title":"Systematic review of insulin management recommendations to improve glycemic control and reduce hypoglycemic events during ramadan fasting in patients with insulin-requiring type 2 diabetes","volume":"12","author":"Kieu","year":"2022","journal-title":"Front Nutr."},{"key":"B4","doi-asserted-by":"publisher","first-page":"1895","DOI":"10.2337\/dc10-0896","article-title":"Recommendations for management of diabetes during Ramadan","volume":"33","author":"Al-Arouj","year":"2010","journal-title":"Diabetes Care"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e47564","DOI":"10.2196\/47564","article-title":"User intentions to use ChatGPT for self-diagnosis and health-related purposes: cross-sectional survey study","volume":"10","author":"Shahsavar","year":"2023","journal-title":"JMIR Hum Factors."},{"key":"B6","doi-asserted-by":"publisher","first-page":"37","DOI":"10.3390\/biomedinformatics5030037","article-title":"Generative artificial intelligence in healthcare: applications, implementation challenges, and future directions","volume":"5","author":"Rabbani","year":"2025","journal-title":"BioMedInformatics"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.64898\/2026.01.01.25342468","article-title":"Are older adults using ChatGPT for medical advice? Results from a cross-sectional survey study","author":"Turchioe","year":"","journal-title":"medRxiv."},{"key":"B8","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1186\/s12911-024-02824-5","article-title":"Assessing the accuracy and quality of artificial intelligence (AI) chatbot-generated responses in making patient-specific drug-therapy and healthcare-related decisions","volume":"24","author":"Shiferaw","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"key":"B9","doi-asserted-by":"publisher","first-page":"6421","DOI":"10.3390\/app11146421","article-title":"What disease does this patient have? A large-scale open domain question answering dataset from medical exams","volume":"11","author":"Jin","year":"2021","journal-title":"Appl Sci."},{"key":"B10","doi-asserted-by":"publisher","first-page":"e2336483","DOI":"10.1001\/jamanetworkopen.2023.36483","article-title":"Accuracy and reliability of chatbot responses to physician questions","volume":"6","author":"Goodman","year":"2023","journal-title":"JAMA Netw Open."},{"key":"B11","doi-asserted-by":"publisher","first-page":"619","DOI":"10.1007\/s13410-024-01401-w","article-title":"Evaluating the accuracy and adequacy of ChatGPT in responding to queries of diabetes patients in primary healthcare","volume":"45","author":"Senoymak","year":"2025","journal-title":"Int J Diabetes Dev Ctries."},{"key":"B12","doi-asserted-by":"publisher","first-page":"4419","DOI":"10.3390\/jcm15124419","article-title":"Assessing the accuracy and precision of artificial intelligence for diabetes mellitus and hypertension management","volume":"15","author":"Al Hamid","year":"2026","journal-title":"J Clin Med"},{"key":"B13","doi-asserted-by":"publisher","first-page":"958","DOI":"10.1016\/j.cegh.2020.03.004","article-title":"The frequency of major complications among fasting people with diabetes according to risk categories of IDF-DAR guidelines 2016","volume":"8","author":"Iqbal","year":"2020","journal-title":"Clin Epidemiol Glob Health."},{"key":"B14","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1186\/s12879-024-09725-y","article-title":"Language discrepancies in the performance of generative artificial intelligence models: an examination of infectious disease queries in English and Arabic","volume":"24","author":"Sallam","year":"2024","journal-title":"BMC Infect Dis"},{"key":"B15","doi-asserted-by":"publisher","first-page":"7810","DOI":"10.3390\/jcm13247810","article-title":"Comparison of the accuracy, completeness, reproducibility, and consistency of different AI Chatbots in providing nutritional advice: an exploratory study","volume":"13","author":"Ponzo","year":"2024","journal-title":"J Clin Med."},{"key":"B16","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1038\/s41746-024-01029-4","article-title":"Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs","volume":"7","author":"Wang","year":"2024","journal-title":"NPJ Digit Med."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1934432\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,3]],"date-time":"2026-09-03T16:15:13Z","timestamp":1788452113000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2026.1934432\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,28]]},"references-count":16,"alternative-id":["10.3389\/fmed.2026.1934432"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2026.1934432","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,28]]},"article-number":"1934432"}}