{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T11:02:41Z","timestamp":1784545361153,"version":"3.55.0"},"reference-count":26,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T00:00:00Z","timestamp":1784505600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Healthcare"],"abstract":"<jats:p>Background\/Objectives: Patient-facing artificial intelligence chatbots are increasingly used as informal digital health education tools. In orthodontics, eating and drinking advice may directly affect appliance integrity, oral hygiene, enamel demineralization, caries risk, and clear aligner use. This study compared the understandability, actionability, overall quality, and potential harmfulness of responses generated by free and paid versions of ChatGPT and Gemini to patient-oriented orthodontic dietary questions. Methods: This cross-sectional comparative observational study evaluated 160 responses generated from 40 Turkish patient-oriented orthodontic eating and drinking questions across five clinically relevant categories. Each question was submitted separately to ChatGPT free version, ChatGPT Plus, Gemini free version, and Gemini Pro on 1 May 2026, using newly opened independent chat sessions without prompt engineering, follow-up prompts, response regeneration, or manual editing. Responses were anonymized, randomly coded, and independently evaluated by two specialist dentists. PEMAT-P actionability was defined as the primary outcome. PEMAT-P understandability, Global Quality Score, and potentially harmful advice classification were secondary outcomes. Repeated-measures comparisons were performed using Friedman tests and Bonferroni-adjusted Wilcoxon signed-rank tests. Results: PEMAT-P understandability was high across all groups, with median scores of 100.0 in every group. Significant group differences were found for understandability, actionability, and Global Quality Score. ChatGPT Plus achieved the highest actionability score and Global Quality Score and produced no responses classified as potentially harmful. Potentially harmful responses were identified in ChatGPT free version, Gemini free version, and Gemini Pro. For the primary outcome, PEMAT-P actionability, the overall group difference was statistically significant with a small effect size (\u03c72 = 20.527, p &lt; 0.001, Kendall\u2019s W = 0.171), while the largest effect was observed for GQS with a moderate effect size (\u03c72 = 46.520, p &lt; 0.001, Kendall\u2019s W = 0.388). Conclusions: All chatbot groups generated highly understandable responses; however, actionability, overall quality, and safety varied across systems. ChatGPT Plus showed the strongest overall performance under the specific interface, subscription, language, and date conditions tested; however, this finding should be interpreted as a time-specific benchmark rather than evidence that paid chatbot systems are intrinsically safer or more clinically reliable. Structured evaluation, transparent reporting, digital health equity considerations, and professional oversight remain necessary before AI-generated orthodontic dietary advice can be integrated into routine patient education.<\/jats:p>","DOI":"10.3390\/healthcare14142192","type":"journal-article","created":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T09:32:54Z","timestamp":1784539974000},"page":"2192","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Patient Facing AI Chatbots in Digital Orthodontic Education: A Comparative Evaluation of Understandability, Actionability, Quality, and Safety of Dietary Advice"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1077-0768","authenticated-orcid":false,"given":"Neslihan","family":"Karao\u011flan","sequence":"first","affiliation":[{"name":"Dental Clinic, Sultan II. Abd\u00fclhamid Han Training and Research Hospital, University of Health Sciences, 34668 Istanbul, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0052-0973","authenticated-orcid":false,"given":"Hakan","family":"Karao\u011flan","sequence":"additional","affiliation":[{"name":"Department of Pediatric Dentistry, University of Health Sciences, 34668 Istanbul, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s41368-024-00335-7","article-title":"Expert consensus on the prevention and treatment of enamel demineralization in orthodontic treatment","volume":"17","author":"Xia","year":"2025","journal-title":"Int. J. Oral Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1111\/jerd.12013","article-title":"A contemporary review of white spot lesions in orthodontics","volume":"25","author":"Heymann","year":"2013","journal-title":"J. Esthet. Restor. Dent."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Al-Blaihed, D., Alshahrani, I., Alqahtani, N., and Alshahrani, A. (2024). White spot lesions in fixed orthodontics: A literature review on etiology, prevention and treatment. J. Clin. Med., 13.","DOI":"10.7759\/cureus.65679"},{"key":"ref_4","first-page":"227","article-title":"White spot lesions in orthodontics: Prevention and treatment. A descriptive review","volume":"35","author":"Marinelli","year":"2021","journal-title":"J. Biol. Regul. Homeost. Agents"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Charavet, C., Gourdain, Z., Graveline, L., and Lupi, L. (2022). Cleaning and disinfection protocols for clear orthodontic aligners: A systematic review. Healthcare, 10.","DOI":"10.3390\/healthcare10020340"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pardo, A., Signoriello, A., Zangani, A., Messina, E., Gheza, S., Faccioni, P., Albanese, M., and Lombardo, G. (2024). Home biofilm management in orthodontic aligners: A systematic review. Dent. J., 12.","DOI":"10.3390\/dj12100335"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1991","DOI":"10.2147\/PPA.S277034","article-title":"Adherence to dietary advice and oral hygiene practices among orthodontic patients","volume":"14","author":"Aljohani","year":"2020","journal-title":"Patient Prefer. Adherence"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern. Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"e2457879","DOI":"10.1001\/jamanetworkopen.2024.57879","article-title":"Large language models for chatbot health advice studies: A systematic review","volume":"8","author":"Huo","year":"2025","journal-title":"JAMA Netw. Open"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"210","DOI":"10.7326\/M23-2772","article-title":"Large language models in medicine: The potentials and pitfalls: A narrative review","volume":"177","author":"Omiye","year":"2024","journal-title":"Ann. Intern. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"263","DOI":"10.2319\/071123-484.1","article-title":"Content analysis of AI-generated (ChatGPT) responses concerning orthodontic clear aligners","volume":"94","author":"Allareddy","year":"2024","journal-title":"Angle Orthod."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chen, J., Ge, X., Yuan, C., Chen, Y., Li, X., Zhang, X., Chen, S., Zheng, W., and Miao, C. (2025). Comparing orthodontic pre-treatment information provided by large language models. BMC Oral Health, 25.","DOI":"10.1186\/s12903-025-06246-1"},{"key":"ref_13","first-page":"761","article-title":"Assessing the accuracy of AI models in orthodontic knowledge: A comparative study between ChatGPT-4 and Google Bard","volume":"34","author":"Naureen","year":"2024","journal-title":"J. Coll. Physicians Surg. Pak."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Salmanpour, F., Camc\u0131, H., and Geni\u015f, \u00d6. (2025). Comparative analysis of AI chatbot (ChatGPT-4.0 and Microsoft Copilot) and expert responses to common orthodontic questions: Patient and orthodontist evaluations. BMC Oral Health, 25.","DOI":"10.1186\/s12903-025-06194-w"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"100","DOI":"10.2319\/021825-146.1","article-title":"Comparative analysis of artificial intelligence chatbots in orthodontic emergency scenarios: ChatGPT-3.5, ChatGPT-4.0, Copilot, and Gemini","volume":"96","author":"Erdem","year":"2025","journal-title":"Angle Orthod."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"483","DOI":"10.2319\/121424-1021.1","article-title":"Can AI chatbots accurately provide information on orthodontic risks?","volume":"95","author":"Fan","year":"2025","journal-title":"Angle Orthod."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"602","DOI":"10.2319\/031224-207.1","article-title":"How reliable is the artificial intelligence product large language model ChatGPT in orthodontics?","volume":"94","author":"Buyuk","year":"2024","journal-title":"Angle Orthod."},{"key":"ref_18","first-page":"151","article-title":"The Turkish proficiency of ChatGPT and Gemini in orthodontics: An evaluation of the accuracy, completeness, and readability of responses to patient questions","volume":"21","year":"2025","journal-title":"7tepe Klin."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Morales Morillo, M., Iturralde Fern\u00e1ndez, N., Pellicer Castillo, L.D., Suarez, A., Freire, Y., and Diaz-Flores Garc\u00eda, V. (2025). Performance of ChatGPT-4 as an auxiliary tool: Evaluation of accuracy and repeatability on orthodontic radiology questions. Bioengineering, 12.","DOI":"10.3390\/bioengineering12101031"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.pec.2014.05.027","article-title":"Development of the Patient Education Materials Assessment Tool: A new measure of understandability and actionability for print and audiovisual patient information","volume":"96","author":"Shoemaker","year":"2014","journal-title":"Patient Educ. Couns."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yeung, A.W.K. (2025). Evaluation of content quality of online health information by Global Quality Score: A case study of researchers misnaming it and citing secondary sources. Publications, 13.","DOI":"10.3390\/publications13020023"},{"key":"ref_22","first-page":"e65820","article-title":"Evaluation of artificial intelligence as a search tool for patients: Can ChatGPT-4 provide accurate evidence-based orthodontic-related information?","volume":"16","author":"Alkhamees","year":"2024","journal-title":"Cureus"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dursun, D., and Bilici Ge\u00e7er, R. (2024). Can artificial intelligence models serve as patient information consultants in orthodontics?. BMC Med. Inform. Decis. Mak., 24.","DOI":"10.1186\/s12911-024-02619-8"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"S\u00f6kmen, T., Kar, \u0130., Tuncer, C., and Balo\u015f Tuncer, B. (2025). Comparison of the information of generative artificial intelligence large language models and professional guidelines regarding nutritional advice for orthodontic patients. BMC Oral Health, 25.","DOI":"10.1186\/s12903-025-07153-1"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chau, R.C.W., Thu, K.M., Yu, O.Y., Hsung, R.T.C., Wang, D.C.P., Man, M.W.H., Wang, J.J., and Lam, W.Y.H. (2025). Evaluation of chatbot responses to text-based multiple-choice questions in prosthodontic and restorative dentistry. Dent. J., 13.","DOI":"10.3390\/dj13070279"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1016\/j.oooo.2024.12.028","article-title":"Efficacy and empathy of AI chatbots in answering frequently asked questions on oral oncology","volume":"139","author":"Rokhshad","year":"2025","journal-title":"Oral Surg. Oral Med. Oral Pathol. Oral Radiol."}],"container-title":["Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9032\/14\/14\/2192\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T10:06:29Z","timestamp":1784541989000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9032\/14\/14\/2192"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,20]]},"references-count":26,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["healthcare14142192"],"URL":"https:\/\/doi.org\/10.3390\/healthcare14142192","relation":{},"ISSN":["2227-9032"],"issn-type":[{"value":"2227-9032","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,20]]}}}