{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T19:23:55Z","timestamp":1793129035094,"version":"4.5.0"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T00:00:00Z","timestamp":1739318400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Oral. Health"],"abstract":"<jats:sec><jats:title>Objectives<\/jats:title><jats:p>Oral and maxillofacial diseases affect approximately 3.5 billion people worldwide. With the continuous advancement of Artificial Intelligence technologies, particularly the application of generative pre-trained transformers like ChatGPT-4, there is potential to enhance public awareness of the prevention and early detection of these diseases. This study evaluated the performance of ChatGPT-4 in addressing oral and maxillofacial disease questions using standard approaches and the Chain of Thought (CoT) method, aiming to gain a deeper understanding of its capabilities, potential, and limitations.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and methods<\/jats:title><jats:p>Three experts, drawing from their extensive experience and the most common questions in clinical settings, selected 130 open-ended questions and 1,805 multiple-choice questions from the national dental licensing examination. These questions encompass 12 areas of oral and maxillofacial surgery, including Prosthodontics, Pediatric Dentistry, Maxillofacial Tumors and Salivary Gland Diseases, and maxillofacial Infections.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Using CoT approach, ChatGPT-4 exhibited marked enhancements in accuracy, structure, completeness, professionalism, and overall impression for open-ended questions, revealing statistically significant differences compared to its performance on general oral and maxillofacial inquiries. In the realm of multiple-choice questions, the application of CoT method boosted ChatGPT-4's accuracy across all major subjects, achieving an overall accuracy increase of 3.1%.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>When employing ChatGPT-4 to address questions in oral and maxillofacial surgery, incorporating CoT as a querying method can enhance its performance and help the public improve their understanding and awareness of such issues. However, it is not advisable to consider it a substitute for doctors.<\/jats:p><\/jats:sec>","DOI":"10.3389\/froh.2025.1541976","type":"journal-article","created":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T07:26:18Z","timestamp":1739345178000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Evaluating ChatGPT-4's performance on oral and maxillofacial queries: Chain of Thought and standard method"],"prefix":"10.3389","volume":"6","author":[{"given":"Kaiyuan","family":"Ji","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhihan","family":"Wu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jing","family":"Han","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guangtao","family":"Zhai","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiannan","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,2,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1111\/eje.12937","article-title":"ChatGPT\u2014a double-edged sword for healthcare education? Implications for assessments of dental students","volume":"28","author":"Ali","year":"2024","journal-title":"Eur J Dent Educ"},{"key":"B2","doi-asserted-by":"publisher","first-page":"101471","DOI":"10.1016\/j.jormas.2023.101471","article-title":"Can ChatGPT be used in oral and maxillofacial surgery?","volume":"124","author":"Balel","year":"2023","journal-title":"J Stomatol Oral Maxillofac Surg"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1909","DOI":"10.1016\/S0140-6736(22)02322-4","article-title":"WHO calls to end the global crisis of oral health","volume":"400","author":"Benzian","year":"2022","journal-title":"Lancet"},{"key":"B4","doi-asserted-by":"publisher","first-page":"703526","DOI":"10.3389\/froh.2021.703526","article-title":"Global perspectives of oral health policies and oral healthcare schemes for older adult populations","volume":"2","author":"Jiang","year":"2021","journal-title":"Front Oral Health"},{"key":"B5","doi-asserted-by":"publisher","first-page":"572","DOI":"10.1136\/bmj.314.7080.572","article-title":"Statistics notes: Cronbach\u2019s alpha","volume":"314","author":"Bland","year":"1997","journal-title":"Br Med J"},{"key":"B6","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1007\/s10916-023-01925-4","article-title":"Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios","volume":"47","author":"Cascella","year":"2023","journal-title":"J Med Syst"},{"key":"B7","doi-asserted-by":"publisher","first-page":"616","DOI":"10.1016\/j.identj.2023.12.007","article-title":"Performance of generative artificial intelligence in dental licensing examinations","volume":"74","author":"Chau","year":"2024","journal-title":"Int Dent J"},{"key":"B8","doi-asserted-by":"publisher","first-page":"864","DOI":"10.1186\/s12909-023-04832-x","article-title":"Assessment of the capacity of ChatGPT as a self-learning tool in medical pharmacology: a study using MCQs","volume":"23","author":"Choi","year":"2023","journal-title":"BMC Med Educ"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1482334","DOI":"10.3389\/froh.2024.1482334","article-title":"An overview of artificial intelligence based automated diagnosis in paediatric dentistry","volume":"5","author":"Rajinikanth","year":"2024","journal-title":"Front Oral Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1007\/BF02310555","article-title":"Coefficient alpha and the internal structure of tests","volume":"16","author":"Cronbach","year":"1951","journal-title":"Psychometrika"},{"key":"B11","doi-asserted-by":"publisher","first-page":"635","DOI":"10.1016\/S0140-6736(24)00536-1","article-title":"The ongoing fight for population oral health","volume":"404","author":"Guarnizo-Herre\u00f1o","year":"2024","journal-title":"Lancet"},{"key":"B12","doi-asserted-by":"publisher","first-page":"105213","DOI":"10.1016\/j.jdent.2024.105213","article-title":"The impact of oral diseases on respiratory health and the influence of respiratory infections on the oral microbiome","volume":"148","author":"Gupta","year":"2024","journal-title":"J Dent"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1865","DOI":"10.1002\/jdd.13294","article-title":"Generative AI use in dental education: efficient exam item writing","volume":"87","author":"Johnson","year":"2023","journal-title":"J Dent Educ"},{"key":"B14","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1002\/ase.2270","article-title":"The rise of ChatGPT: exploring its potential in medical education","volume":"17","author":"Lee","year":"2024","journal-title":"Anat Sci Educ"},{"key":"B15","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1111\/ipd.13098","article-title":"Global bibliometric mapping of dental development research for the period 2012\u20132021","volume":"34","author":"Li","year":"2024","journal-title":"Int J Paediatr Dent"},{"key":"B16","doi-asserted-by":"publisher","first-page":"159","DOI":"10.36106\/gjra\/7112189","article-title":"Applications of ChatGPT in oral healthcare: a current and brief overview","volume":"12","author":"Mago","year":"2023","journal-title":"Glob J Res Anal"},{"key":"B17","doi-asserted-by":"publisher","first-page":"3449","DOI":"10.3390\/ijerph19063449","article-title":"Artificial intelligence in dentistry\u2014narrative review","volume":"19","author":"Ossowska","year":"2022","journal-title":"Int J Environ Res Public Health"},{"key":"B18","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/S0140-6736(19)31146-8","article-title":"Oral diseases: a global public health challenge","volume":"394","author":"Peres","year":"2019","journal-title":"Lancet"},{"key":"B19","doi-asserted-by":"publisher","first-page":"104938","DOI":"10.1016\/j.jdent.2024.104938","article-title":"Accuracy and consistency of chatbots versus clinicians for answering pediatric dentistry questions: a pilot study","volume":"144","author":"Rokhshad","year":"2024","journal-title":"J Dent"},{"key":"B20","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1177\/0022034520915714","article-title":"Artificial intelligence in dentistry: chances and challenges","volume":"99","author":"Schwendicke","year":"2020","journal-title":"J Dent Res"},{"key":"B21","doi-asserted-by":"publisher","first-page":"1315663","DOI":"10.3389\/froh.2023.1315663","article-title":"Education in oral health","volume":"4","author":"Javed","year":"2023","journal-title":"Front Oral Health"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1273","DOI":"10.1007\/s11165-016-9602-2","article-title":"The use of Cronbach\u2019s alpha when developing and reporting research instruments in science education","volume":"48","author":"Taber","year":"2018","journal-title":"Res Sci Educ"},{"key":"B23","doi-asserted-by":"publisher","first-page":"1269","DOI":"10.3390\/healthcare10071269","article-title":"Where is the artificial intelligence applied in dentistry? Systematic review and literature analysis","volume":"10","author":"Thurzo","year":"2022","journal-title":"Healthcare"},{"key":"B24","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1002\/9781119790686.ch11","article-title":"AI In dentistry","volume-title":"AI in Clinical Medicine: A Practical Guide for Healthcare Professionals","author":"Tuzova","year":"2023"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1602","DOI":"10.21037\/qims-23-1180","article-title":"Assessing the role of GPT-4 in thyroid ultrasound diagnosis and treatment recommendations: enhancing interpretability with a chain of thought approach","volume":"14","author":"Wang","year":"2024","journal-title":"Quant Imaging Med Surg"},{"key":"B26","doi-asserted-by":"publisher","first-page":"104727","DOI":"10.1016\/j.jdent.2023.104727","article-title":"Advancements in oral and maxillofacial surgery medical image segmentation techniques: an overview","volume":"138","author":"Zhang","year":"2023","journal-title":"J Dent"},{"key":"B27","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1186\/s12903-023-03027-6","article-title":"Artificial intelligence in the diagnosis of dental diseases on panoramic radiographs: a preliminary study","volume":"23","author":"Zhu","year":"2023","journal-title":"BMC Oral Health"},{"key":"B28","doi-asserted-by":"publisher","first-page":"e48291","DOI":"10.2196\/48291","article-title":"Large language models in medical education: opportunities, challenges, and future directions","volume":"9","author":"Abd-Alrazaq","year":"2023","journal-title":"JMIR Med Educ"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.38317","article-title":"ChatGPT in dentistry: a comprehensive review","volume":"15","author":"Alhaidry","year":"2023","journal-title":"Cureus"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1368121","DOI":"10.3389\/froh.2024.1368121","article-title":"Dental education and practice: past, present, and future trends","volume":"5","author":"Spielman","year":"2024","journal-title":"Front Oral Health"},{"key":"B31","doi-asserted-by":"publisher","first-page":"936","DOI":"10.1016\/j.surg.2023.12.014","article-title":"Evaluating capabilities of large language models: performance of GPT-4 on surgical knowledge assessments","volume":"175","author":"Beaulieu-Jones","year":"2024","journal-title":"Surgery"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1098","DOI":"10.1111\/jerd.13046","article-title":"Implications of large language models such as ChatGPT for dental medicine","volume":"35","author":"Eggmann","year":"2023","journal-title":"J Esthet Restor Dent"},{"key":"B33","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","year":"2023","journal-title":"PLoS Digital Health"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.50369","article-title":"The performance of GPT-3.5, GPT-4, and bard on the Japanese national dentist examination: a comparison study","volume":"15","author":"Ohta","year":"2023","journal-title":"Cureus"},{"key":"B35","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"B36","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1111\/iej.13985","article-title":"Unveiling the ChatGPT phenomenon: evaluating the consistency and accuracy of endodontic question answers","volume":"57","author":"Su\u00e1rez","year":"2024","journal-title":"Int Endod J"}],"container-title":["Frontiers in Oral Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/froh.2025.1541976\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T07:26:21Z","timestamp":1739345181000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/froh.2025.1541976\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,12]]},"references-count":36,"alternative-id":["10.3389\/froh.2025.1541976"],"URL":"https:\/\/doi.org\/10.3389\/froh.2025.1541976","relation":{},"ISSN":["2673-4842"],"issn-type":[{"value":"2673-4842","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,12]]},"article-number":"1541976"}}