{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T20:14:58Z","timestamp":1792700098894,"version":"4.5.0"},"reference-count":27,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2025,1,29]],"date-time":"2025-01-29T00:00:00Z","timestamp":1738108800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS ONE"],"abstract":"\n Objectives<\/jats:title>\n This study aims to evaluate the performance of the latest large language models (LLMs) in answering dental multiple choice questions (MCQs), including both text-based and image-based questions.<\/jats:p>\n <\/jats:sec>\n \n Material and methods<\/jats:title>\n \n A total of 1490 MCQs from two board review books for the United States National Board Dental Examination were selected. This study evaluated six of the latest LLMs as of August 2024, including ChatGPT 4.0 omni (OpenAI), Gemini Advanced 1.5 Pro (Google), Copilot Pro with GPT-4 Turbo (Microsoft), Claude 3.5 Sonnet (Anthropic), Mistral Large 2 (Mistral AI), and Llama 3.1 405b (Meta). \u03c7\n 2<\/jats:sup>\n tests were performed to determine whether there were significant differences in the percentages of correct answers among LLMs for both the total sample and each discipline (p < 0.05).\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Significant differences were observed in the percentage of accurate answers among the six LLMs across text-based questions, image-based questions, and the total sample (p<0.001). For the total sample, Copilot (85.5%), Claude (84.0%), and ChatGPT (83.8%) demonstrated the highest accuracy, followed by Mistral (78.3%) and Gemini (77.1%), with Llama (72.4%) exhibiting the lowest.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Newer versions of LLMs demonstrate superior performance in answering dental MCQs compared to earlier versions. Copilot, Claude, and ChatGPT achieved high accuracy on text-based questions and low accuracy on image-based questions. LLMs capable of handling image-based questions demonstrated superior performance compared to LLMs limited to text-based questions.<\/jats:p>\n <\/jats:sec>\n \n Clinical relevance<\/jats:title>\n Dental clinicians and students should prioritize the most up-to-date LLMs when supporting their learning, clinical practice, and research.<\/jats:p>\n <\/jats:sec>","DOI":"10.1371\/journal.pone.0317423","type":"journal-article","created":{"date-parts":[[2025,1,29]],"date-time":"2025-01-29T13:25:52Z","timestamp":1738157152000},"page":"e0317423","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":51,"title":["Accuracy of latest large language models in answering multiple choice questions in dentistry: A comparative study"],"prefix":"10.1371","volume":"20","author":[{"given":"Huy Cong","family":"Nguyen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hai Phong","family":"Dang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thuy Linh","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Viet","family":"Hoang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2185-6097","authenticated-orcid":true,"given":"Viet Anh","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,1,29]]},"reference":[{"key":"pone.0317423.ref001","doi-asserted-by":"crossref","first-page":"1098","DOI":"10.1111\/jerd.13046","article-title":"Implications of large language models such as ChatGPT for dental medicine","volume":"35","author":"F Eggmann","year":"2023","journal-title":"J Esthet Restor Dent"},{"key":"pone.0317423.ref002","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1038\/s41405-024-00198-4","article-title":"Generative artificial intelligence: synthetic datasets in dentistry","volume":"10","author":"F Umer","year":"2024","journal-title":"BDJ Open"},{"key":"pone.0317423.ref003","doi-asserted-by":"crossref","first-page":"e51344","DOI":"10.2196\/51344","article-title":"Evaluation of ChatGPT\u2019s Real-Life Implementation in Undergraduate Dental Education: Mixed Methods Study","volume":"10","author":"A Kavadella","year":"2024","journal-title":"JMIR Med Educ"},{"key":"pone.0317423.ref004","first-page":"636331","article-title":"Using Large Language Models to Generate Script Concordance Test in Medical Education: ChatGPT and Claude","volume":"6","author":"YS K\u0131yak","year":"2025","journal-title":"Rev Esp Edu Med"},{"key":"pone.0317423.ref005","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.csbj.2023.11.058","article-title":"Beyond the Scalpel: Assessing ChatGPT\u2019s potential as an auxiliary intelligent virtual assistant in oral surgery","volume":"24","author":"A Su\u00e1rez","year":"2024","journal-title":"Comput Struct Biotechnol J"},{"key":"pone.0317423.ref006","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1038\/s41405-024-00226-3","article-title":"Leveraging Large Language Models in the delivery of post-operative dental care: a comparison between an embedded GPT model and ChatGPT","volume":"10","author":"I Batool","year":"2024","journal-title":"BDJ Open"},{"key":"pone.0317423.ref007","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1111\/iej.13985","article-title":"Unveiling the ChatGPT phenomenon: Evaluating the consistency and accuracy of endodontic question answers","volume":"57","author":"A Su\u00e1rez","year":"2024","journal-title":"Int Endod J"},{"key":"pone.0317423.ref008","doi-asserted-by":"crossref","first-page":"104938","DOI":"10.1016\/j.jdent.2024.104938","article-title":"Accuracy and consistency of chatbots versus clinicians for answering pediatric dentistry questions: A pilot study","volume":"144","author":"R Rokhshad","year":"2024","journal-title":"J Dent"},{"key":"pone.0317423.ref009","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1016\/j.identj.2023.12.007","article-title":"Performance of Generative Artificial Intelligence in Dental Licensing Examinations","volume":"74","author":"RCW Chau","year":"2024","journal-title":"Int Dent J"},{"key":"pone.0317423.ref010","doi-asserted-by":"crossref","first-page":"970","DOI":"10.1016\/j.adaj.2023.07.016","article-title":"The performance of artificial intelligence language models in board-style dental knowledge assessment: A preliminary study on ChatGPT","volume":"154","author":"A Danesh","year":"2023","journal-title":"J Am Dent Assoc"},{"key":"pone.0317423.ref011","doi-asserted-by":"crossref","first-page":"881","DOI":"10.1016\/j.ijom.2024.06.003","article-title":"Performance of large language models in oral and maxillofacial surgery examinations","volume":"53","author":"B Quah","year":"2024","journal-title":"Int J Oral Maxillofac Surg"},{"key":"pone.0317423.ref012","first-page":"134","article-title":"ChatGPT\u2019s performance in dentistry and allergy-immunology assessments: a comparative study","author":"A Fuchs","year":"2023","journal-title":"Swiss Dent J"},{"key":"pone.0317423.ref013","doi-asserted-by":"crossref","first-page":"1500","DOI":"10.1016\/j.ijsu.2014.07.014","article-title":"Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration","volume":"12","author":"JP Vandenbroucke","year":"2014","journal-title":"Int J Surg"},{"key":"pone.0317423.ref014","doi-asserted-by":"crossref","first-page":"103610","DOI":"10.1016\/j.jdent.2021.103610","article-title":"Artificial intelligence in dental research: Checklist for Authors, Reviewers, Readers","volume":"107","author":"F Schwendicke","year":"2021","journal-title":"J Dent"},{"key":"pone.0317423.ref015","volume-title":"First Aid Q&A for the NBDE Part I","author":"DM Steinbacher","year":"2008"},{"key":"pone.0317423.ref016","volume-title":"First Aid Q&A for the NBDE Part II","author":"JE Portnof","year":"2010"},{"key":"pone.0317423.ref017","unstructured":"First Aid Q&A for the NBDE Part 1 [Internet] Seattle (WA): Amazon.com; c2008 [2024-12-12]. Available from: https:\/\/www.amazon.com\/First-Aid-NBDE-Part-Pt\/dp\/007150866X\/."},{"key":"pone.0317423.ref018","doi-asserted-by":"crossref","first-page":"107523","DOI":"10.1016\/j.infsof.2024.107523","article-title":"Fine-tuning and prompt engineering for large language models-based code review automation","volume":"175","author":"C Pornprasit","year":"2024","journal-title":"Inf Softw Technol"},{"key":"pone.0317423.ref019","doi-asserted-by":"crossref","first-page":"111734","DOI":"10.1016\/j.jss.2023.111734","article-title":"GitHub Copilot AI pair programmer: Asset or Liability?","volume":"203","author":"A Moradi Dakhel","year":"2023","journal-title":"J Syst Softw"},{"key":"pone.0317423.ref020","article-title":"Currently Available Large Language Models Do Not Provide Musculoskeletal Treatment Recommendations That Are Concordant With Evidence-Based Clinical Practice Guidelines","author":"BU Nwachukwu","journal-title":"Arthrosc\u2014J Arthrosc Relat Surg In Press"},{"key":"pone.0317423.ref021","doi-asserted-by":"crossref","first-page":"e55048","DOI":"10.2196\/55048","article-title":"Exploring the Performance of ChatGPT Versions 3.5, 4, and 4 With Vision in the Chilean Medical Licensing Examination: Observational Study","volume":"10","author":"M Rojas","year":"2024","journal-title":"JMIR Med Educ"},{"key":"pone.0317423.ref022","doi-asserted-by":"crossref","first-page":"e54393","DOI":"10.2196\/54393","article-title":"Capability of GPT-4V(ision) in the Japanese National Medical Licensing Examination: Evaluation Study","volume":"10","author":"T Nakao","year":"2024","journal-title":"JMIR Med Educ"},{"key":"pone.0317423.ref023","doi-asserted-by":"crossref","first-page":"105673","DOI":"10.1016\/j.ijmedinf.2024.105673","article-title":"Evaluating the Effectiveness of advanced large language models in medical Knowledge: A Comparative study using Japanese national medical examination","volume":"193","author":"M Liu","year":"2025","journal-title":"Int J Med Inform"},{"key":"pone.0317423.ref024","doi-asserted-by":"crossref","first-page":"e55627","DOI":"10.2196\/55627","article-title":"Evaluating ChatGPT-4\u2019s Diagnostic Accuracy: Impact of Visual Data Integration","volume":"12","author":"T Hirosawa","year":"2024","journal-title":"JMIR Med Inform"},{"key":"pone.0317423.ref025","article-title":"Legal aspects of generative artificial intelligence and large language models in examinations and theses","volume":"41","author":"M M\u00e4rz","year":"2024","journal-title":"GMS J Med Educ"},{"key":"pone.0317423.ref026","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/0142159X.2024.2385667","article-title":"Is use of ChatGPT cheating? Students of health professions perceptions","author":"AS Kazley","year":"2024","journal-title":"Med Teach"},{"key":"pone.0317423.ref027","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1152\/advan.00181.2023","article-title":"The conversational AI \"ChatGPT\" outperforms medical students on a physiology university examination","volume":"48","author":"CO Soulage","year":"2024","journal-title":"Adv Physiol Educ"}],"container-title":["PLOS ONE"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0317423","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,29]],"date-time":"2025-01-29T13:26:17Z","timestamp":1738157177000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0317423"}},"subtitle":[],"editor":[{"given":"Jinran","family":"Wu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,1,29]]},"references-count":27,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1,29]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0317423","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,29]]}}}