{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:57:03Z","timestamp":1781326623989,"version":"3.54.1"},"reference-count":35,"publisher":"Wiley","license":[{"start":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T00:00:00Z","timestamp":1781222400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"},{"start":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T00:00:00Z","timestamp":1781222400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Eur J Dental Education"],"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>The performance of five popular, widely available large language models (LLMs): ChatGPT\u20104o, Gemini 2.5 Flash, Llama 4, DeepSeek\u2010V3, and Microsoft Copilot in operating dentistry education was evaluated by employing a multiple\u2010choice question\u2010based assessment system.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Material and Methods<\/jats:title>\n                    <jats:p>This was done using a set of 150 MCQs covering areas of endodontics, dental caries, paediatric, preventive, aesthetic and restorative dentistry, biomaterials, and periodontics. The LLM's performance was assessed using classification metrics (accuracy, sensitivity, predictive reliability), textual similarity metrics (BLEU score, cosine similarity, Word Error Rate), and readability metrics (Flesch Reading Ease score).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The highest classification accuracy was achieved by Gemini 2.5 Flash and ChatGPT\u20104o, showing their high sensitivity and high overall predictive reliability. The model with the most textual similarity to the reference answers was ChatGPT\u20104o with BLEU of 0.10\u2009\u00b1\u20090.0279, a high cosine similarity of 0.48\u2009\u00b1\u20090.0422, and a relatively low Word Error Rate (WER) of 5.57\u2009\u00b1\u20090.7301, and a Flesch Reading Ease score of 13.53\u2009\u00b1\u20094.9449.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>In medical education, ChatGPT\u20104o exhibited the highest accuracy, reference textual overlap, semantic alignment, lower number of errors, and readability among the five evaluated LLMs, making it a valuable assistant for dental healthcare professionals.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1111\/eje.70213","type":"journal-article","created":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:20:57Z","timestamp":1781324457000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Exploring the Educational Utility of\n                    <scp>LLMs<\/scp>\n                    in Operative Dentistry Through\n                    <scp>MCQ<\/scp>\n                    \u2010Based Accuracy, Semantic, and Readability Testing"],"prefix":"10.1111","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3354-7956","authenticated-orcid":false,"given":"Shilpa","family":"Bhandi","sequence":"first","affiliation":[{"name":"College of Dental Medicine Roseman University of Health Sciences  South Jordan Utah USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-9216-4773","authenticated-orcid":false,"given":"Adam","family":"Lawlor","sequence":"additional","affiliation":[{"name":"College of Dental Medicine Roseman University of Health Sciences  South Jordan Utah USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5613-2924","authenticated-orcid":false,"given":"Venkata Suresh","family":"Venkataiah","sequence":"additional","affiliation":[{"name":"Clinical Sciences Department, Centre of Medical and Bio\u2010Allied Health Science Research, College of Dentistry Ajman University  Ajman UAE"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jagadish","family":"Hosmani","sequence":"additional","affiliation":[{"name":"Oral Pathology Section, Department of Oral Diagnosis, Oral Biology &amp; Periodontology, College of Dentistry King Khalid University  Abha Asir Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6230-0103","authenticated-orcid":false,"given":"Frank W.","family":"Licari","sequence":"additional","affiliation":[{"name":"College of Dental Medicine Roseman University of Health Sciences  South Jordan Utah USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shankargouda","family":"Patil","sequence":"additional","affiliation":[{"name":"College of Dental Medicine Roseman University of Health Sciences  South Jordan Utah USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2026,6,12]]},"reference":[{"key":"e_1_2_11_2_1","volume-title":"Advanced Operative Dentistry: A Practical Approach","author":"Ricketts D.","year":"2011"},{"key":"e_1_2_11_3_1","doi-asserted-by":"crossref","first-page":"903","DOI":"10.2147\/AMEP.S522300","article-title":"The Technical Quality of Preclinical Tooth Preparation by Undergraduate Dental Students","volume":"16","author":"Almohefer S. A.","year":"2025","journal-title":"Advances in Medical Education and Practice"},{"key":"e_1_2_11_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41415\u2010020\u20101195\u20105"},{"key":"e_1_2_11_5_1","doi-asserted-by":"crossref","first-page":"190","DOI":"10.2174\/1874210601509010190","article-title":"Iatrogenic Damage to the Periodontium Caused by Fixed Prosthodontic Treatment Procedures","volume":"9","author":"Harish P. V.","year":"2015","journal-title":"Open Dentistry Journal"},{"issue":"1","key":"e_1_2_11_6_1","doi-asserted-by":"crossref","DOI":"10.1186\/s12909-024-06525-5","article-title":"Developing Entrustable Professional Activities for Undergraduate Operative Dentistry Clerkship","volume":"24","author":"Sethi A.","year":"2024","journal-title":"BMC Medical Education"},{"issue":"8","key":"e_1_2_11_7_1","article-title":"Dental Education: A Guide for Novice Tutors","volume":"15","author":"Rubaia'an M. A. B.","year":"2023","journal-title":"Cureus"},{"issue":"1","key":"e_1_2_11_8_1","doi-asserted-by":"crossref","DOI":"10.1186\/s12909-023-04698-z","article-title":"Revolutionizing Healthcare: The Role of Artificial Intelligence in Clinical Practice","volume":"23","author":"Alowais S. A.","year":"2023","journal-title":"BMC Medical Education"},{"issue":"1","key":"e_1_2_11_9_1","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1111\/cts.12884","article-title":"Precision Medicine, AI, and the Future of Personalized Health Care","volume":"14","author":"Johnson K. B.","year":"2021","journal-title":"Clinical and Translational Science"},{"key":"e_1_2_11_10_1","doi-asserted-by":"crossref","DOI":"10.3389\/fdmed.2023.1085251","article-title":"Artificial Intelligence in Dentistry\u2014A Review","volume":"4","author":"Ding H.","year":"2023","journal-title":"Frontiers in Dental Medicine"},{"key":"e_1_2_11_11_1","doi-asserted-by":"crossref","DOI":"10.1177\/23333928241234863","article-title":"Artificial Intelligence and Decision\u2010Making in Healthcare: A Thematic Analysis of a Systematic Review of Reviews","volume":"11","author":"Khosravi M.","year":"2024","journal-title":"Health Services Research and Managerial Epidemiology"},{"key":"e_1_2_11_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/eje.13002"},{"issue":"1","key":"e_1_2_11_13_1","doi-asserted-by":"crossref","DOI":"10.1186\/s12903-025-05989-1","article-title":"Artificial Intelligence (AI) in Restorative Dentistry: Current Trends and Future Prospects","volume":"25","author":"Najeeb M.","year":"2025","journal-title":"BMC Oral Health"},{"issue":"2","key":"e_1_2_11_14_1","article-title":"Benchmarking Open\u2010Source Large Language Models, GPT\u20104 and Claude 2 on Multiple\u2010Choice Questions in Nephrology","volume":"1","author":"Wu S.","year":"2024","journal-title":"NEJM Ai"},{"key":"e_1_2_11_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12553\u2010021\u201000574\u20102"},{"key":"e_1_2_11_16_1","volume-title":"Operative Dentistry","author":"Devlin H.","year":"2006"},{"key":"e_1_2_11_17_1","first-page":"3","volume-title":"Applications of Biomedical Engineering in Dentistry","author":"Tabatabaei F. S.","year":"2019"},{"issue":"1","key":"e_1_2_11_18_1","doi-asserted-by":"crossref","DOI":"10.1038\/s41405-024-00277-6","article-title":"Innovation and Application of Large Language Models in Dentistry\u2014A Scoping Review","volume":"10","author":"Umer F.","year":"2024","journal-title":"BDJ Open"},{"issue":"1","key":"e_1_2_11_19_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12903-025-06050-x","article-title":"Evaluation of the Performance of Large Language Models in Clinical Decision\u2010Making in Endodontics","volume":"25","author":"\u00d6zbay Y.","year":"2025","journal-title":"BMC Oral Health"},{"issue":"1","key":"e_1_2_11_20_1","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0317423","article-title":"Accuracy of Latest Large Language Models in Answering Multiple Choice Questions in Dentistry: A Comparative Study","volume":"20","author":"Nguyen H. C.","year":"2025","journal-title":"PLoS One"},{"issue":"12","key":"e_1_2_11_21_1","doi-asserted-by":"crossref","DOI":"10.3390\/jcm14124303","article-title":"AI\u2010Assisted Design of 3D\u2010Printed Lingual Indirect Bonding Trays: A Comparative Evaluation of Bracket Transfer Accuracy","volume":"14","author":"Hoang T. T.","year":"2025","journal-title":"Journal of Clinical Medicine"},{"key":"e_1_2_11_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010025\u201027256\u20107"},{"issue":"2","key":"e_1_2_11_23_1","first-page":"673","article-title":"Evaluation of Large Language Models in Generating Clinically Relevant Dental Board\u2010Style Multiple\u2010Choice Questions","volume":"30","author":"Nguyen T. A.","year":"2025","journal-title":"European Journal of Dental Education"},{"key":"e_1_2_11_24_1","doi-asserted-by":"publisher","DOI":"10.1186\/s12909\u2010024\u201005630\u20109"},{"issue":"2","key":"e_1_2_11_25_1","doi-asserted-by":"crossref","DOI":"10.62347\/UIAP7979","article-title":"DeepSeek\u2010V3 vs ChatGPT\u20104o: A Comparison Study of Their Performance in Answering Prostate Cancer Radiotherapy Questions in Multiple Languages","volume":"13","author":"Luo P. W.","year":"2025","journal-title":"Am J Clin Exp Urol"},{"key":"e_1_2_11_26_1","doi-asserted-by":"publisher","DOI":"10.4103\/jpgm.jpgm_775_24"},{"key":"e_1_2_11_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2023.e20962"},{"key":"e_1_2_11_28_1","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.57348"},{"key":"e_1_2_11_29_1","doi-asserted-by":"publisher","DOI":"10.5041\/RMMJ.10351"},{"key":"e_1_2_11_30_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010022\u201009954\u20108"},{"key":"e_1_2_11_31_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598\u2010024\u201066840\u20101"},{"key":"e_1_2_11_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.surg.2023.05.023"},{"key":"e_1_2_11_33_1","doi-asserted-by":"crossref","DOI":"10.3390\/healthcare11142046","article-title":"ChatGPT\u20104o Knowledge Evaluation in Basic and Clinical Medical Sciences: Multiple Choice Question Examination\u2010Based Performance","volume":"11","author":"Meo S. A.","year":"2023","journal-title":"Health"},{"issue":"4","key":"e_1_2_11_34_1","doi-asserted-by":"crossref","first-page":"985","DOI":"10.17507\/tpls.1404.07","article-title":"An Evaluation of ChatGPT\u20104o's Translation Accuracy Using BLEU Score","volume":"14","author":"Ghassemiazghandi M.","year":"2024","journal-title":"Theory Pract Lang Stud"},{"issue":"2","key":"e_1_2_11_35_1","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s40265-024-02124-2","article-title":"Aligning Large Language Models With Humans: A Comprehensive Survey of ChatGPT\u20104o's Aptitude in Pharmacology","volume":"85","author":"Zhang Y.","year":"2024","journal-title":"Drugs"},{"issue":"1","key":"e_1_2_11_36_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12903-025-05857-y","article-title":"Comparing Diagnostic Skills in Endodontic Cases: Dental Students Versus ChatGPT\u20104o","volume":"25","author":"Durmazpinar P. M.","year":"2025","journal-title":"BMC Oral Health"}],"container-title":["European Journal of Dental Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/eje.70213","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/eje.70213","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/eje.70213","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T04:21:06Z","timestamp":1781324466000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/eje.70213"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,12]]},"references-count":35,"alternative-id":["10.1111\/eje.70213"],"URL":"https:\/\/doi.org\/10.1111\/eje.70213","archive":["Portico"],"relation":{},"ISSN":["1396-5883","1600-0579"],"issn-type":[{"value":"1396-5883","type":"print"},{"value":"1600-0579","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,12]]},"assertion":[{"value":"2025-10-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-06-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eje.70213"}}