{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T13:08:05Z","timestamp":1790168885343,"version":"4.1.0"},"reference-count":29,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Clin. Med.","JCM"],"abstract":"<jats:p>Background\/Objectives: Large language models (LLMs) are increasingly investigated in dentistry and oral and maxillofacial surgery, but comparative evidence regarding specialist-level response quality, clinical content, and potential safety concerns remains limited. This study compared four contemporary LLM interfaces in third molar surgery complication scenarios. Methods: Thirty open-ended specialist-level scenarios were submitted once to ChatGPT 5.4 Thinking, DeepSeek-V3.2, Gemini 3.1 Pro, and Microsoft Copilot (Work and Learning mode) in isolated single-turn sessions, yielding 120 responses. Two oral and maxillofacial surgeons, blinded to interface identity, independently assessed responses using a task-adapted five-point Global Quality Score (GQS). An additional post hoc criterion-referenced evaluation assessed diagnosis, management, escalation\/referral, potentially harmful recommendations, safety-critical omissions, and potential clinical risk. Readability and externally observable response latency were also evaluated. Results: Among the sampled outputs, mean GQSs were 4.53 for Gemini, 4.40 for ChatGPT, 4.30 for DeepSeek, and 3.92 for Copilot (p &lt; 0.001). Mean Clinical Content Scores were 4.80\/5 for ChatGPT, 4.77 for Gemini, 4.75 for DeepSeek, and 3.90 for Copilot (p &lt; 0.001). All responses were rated as recognizing the target complication. Within the 120 sampled outputs, one Copilot response met the study-specific criterion for a potentially harmful recommendation, and two Copilot responses met the predefined criteria for safety-critical omissions. Within this benchmark, the sampled Copilot outputs had higher study-specific potential clinical-risk scores than the sampled outputs from the other evaluated interfaces (p &lt; 0.001). Readability and externally observable response latency also differed across the sampled interface outputs. Conclusions: In this single-generation benchmark, the sampled outputs showed different profiles in specialist-rated quality, clinical content, potential clinical risk, readability, and externally observable response latency under the specific interface configurations tested. These findings apply to the observed responses from the single synchronized testing session and should not be interpreted as establishing superiority of any underlying LLM, independent diagnostic accuracy, clinical equivalence, clinical safety, effectiveness, or readiness for autonomous decision support.<\/jats:p>","DOI":"10.3390\/jcm15197397","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T12:36:07Z","timestamp":1790166967000},"page":"7397","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Comparative Evaluation of Large Language Model Interfaces in Third Molar Surgery Complication Scenarios: Response Quality, Clinical Content, Potential Clinical Risk, Readability, and Externally Observable Response Latency\u2014A Cross-Sectional Comparative Benchmark Study"],"prefix":"10.3390","volume":"15","author":[{"given":"\u0130nci Rana","family":"Karaca","sequence":"first","affiliation":[{"name":"Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Gazi University, Ankara 06490, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"supervision","vocabulary":"credit"},{"role":"project-administration","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5839-3341","authenticated-orcid":false,"given":"Esmanur","family":"Ba\u015fer","sequence":"additional","affiliation":[{"name":"Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Gazi University, Ankara 06490, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4829-6472","authenticated-orcid":false,"given":"Emre","family":"Uluba\u015f","sequence":"additional","affiliation":[{"name":"Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Gazi University, Ankara 06490, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7396-9697","authenticated-orcid":false,"given":"Elif Bet\u00fcl","family":"Y\u0131ld\u0131r\u0131m","sequence":"additional","affiliation":[{"name":"Department of Oral and Maxillofacial Surgery, Faculty of Dentistry, Gazi University, Ankara 06490, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"methodology","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1016\/j.coms.2006.11.013","article-title":"Complications of third molar surgery","volume":"19","author":"Bouloux","year":"2007","journal-title":"Oral Maxillofac. Surg. Clin. N. Am."},{"key":"ref_2","first-page":"257","article-title":"Patients\u2019 satisfaction and prevalence of complications on surgical extraction of third molar","volume":"9","author":"Lee","year":"2015","journal-title":"Patient Prefer. Adherence"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e230","DOI":"10.18295\/squmj.2019.19.03.009","article-title":"Complications of third molar extraction: A retrospective study from a tertiary healthcare centre in Oman","volume":"19","author":"Sayed","year":"2019","journal-title":"Sultan Qaboos Univ. Med. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1002\/ca.23693","article-title":"Comprehensive review of lower third molar management: A guide for improved informed consent","volume":"34","author":"Iwanaga","year":"2021","journal-title":"Clin. Anat."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1016\/j.ijom.2019.10.009","article-title":"Patient, radiological, and operative factors associated with surgical difficulty in the extraction of third molars: A systematic review","volume":"49","year":"2020","journal-title":"Int. J. Oral Maxillofac. Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e29989","DOI":"10.1097\/MD.0000000000029989","article-title":"Prevalence of and risk factors for postoperative complications after lower third molar extraction: A multicenter prospective observational study in Japan","volume":"101","author":"Yamada","year":"2022","journal-title":"Medicine"},{"key":"ref_7","first-page":"565","article-title":"Unusual complications associated with third molar surgery: A systematic review","volume":"40","author":"Brauer","year":"2009","journal-title":"Quintessence Int."},{"key":"ref_8","first-page":"e85","article-title":"Prevalence of intraoperative and postoperative iatrogenic mandibular fractures after lower third molar extraction: A systematic review","volume":"14","year":"2022","journal-title":"J. Clin. Exp. Dent."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.joms.2010.05.036","article-title":"Informed consent in oral surgery: The value of written information","volume":"69","year":"2011","journal-title":"J. Oral Maxillofac. Surg."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/S0266-4356(97)90042-5","article-title":"Third molar surgery\u2014A preliminary report on aspects affecting quality of life in the early postoperative period","volume":"35","author":"Savin","year":"1997","journal-title":"Br. J. Oral Maxillofac. Surg."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1016\/j.ijom.2009.11.010","article-title":"Incidence of neurosensory deficits and recovery after lower third molar surgery: A prospective clinical study of 4338 cases","volume":"39","author":"Cheung","year":"2010","journal-title":"Int. J. Oral Maxillofac. Surg."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1016\/j.ijom.2013.01.023","article-title":"Risk factors associated with inferior alveolar nerve injury after extraction of the mandibular third molar: A comparative study of preoperative images by panoramic radiography and computed tomography","volume":"42","author":"Hasegawa","year":"2013","journal-title":"Int. J. Oral Maxillofac. Surg."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2508","DOI":"10.1016\/j.joms.2012.06.004","article-title":"Which risk factors are associated with neurosensory deficits of inferior alveolar nerve after mandibular third molar extraction?","volume":"70","author":"Kim","year":"2012","journal-title":"J. Oral Maxillofac. Surg."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1435348","DOI":"10.1155\/2017\/1435348","article-title":"An observational cohort study on delayed-onset infections after mandibular third-molar extractions","volume":"2017","author":"Brunello","year":"2017","journal-title":"Int. J. Dent."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1685","DOI":"10.1016\/j.joms.2007.04.019","article-title":"Age as a risk factor for third molar surgery complications","volume":"65","author":"Chuang","year":"2007","journal-title":"J. Oral Maxillofac. Surg."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1007\/s10266-025-01214-1","article-title":"Can chatbots replace experts? Diagnostic accuracy of AI models in classifying impacted mandibular third molars","volume":"114","author":"Erden","year":"2026","journal-title":"Odontology"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sezer, B., and Aydo\u011fdu, T. (2025). Performance of advanced artificial intelligence models in traumatic dental injuries in primary dentition: A comparative evaluation of ChatGPT-4 Omni, DeepSeek, Gemini Advanced, and Claude 3.7 in terms of accuracy, completeness, response time, and readability. Appl. Sci., 15.","DOI":"10.3390\/app15147778"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1016\/j.joms.2024.06.177","article-title":"Is ChatGPT an accurate and readable patient aid for third molar extractions?","volume":"82","author":"Jacobs","year":"2024","journal-title":"J. Oral Maxillofac. Surg."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ossowska, A., Kusiak, A., Camlet, A., and \u015awietlik, D. (2026). Diagnostic performance and error patterns of a large language model and neural network in periodontitis classification: A comparative study. J. Clin. Med., 15.","DOI":"10.3390\/jcm15124837"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"CHART Collaborative (2025). Reporting guidelines for chatbot health advice studies: Explanation and elaboration for the Chatbot Assessment Reporting Tool (CHART). BMJ, 390, e083305.","DOI":"10.1136\/bmj-2024-083305"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Miloro, M., Ghali, G.E., Larsen, P.E., and Waite, P. (2022). Peterson\u2019s Principles of Oral and Maxillofacial Surgery, Springer. [4th ed.].","DOI":"10.1007\/978-3-030-91920-7"},{"key":"ref_22","unstructured":"Fonseca, R.J. (2018). Oral and Maxillofacial Surgery, Elsevier. [3rd ed.]."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Eufinger, H., K\u00fcbler, A., and Schliephake, H. (2023). Oral and Maxillofacial Surgery: Surgical Textbook and Atlas, Springer.","DOI":"10.1007\/978-3-662-66844-3"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2070","DOI":"10.1111\/j.1572-0241.2007.01325.x","article-title":"A systematic review of patient inflammatory bowel disease information resources on the World Wide Web","volume":"102","author":"Bernard","year":"2007","journal-title":"Am. J. Gastroenterol."},{"key":"ref_25","unstructured":"World Health Organization (2009). The Conceptual Framework for the International Classification for Patient Safety, Version 1.1: Final Technical Report, World Health Organization. WHO\/IER\/PSP\/2010.2."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e2336483","DOI":"10.1001\/jamanetworkopen.2023.36483","article-title":"Accuracy and Reliability of Chatbot Responses to Physician Questions","volume":"6","author":"Goodman","year":"2023","journal-title":"JAMA Netw. Open"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and Mitigation of the Limitations of Large Language Models in Clinical Decision-Making","volume":"30","author":"Hager","year":"2024","journal-title":"Nat. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/j.bjoms.2025.03.006","article-title":"The impact of the large language model ChatGPT in oral and maxillofacial surgery: A systematic review","volume":"63","author":"Torres","year":"2025","journal-title":"Br. J. Oral Maxillofac. Surg."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ronsivalle, V., Santonocito, S., Cammarata, U., Lo Muzio, E., and Cicci\u00f9, M. (2025). Current Applications of Chatbots Powered by Large Language Models in Oral and Maxillofacial Surgery: A Systematic Review. Dent. J., 13.","DOI":"10.3390\/dj13060261"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/15\/19\/7397\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T12:52:00Z","timestamp":1790167920000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/15\/19\/7397"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":29,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["jcm15197397"],"URL":"https:\/\/doi.org\/10.3390\/jcm15197397","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]}}}