{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,31]],"date-time":"2026-10-31T05:33:48Z","timestamp":1793424828168,"version":"4.5.0"},"reference-count":44,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T00:00:00Z","timestamp":1758758400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Medicina"],"abstract":"<jats:p>Background and Objectives: The growing academic interest within the biomedical sciences regarding the diagnostic capabilities of multimodal language models, such as ChatGPT-4o, is clear. However, their ability to interpret oral clinical images remains insufficiently explored. This exploratory pilot study aimed to provide preliminary observations about the diagnostic validity of ChatGPT-4o in identifying oral squamous cell carcinoma (OSCC), oral leukoplakia (OL), and oral lichen planus (OLP) using only clinical photographs, without the inclusion of additional clinical data. Materials and Methods: Two general dentists selected 23 images of oral lesions suspected to be OSCC, OL, or OLP. ChatGPT-4o was asked to provide a probable diagnosis for each image on 30 occasions, generating a total of 690 responses. The responses were then evaluated against the reference diagnosis set up by an expert to calculate sensitivity, specificity, predictive values, and the area under the ROC curve. Results: ChatGPT-4o demonstrated high specificity across the three conditions (97.1% for OSCC, 100% for OL, and 96.1% for OLP), correctly classifying 90% of OSCC cases (AUC = 0.81). However, this overall accuracy was largely driven by correct negative classifications, while the clinically relevant sensitivity for OSCC was only 65%. In spite of that, sensitivity was highly variable: 60% for OL and just 25% for OLP, which limits its usefulness in a clinical setting for ruling out these conditions. The model achieved positive predictive values of 86.7% for OSCC and 100% for OL. Given the small dataset, these findings should be interpreted only as preliminary evidence. Conclusions: ChatGPT-4o demonstrates potential as a complementary tool for the screening of OSCC in clinical oral images. Nevertheless, the pilot nature of this study and the reduced sample size highlight that larger, adequately powered studies (with several hundred cases per pathology) are needed to obtain robust and generalizable results. Nevertheless, its sensitivity remains insufficient, as a significant proportion of true cases were missed, underscoring that the model cannot be relied upon as a standalone diagnostic tool.<\/jats:p>","DOI":"10.3390\/medicina61101744","type":"journal-article","created":{"date-parts":[[2025,9,25]],"date-time":"2025-09-25T12:05:13Z","timestamp":1758801913000},"page":"1744","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["ChatGPT in Oral Pathology: Bright Promise or Diagnostic Mirage"],"prefix":"10.3390","volume":"61","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2448-6669","authenticated-orcid":false,"given":"Ana","family":"Su\u00e1rez","sequence":"first","affiliation":[{"name":"Department of Pre-Clinic Dentistry II, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0727-777X","authenticated-orcid":false,"given":"Yolanda","family":"Freire","sequence":"additional","affiliation":[{"name":"Department of Pre-Clinic Dentistry II, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6141-0546","authenticated-orcid":false,"given":"V\u00edctor","family":"D\u00edaz-Flores Garc\u00eda","sequence":"additional","affiliation":[{"name":"Department of Pre-Clinic Dentistry I, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6082-6056","authenticated-orcid":false,"given":"Andrea","family":"Santamar\u00eda Laorden","sequence":"additional","affiliation":[{"name":"Department of Pre-Clinic Dentistry II, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9756-1709","authenticated-orcid":false,"given":"Jaime","family":"Orejas P\u00e9rez","sequence":"additional","affiliation":[{"name":"Department of Pre-Clinic Dentistry II, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6059-0549","authenticated-orcid":false,"given":"Mar\u00eda","family":"Su\u00e1rez Ajuria","sequence":"additional","affiliation":[{"name":"Department of Pre-Clinic Dentistry II, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3616-3210","authenticated-orcid":false,"given":"Juan","family":"Algar","sequence":"additional","affiliation":[{"name":"Department of Clinic Dentistry, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0937-0994","authenticated-orcid":false,"given":"Carmen","family":"Mart\u00edn Carreras-Presas","sequence":"additional","affiliation":[{"name":"Department of Clinical Dentistry-Postgraduate Studies, Faculty of Biomedical and Health Sciences, Universidad Europea de Madrid, 28670 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1038\/s41405-024-00277-6","article-title":"Innovation and Application of Large Language Models (LLMs) in Dentistry\u2014A Scoping Review","volume":"10","author":"Umer","year":"2024","journal-title":"BDJ Open"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Nia, M.F., Ahmadi, M., and Irankhah, E. (2025). Transforming dental diagnostics with artificial intelligence: Advanced integration of ChatGPT and large language models for patient care. Front. Dent. Med., 5.","DOI":"10.3389\/fdmed.2024.1456208"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e087469","DOI":"10.1136\/bmjopen-2024-087469","article-title":"Effects of Interacting with a Large Language Model Compared with a Human Coach on the Clinical Diagnostic Process and Outcomes among Fourth-Year Medical Students: Study Protocol for a Prospective, Randomised Experiment Using Patient Vignettes","volume":"14","author":"Hautz","year":"2024","journal-title":"BMJ Open"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1038\/s41746-024-01010-1","article-title":"Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine","volume":"7","author":"Savage","year":"2024","journal-title":"NPJ Digit. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1789","DOI":"10.1002\/jum.16524","article-title":"Interpretation of Artificial Intelligence Models in Healthcare","volume":"43","author":"Airom","year":"2024","journal-title":"J. Ultrasound Med."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Brasil, S., Pascoal, C., Francisco, R., dos Reis Ferreira, V., Videira, P.A., and Valad\u00e3o, G. (2019). Artificial Intelligence (AI) in Rare Diseases: Is the Future Brighter?. Genes, 10.","DOI":"10.3390\/genes10120978"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1001\/jamainternmed.2024.0295","article-title":"Clinical Reasoning of a Generative Artificial Intelligence Model Compared with Physicians","volume":"184","author":"Cabral","year":"2024","journal-title":"JAMA Intern. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1016\/j.csbj.2025.04.010","article-title":"Decoding Wisdom: Evaluating ChatGPT\u2019s Accuracy and Reproducibility in Analyzing Orthopantomographic Images for Third Molar Assessment","volume":"28","author":"Arena","year":"2025","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Warin, K., Limprasert, W., Suebnukarn, S., Jinaporntham, S., Jantana, P., and Vicharueang, S. (2022). AI-Based Analysis of Oral Lesions Using Novel Deep Convolutional Neural Networks for Early Detection of Oral Cancer. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0273508"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Xu, Z., Lin, A., and Han, X. (2025). Current AI Applications and Challenges in Oral Pathology. Oral, 5.","DOI":"10.3390\/oral5010002"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Perumal, M.K.K., Renuka, R.R., Subbiah, S.K., and Natarajan, P.M. (2025). Artificial Intelligence-Driven Clinical Decision Support Systems for Early Detection and Precision Therapy in Oral Cancer: A Mini Review. Front. Oral Health, 6.","DOI":"10.3389\/froh.2025.1592428"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mirfendereski, P., Li, G.Y., Pearson, A.T., and Kerr, A.R. (2025). Artificial Intelligence and the Diagnosis of Oral Cavity Cancer and Oral Potentially Malignant Disorders from Clinical Photographs: A Narrative Review. Front. Oral Health, 6.","DOI":"10.3389\/froh.2025.1569567"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Talwar, V., Singh, P., Mukhia, N., Shetty, A., Birur, P., Desai, K.M., Sunkavalli, C., Varma, K.S., Sethuraman, R., and Jawahar, C.V. (2023). AI-Assisted Screening of Oral Potentially Malignant Disorders Using Smartphone-Based Photographic Images. Cancers, 15.","DOI":"10.3390\/cancers15164120"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"100133","DOI":"10.1016\/j.apjon.2022.100133","article-title":"Artificial Intelligence in Early Diagnosis and Prevention of Oral Cancer","volume":"9","author":"Hegde","year":"2022","journal-title":"Asia Pac. J. Oncol. Nurs."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Abati, S., Bramati, C., Bondi, S., Lissoni, A., and Trimarchi, M. (2020). Oral Cancer and Precancer: A Narrative Review on the Relevance of Early Diagnosis. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17249160"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1016\/j.oooo.2017.12.011","article-title":"Oral Potentially Malignant Disorders: Risk of Progression to Malignancy","volume":"125","author":"Speight","year":"2018","journal-title":"Oral Surg. Oral Med. Oral Pathol. Oral Radiol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"106442","DOI":"10.1016\/j.oraloncology.2023.106442","article-title":"The Importance of Early Treatment of Oral Squamous Cell Carcinoma: Case Report","volume":"144","author":"Silva","year":"2023","journal-title":"Oral Oncol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/s00403-016-1667-2","article-title":"Oral Lichen Planus: A Literature Review and Update","volume":"308","author":"Alrashdan","year":"2016","journal-title":"Arch. Dermatol. Res."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1007\/s40257-023-00814-3","article-title":"Oral Lichen Planus: An Update on Diagnosis and Management","volume":"25","author":"Louisy","year":"2024","journal-title":"Am. J. Clin. Dermatol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Lodolo, M., Valor, J., and Villa, A. (2025). Randomized Controlled Trials for Oral Leukoplakia. Oral Dis.","DOI":"10.1111\/odi.15399"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1038\/s41415-024-7063-y","article-title":"Oral Lichen Planus and Lichenoid Lesions\u2014Challenges and Pitfalls for the General Dental Practitioner","volume":"236","author":"Binnie","year":"2024","journal-title":"Br. Dent. J."},{"key":"ref_22","first-page":"e50804","article-title":"Oral Squamous Cell Carcinoma Mimicking Lichenoid Reaction After Implant Placement: A Case Report","volume":"15","author":"Albagieh","year":"2023","journal-title":"Cureus"},{"key":"ref_23","first-page":"746874","article-title":"Clinicopathologic Correlation of Oral Lichen Planus and Oral Lichenoid Lesions: A Preliminary Study","volume":"2014","author":"Sabol","year":"2014","journal-title":"Sci. World J."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1563\/aaid-joi-D-24-00056","article-title":"Oral Carcinoma Arising Under Implant-Supported Prosthesis: Progression of Proliferative Verrucous Leukoplakia Initially Mimicking Lichen Planus","volume":"50","author":"Silva","year":"2024","journal-title":"J. Oral. Implantol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.anl.2011.06.004","article-title":"Carcinoma Cuniculatum Mimicking Leukoplakia of the Mandibular Gingiva","volume":"39","author":"Suzuki","year":"2012","journal-title":"Auris Nasus Larynx"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e48659","DOI":"10.2196\/48659","article-title":"Assessing the Utility of ChatGPT Throughout the Entire Clinical Workflow: Development and Usability Study","volume":"25","author":"Rao","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_27","first-page":"108","article-title":"Unveiling the ChatGPT Phenomenon: Evaluating the Consistency and Accuracy of Endodontic Question Answers","volume":"57","author":"Algar","year":"2023","journal-title":"Int. Endod. J."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Freire, Y., Santamar\u00eda Laorden, A., Orejas P\u00e9rez, J., G\u00f3mez S\u00e1nchez, M., D\u00edaz-Flores Garc\u00eda, V., and Su\u00e1rez, A. (2024). ChatGPT Performance in Prosthodontics: Assessment of Accuracy and Repeatability in Answer Generation. J. Prosthet. Dent., 131.","DOI":"10.1016\/j.prosdent.2024.01.018"},{"key":"ref_29","first-page":"46","article-title":"Beyond the Scalpel: Assessing ChatGPT\u2019s Potential as an Auxiliary Intelligent Virtual Assistant in Oral Surgery","volume":"24","author":"Freire","year":"2023","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"\u00d6nc\u00fc, S., Torun, F., and \u00dclk\u00fc, H.H. (2025). AI-Powered Standardised Patients: Evaluating ChatGPT-4o\u2019s Impact on Clinical Case Management in Intern Physicians. BMC Med. Educ., 25.","DOI":"10.1186\/s12909-025-06877-6"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e49877","DOI":"10.2196\/49877","article-title":"ChatGPT Interactive Medical Simulations for Early Clinical Education: Case Study","volume":"9","author":"Scherr","year":"2023","journal-title":"JMIR Med. Educ."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Frosolini, A., Catarzi, L., Benedetti, S., Latini, L., Chisci, G., Franz, L., Gennaro, P., and Gabriele, G. (2024). The Role of Large Language Models (LLMs) in Providing Triage for Maxillofacial Trauma Cases: A Preliminary Study. Diagnostics, 14.","DOI":"10.3390\/diagnostics14080839"},{"key":"ref_33","unstructured":"OpenAI (2023). GPT-4 Technical Report. arXiv."},{"key":"ref_34","unstructured":"OpenAI (2025, September 22). OpenAI Temporary Chat FAQ. Available online: https:\/\/help.openai.com\/en\/articles\/8914046-temporary-chat-faq."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhan, Z.-Z., Xiong, Y.-T., Wang, C.-Y., Zhang, B.-T., Lian, W.-J., Zeng, Y.-M., Liu, W., Tang, W., and Liu, C. (2025). Utilizing GPT-4 to Interpret Oral Mucosal Disease Photographs for Structured Report Generation. Sci. Rep., 15.","DOI":"10.1038\/s41598-025-89328-y"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Schmidl, B., H\u00fctten, T., Pigorsch, S., St\u00f6gbauer, F., Hoch, C.C., Hussain, T., Wollenberg, B., and Wirth, M. (2025). Artificial Intelligence for Image Recognition in Diagnosing Oral and Oropharyngeal Cancer and Leukoplakia. Sci. Rep., 15.","DOI":"10.1038\/s41598-025-85920-4"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1038\/s41746-024-01185-7","article-title":"Hidden Flaws behind Expert-Level Accuracy of Multimodal GPT-4 Vision in Medicine","volume":"7","author":"Jin","year":"2024","journal-title":"npj Digit. Med."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1111\/odi.15169","article-title":"Challenging ChatGPT-4V for the Diagnosis of Oral Diseases and Conditions","volume":"31","year":"2025","journal-title":"Oral Dis."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1038\/s41368-023-00239-y","article-title":"ChatGPT for Shaping the Future of Dentistry: The Potential of Multi-Modal Large Language Model","volume":"15","author":"Huang","year":"2023","journal-title":"Int. J. Oral Sci."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Su\u00e1rez, A., Freire, Y., Su\u00e1rez, M., D\u00edaz-Flores Garc\u00eda, V., Andreu-V\u00e1zquez, C., Thuissard Vasallo, I.J., Castillo Var\u00f3n, A.I., and Mart\u00edn, C. (2025). Diagnostic Performance of Multimodal Large Language Models in the Analysis of Oral Pathology. Oral Dis.","DOI":"10.1111\/odi.70009"},{"key":"ref_41","first-page":"e224","article-title":"Accuracy of ChatGPT 3.5, 4.0, 4o and Gemini in Diagnosing Oral Potentially Malignant Lesions Based on Clinical Case Reports and Image Recognition","volume":"30","author":"Pradhan","year":"2020","journal-title":"Med. Oral Patol. Oral Cir. Bucal"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Kaygisiz, \u00d6.F., and Teke, M.T. (2025). Can Deepseek and ChatGPT Be Used in the Diagnosis of Oral Pathologies?. BMC Oral Health, 25.","DOI":"10.1186\/s12903-025-06034-x"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e70566","DOI":"10.2196\/70566","article-title":"The Diagnostic Performance of Large Language Models and Oral Medicine Consultants for Identifying Oral Lesions in Text-Based Clinical Scenarios: Prospective Comparative Study","volume":"4","author":"AlDehlawi","year":"2025","journal-title":"JMIR AI"},{"key":"ref_44","first-page":"ooae080","article-title":"Prompt Engineering with a Large Language Model to Assist Providers in Responding to Patient Inquiries: A Real-Time Implementation in the Electronic Health Record","volume":"7","author":"Afshar","year":"2024","journal-title":"JAMIA Open"}],"container-title":["Medicina"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1648-9144\/61\/10\/1744\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:49:30Z","timestamp":1760035770000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1648-9144\/61\/10\/1744"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,25]]},"references-count":44,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["medicina61101744"],"URL":"https:\/\/doi.org\/10.3390\/medicina61101744","relation":{},"ISSN":["1648-9144"],"issn-type":[{"value":"1648-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,25]]}}}