{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T03:02:51Z","timestamp":1790218971908,"version":"4.1.0"},"reference-count":35,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Accurate assessment of pulpal status is essential for achieving successful endodontic outcomes. However, direct evaluation remains inherently challenging because the pulp is surrounded by calcified tissue, necessitating reliance on clinical and radiographic examinations for diagnostic and prognostic decision-making. These procedures demand substantial clinical expertise and time, and less-experienced clinicians often face challenges that may lead to errors in diagnosis and treatment planning. Recent advancements in large language models (LLMs) offer promising opportunities to enhance clinical reasoning by facilitating the integration of evidence and supporting methodical diagnostic decision-making.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate the clinical applicability of LLMs by comparing their text-based clinical screening performance and the clinical validity of their treatment plan responses with those of human evaluators.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Between January 2011 and December 2022, 100 clinical cases involving primary endodontic disease were randomly selected from the clinical records of outpatients who visited the Department of Conservative Dentistry or Advanced General Dentistry (AGD) at Yonsei University Dental Hospital. Four prompt types, combining 2 variables (language and role), were used as input for 4 LLMs. Both LLMs and human evaluators (AGD specialists, AGD residents, endodontic residents, and senior dental students) assessed the cases using text-based clinical records. Radiographic images were not directly provided. Screening performance was evaluated using a 0-to-2-point concordance scale, and treatment plan validity and relevance were assessed using a 5-point Likert scale.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Among the 4 LLMs evaluated, ChatGPT achieved the highest mean concordance score on Korean-doctor prompts (mean 0.98, SD 0.82). However, this score did not reach the partially correct criterion of 1 on the 0 to 2-point scale. Clova X recorded the lowest mean score on English-patient prompts (mean 0.23, SD 0.63). Across both diagnostic categories, AGD specialists demonstrated the highest diagnostic accuracy (pulpal: 0.70; periapical: 0.65), with higher sensitivity but lower specificity than those exhibited by the other groups. ChatGPT also showed favorable performance among the LLMs, with accuracies of 0.65 (95% CI 0.55\u20100.74) for pulpal disease and 0.57 (95% CI 0.47\u20100.69) for periapical disease, which were comparable to those of AGD and endodontic residents.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Under image-free clinical record review conditions, ChatGPT 4.0 showed relatively higher and more consistent performance in symptom screening and treatment planning compared to the other LLMs evaluated. However, its highest mean score of 0.98 (SD 0.82) did not reach the partially correct criterion of 1 on the 0 to 2-point scale. Hallucinations generated by LLMs and experience-dependent interpretation biases among human evaluators remain key challenges that require attention. Therefore, continuous clinical supervision and comprehensive user training are necessary for the safe and effective clinical application.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/86145","type":"journal-article","created":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T14:44:09Z","timestamp":1781621049000},"page":"e86145-e86145","source":"Crossref","is-referenced-by-count":1,"title":["Large Language Models for Endodontic Symptom Assessment and Treatment Planning Using Image-Free Clinical Records: Comparative Evaluation Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0320-6705","authenticated-orcid":false,"given":"Dahyun","family":"Seo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0313-2511","authenticated-orcid":false,"given":"Jieun","family":"Cheong","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6283-9686","authenticated-orcid":false,"given":"Yiseul","family":"Choi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1121-2570","authenticated-orcid":false,"given":"Yooseok","family":"Shin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2081-1156","authenticated-orcid":false,"given":"Wonse","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,7,24]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1111\/j.1601-1546.2003.00024.x","article-title":"Pulpal diagnosis","volume":"5","author":"Sigurdsson","journal-title":"Endod Topics"},{"key":"R2","doi-asserted-by":"crossref","unstructured":"Rosen E Paul R Tsesis I . Evidence-based decision making in dentistry: the endodontic perspective. In: Rosen E Nemcovsky CE Tsesis I , editors. Evidence-Based Decision Making in Dentistry: Multidisciplinary Management of the Natural Dentition. Springer; 2017:19-37. [doi: 10.1007\/978-3-319-45733-8_3]","DOI":"10.1007\/978-3-319-45733-8_3"},{"issue":"8","key":"R3","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.14219\/jada.archive.2009.0312","article-title":"The validity of pulp testing: a clinical study","volume":"140","author":"Weisleder","journal-title":"J Am Dent Assoc"},{"issue":"8","key":"R4","doi-asserted-by":"publisher","first-page":"1138","DOI":"10.1111\/iej.13115","article-title":"Cone beam computed tomography in endodontics\u2014a review of the literature","volume":"52","author":"Patel","journal-title":"Int Endod J"},{"issue":"9","key":"R5","doi-asserted-by":"publisher","first-page":"783","DOI":"10.1111\/j.1365-2591.2012.02034.x","article-title":"Radiological diagnosis of periapical bone tissue lesions in endodontics: a systematic review","volume":"45","author":"Petersson","journal-title":"Int Endod J"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1259\/dmfr\/18200441","article-title":"Evidence-based diagnosis and clinical decision making","volume":"38","author":"Mileman","journal-title":"Dentomaxillofac Radiol"},{"issue":"6","key":"R7","doi-asserted-by":"publisher","first-page":"747","DOI":"10.14219\/jada.archive.2004.0301","article-title":"Conducting systematic reviews and creating clinical practice guidelines in dentistry: lessons learned","volume":"135","author":"Sutherland","journal-title":"J Am Dent Assoc"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1038\/s41368-023-00239-y","article-title":"ChatGPT for shaping the future of dentistry: the potential of multi-modal large language model","volume":"15","author":"Huang","journal-title":"Int J Oral Sci"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.2196\/48808","article-title":"ChatGPT-generated differential diagnosis lists for complex case-derived clinical vignettes: diagnostic accuracy evaluation","volume":"11","author":"Hirosawa","journal-title":"JMIR Med Inform"},{"issue":"7","key":"R10","doi-asserted-by":"publisher","DOI":"10.7759\/cureus.42133","article-title":"The potential usefulness of ChatGPT in oral and maxillofacial radiology","volume":"15","author":"Mago","journal-title":"Cureus"},{"issue":"3","key":"R11","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1111\/odi.15082","article-title":"Innovating dental diagnostics: ChatGPT\u2019s accuracy on diagnostic challenges","volume":"31","author":"Danesh","journal-title":"Oral Dis"},{"issue":"5","key":"R12","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1016\/j.joen.2025.01.002","article-title":"Performance of 4 artificial intelligence chatbots in answering endodontic questions","volume":"51","author":"Abdulrab","journal-title":"J Endod"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2025.105948","article-title":"Readability, accuracy and appropriateness and quality of AI chatbot responses as a patient information source on root canal retreatment: a comparative assessment","volume":"201","author":"B\u00fcker","journal-title":"Int J Med Inform"},{"issue":"10","key":"R14","doi-asserted-by":"publisher","first-page":"1413","DOI":"10.1016\/j.joen.2025.06.014","article-title":"Performance of 7 artificial intelligence chatbots on board-style endodontic questions","volume":"51","author":"Jalali","journal-title":"J Endod"},{"issue":"11","key":"R15","doi-asserted-by":"publisher","first-page":"1617","DOI":"10.1016\/j.joen.2025.04.015","article-title":"Evaluating large language models in addressing patient questions on endodontic pain: a comparative analysis of accessible chatbots","volume":"51","author":"Aljamani","journal-title":"J Endod"},{"issue":"6","key":"R16","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1111\/iej.14217","article-title":"Evaluation of the performance of ChatGPT-4 and ChatGPT-4o as a learning tool in endodontics","volume":"59","author":"Ar\u0131l\u0131 \u00d6zt\u00fcrk","journal-title":"Int Endod J"},{"key":"R17","unstructured":"International statistical classification of diseases and related health problems. World Health Organization. URL: https:\/\/icd.who.int\/browse10\/2019\/en [Accessed 25-06-2026]"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.identj.2024.06.022","article-title":"Comprehensiveness of large language models in patient queries on gingival and endodontic health","volume":"75","author":"Zhang","journal-title":"Int Dent J"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1038\/s41391-023-00705-y","article-title":"Quality of information and appropriateness of ChatGPT outputs for urology patients","volume":"27","author":"Cocci","journal-title":"Prostate Cancer Prostatic Dis"},{"key":"R20","unstructured":"Diagnostic test evaluation. Medcalc. URL: https:\/\/www.medcalc.org\/en\/calc\/diagnostic_test.php [Accessed 10-06-2026]"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1111\/j.1365-2929.2004.01972.x","article-title":"What every teacher needs to know about clinical reasoning","volume":"39","author":"Eva","journal-title":"Med Educ"},{"issue":"13","key":"R22","doi-asserted-by":"publisher","first-page":"1602","DOI":"10.1001\/jama.292.13.1602","article-title":"Accuracy of diagnostic tests read with and without clinical information: a systematic review","volume":"292","author":"Loy","journal-title":"JAMA"},{"issue":"9","key":"R23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3560815","article-title":"Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing","volume":"55","author":"Liu","journal-title":"ACM Comput Surv"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.2196\/50638","article-title":"Prompt engineering as an important emerging skill for medical professionals: tutorial","volume":"25","author":"Mesk\u00f3","journal-title":"J Med Internet Res"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.2196\/60501","article-title":"Prompt engineering paradigms for medical applications: scoping review","volume":"26","author":"Zaghir","journal-title":"J Med Internet Res"},{"issue":"12","key":"R26","doi-asserted-by":"publisher","DOI":"10.1097\/GOX.0000000000005471","article-title":"Can AI think like a plastic surgeon? Evaluating GPT-4\u2019s clinical judgment in reconstructive procedures of the upper extremity","volume":"11","author":"Leypold","journal-title":"Plast Reconstr Surg Glob Open"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.21037\/jmai-23-71","article-title":"The impact of prompt engineering in large language model performance: a psychiatric example","volume":"6","author":"Grabb","journal-title":"J Med Artif Intell"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","DOI":"10.2196\/50658","article-title":"Using ChatGPT for clinical practice and medical education: cross-sectional survey of medical students\u2019 and physicians\u2019 perceptions","volume":"9","author":"Tangadulrat","journal-title":"JMIR Med Educ"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/57592","article-title":"Evaluating Bard Gemini Pro and GPT-4 Vision against student performance in medical visual question answering: comparative case study","volume":"8","author":"Roos","journal-title":"JMIR Form Res"},{"issue":"1","key":"R30","doi-asserted-by":"publisher","DOI":"10.1148\/radiol.240273","article-title":"Comparing diagnostic accuracy of radiologists versus GPT-4V and Gemini Pro Vision using image inputs from diagnosis please cases","volume":"312","author":"Suh","journal-title":"Radiology"},{"issue":"11","key":"R31","doi-asserted-by":"publisher","DOI":"10.1007\/s00784-024-05968-w","article-title":"Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments","volume":"28","author":"K\u00fcnzle","journal-title":"Clin Oral Investig"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","DOI":"10.1186\/s12903-025-05857-y","article-title":"Comparing diagnostic skills in endodontic cases: dental students versus ChatGPT-4o","volume":"25","author":"Durmazpinar","journal-title":"BMC Oral Health"},{"issue":"1","key":"R33","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0317423","article-title":"Accuracy of latest large language models in answering multiple choice questions in dentistry: a comparative study","volume":"20","author":"Nguyen","journal-title":"PLoS One"},{"issue":"1","key":"R34","doi-asserted-by":"publisher","DOI":"10.1186\/s12903-025-06149-1","article-title":"Assessment of various artificial intelligence applications in responding to technical questions in endodontic surgery","volume":"25","author":"Baris","journal-title":"BMC Oral Health"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01029-4","article-title":"Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs","volume":"7","author":"Wang","journal-title":"NPJ Digit Med"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e86145","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:36Z","timestamp":1785536736000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e86145"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,24]]},"references-count":35,"URL":"https:\/\/doi.org\/10.2196\/86145","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,24]]},"article-number":"v14i1e86145"}}