{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T22:57:07Z","timestamp":1791241027101,"version":"4.1.0"},"reference-count":25,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T00:00:00Z","timestamp":1771977600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background\/Objectives: Large language models (LLMs) have shown promising results in medical decision support; however, their effectiveness in managing acute cholecystitis and other gallbladder diseases remains insufficiently examined. This study evaluated the performance of a neuro-symbolic LLM system that integrates multiple AI agents with neural\u2013symbolic reasoning for acute cholecystitis management and compared its diagnostic accuracy with that of human expert physicians across three clinical specialties. Methods: This multi-center cross-sectional study included 30 case-based questions covering acute cholecystitis and gallbladder diseases, stratified across eight predefined disease categories: acute calculous cholecystitis (n = 6), acute acalculous cholecystitis (n = 2), complicated cholecystitis including gangrenous, emphysematous, and perforated variants (n = 5), chronic cholecystitis and biliary colic (n = 4), gallbladder polyps and adenomyomatosis (n = 3), Mirizzi syndrome (n = 2), gallbladder carcinoma (n = 4), and post-cholecystectomy complications (n = 4). Questions were categorized into diagnosis (n = 10), treatment (n = 10), and complications\/prognosis (n = 10). Gold standard answers were established through consensus by an expert panel consisting of two senior general surgery expert clinicians and one senior emergency medicine expert clinician, each with more than 20 years of clinical experience, utilizing the Tokyo Guidelines 2018 (TG18) as the reference standard for diagnostic criteria, severity grading, and management recommendations. The expert panel achieved unanimous consensus on all 30 gold standard answers. All responses were cross-referenced against the primary TG18 publications to ensure guideline-based rather than solely opinion-based reference standards. This consensus-based, guideline-anchored approach is consistent with established methodologies for gold standard establishment in AI diagnostic accuracy studies. Performance of a neuro-symbolic LLM system orchestrated via LangGraph v1.0 was compared against 10 general surgery specialists, 10 emergency medicine physicians, and 10 gastroenterology specialists from four tertiary centers in Turkey. The neuro-symbolic system incorporated the Tokyo Guidelines 2018 (TG18) as its symbolic knowledge base for diagnostic criteria, severity grading, and management algorithms. Results: The neuro-symbolic system attained the highest overall accuracy rate of 96.7% (29\/30), markedly surpassing the performance of general surgery specialists (average 82.3% \u00b1 6.8%), emergency medicine physicians (average 71.0% \u00b1 8.2%), and gastroenterology specialists (average 78.7% \u00b1 7.4%). Furthermore, the neuro-symbolic system exhibited superior performance across all clinical categories. Among human participants, general surgeons showed the highest accuracy in treatment decisions (88.0%), while gastroenterologists excelled in diagnostic questions (82.0%). Emergency medicine physicians showed comparable performance to other specialties in acute presentation scenarios. ROC analysis revealed excellent discrimination for the neuro-symbolic system (AUC = 0.983) compared to general surgery (AUC = 0.856), gastroenterology (AUC = 0.821), and emergency medicine (AUC = 0.764). Conclusions: The neuro-symbolic LLM system exhibited superior performance in standardized guideline-concordant case-based assessment of acute cholecystitis management compared to all human expert groups, reflecting its consistent application of encoded guideline criteria. These findings support its potential role as a clinical decision-support tool that augments, rather than replaces, physician expertise. The system\u2019s consistent application of standardized guidelines indicates its potential utility as a clinical decision support tool, particularly in settings where specialist expertise is limited. However, these results should be interpreted within the constraints of a structured case-based evaluation and do not imply global clinical superiority over human experts.<\/jats:p>","DOI":"10.3390\/jcm15051730","type":"journal-article","created":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T10:59:16Z","timestamp":1772017156000},"page":"1730","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Performance Comparison of a Neuro-Symbolic Large Language Model System Versus Human Experts in Acute Cholecystitis Management"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4895-2345","authenticated-orcid":false,"given":"Evren","family":"Ekingen","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, Etimesgut Sehit Sait Erturk State Hospital, Ankara 06790, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2321-2642","authenticated-orcid":false,"given":"Mete","family":"Ucdal","sequence":"additional","affiliation":[{"name":"Department of Internal Medicine, Etimesgut \u015eehit Sait Erturk State Hospital, Ankara 06790, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1186\/s13017-020-00336-x","article-title":"2020 World Society of Emergency Surgery updated guidelines for the diagnosis and treatment of acute calculus cholecystitis","volume":"15","author":"Pisano","year":"2020","journal-title":"World J. Emerg. Surg."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1002\/jhbp.515","article-title":"Tokyo Guidelines 2018: Diagnostic criteria and severity grading of acute cholecystitis (with videos)","volume":"25","author":"Yokoe","year":"2018","journal-title":"J. Hepatobiliary Pancreat Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"102880","DOI":"10.1016\/j.eclinm.2024.102880","article-title":"Clinical update on acute cholecystitis and biliary pancreatitis: Between certainties and grey areas","volume":"77","author":"Fugazzola","year":"2024","journal-title":"eClinicalMedicine"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102323","DOI":"10.1016\/j.xcrm.2025.102323","article-title":"Large language model as clinical decision support system augments medication safety in 16 clinical specialties","volume":"6","author":"Ong","year":"2025","journal-title":"Cell Rep. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1038\/s41746-025-01684-1","article-title":"Evaluating large language model workflows in clinical decision support for triage and referral and diagnosis","volume":"8","author":"Gaber","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Maity, S., and Saikia, M.J. (2025). Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering, 12.","DOI":"10.3390\/bioengineering12060631"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e59823","DOI":"10.2196\/59823","article-title":"The Clinicians\u2019 Guide to Large Language Models: A General Perspective With a Focus on Hallucinations","volume":"14","author":"Roustan","year":"2025","journal-title":"Interact. J. Med. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1038\/s43856-025-01021-3","article-title":"Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support","volume":"5","author":"Omar","year":"2025","journal-title":"Commun. Med."},{"key":"ref_9","first-page":"332","article-title":"Explainable Diagnosis Prediction through Neuro-Symbolic Integration","volume":"2025","author":"Lu","year":"2025","journal-title":"AMIA Jt. Summits Transl. Sci. Proc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1038\/s43856-025-01194-x","article-title":"Neuro-symbolic AI for auditable cognitive information extraction from medical reports","volume":"5","author":"Prenosil","year":"2025","journal-title":"Commun. Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"NAI-240740","DOI":"10.3233\/NAI-240740","article-title":"Towards a neuro-symbolic cycle for human-centered explainability","volume":"1","author":"Mileo","year":"2025","journal-title":"Neurosymb. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"29498732251377499","DOI":"10.1177\/29498732251377499","article-title":"Design Patterns for Large Language Model Based Neuro-Symbolic Systems","volume":"1","author":"Smit","year":"2025","journal-title":"Neurosymb. Artif. Intell."},{"key":"ref_13","first-page":"1265","article-title":"Is neuro-symbolic AI meeting its promises in natural language processing? A structured review","volume":"15","author":"Ebrahimi","year":"2024","journal-title":"Semant. Web"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1038\/s41746-025-01543-z","article-title":"A systematic review and meta-analysis of diagnostic performance comparison between generative AI and physicians","volume":"8","author":"Takita","year":"2025","journal-title":"npj Digit. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e64963","DOI":"10.2196\/64963","article-title":"Comparing Diagnostic Accuracy of Clinical Professionals and Large Language Models: Systematic Review and Meta-Analysis","volume":"13","author":"Shan","year":"2025","journal-title":"JMIR Med. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e71252","DOI":"10.2196\/71252","article-title":"Large Language Models for Automating Clinical Trial Criteria Conversion to Observational Medical Outcomes Partnership Common Data Model Queries: Validation and Evaluation Study","volume":"13","author":"Lee","year":"2025","journal-title":"JMIR Med. Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1016\/j.infoh.2025.03.001","article-title":"Next-generation agentic AI for transforming healthcare","volume":"2","author":"Karunanayake","year":"2025","journal-title":"Inform. Health"},{"key":"ref_18","first-page":"5","article-title":"Artificial intelligence in modern clinical practice (Review)","volume":"6","author":"Thomas","year":"2026","journal-title":"Med. Int."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Artsi, Y., Sorin, V., Glicksberg, B.S., Korfiatis, P., Nadkarni, G.N., and Klang, E. (2025). Large language models in real-world clinical workflows: A systematic review of applications and implementation. Front. Digit. Health, 7.","DOI":"10.3389\/fdgth.2025.1659134"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e2440969","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw. Open"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1038\/s43856-025-00781-2","article-title":"Physician clinical decision modification and bias assessment in a randomized controlled trial of AI assistance","volume":"5","author":"Goh","year":"2025","journal-title":"Commun. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"AIdbp2500120","DOI":"10.1056\/AIdbp2500120","article-title":"Assessment of Large Language Models in Clinical Reasoning: A Novel Benchmarking Study","volume":"2","author":"McCoy","year":"2025","journal-title":"NEJM AI"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1016\/j.joim.2025.06.003","article-title":"Integrated-omics analysis defines subtypes of hepatocellular carcinoma based on circadian rhythm","volume":"23","author":"Li","year":"2025","journal-title":"J. Integr. Med."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"234","DOI":"10.1007\/s43657-023-00152-8","article-title":"A Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video","volume":"4","author":"Zhang","year":"2024","journal-title":"Phenomics"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/15\/5\/1730\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T11:25:42Z","timestamp":1772018742000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/15\/5\/1730"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,25]]},"references-count":25,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["jcm15051730"],"URL":"https:\/\/doi.org\/10.3390\/jcm15051730","relation":{},"ISSN":["2077-0383"],"issn-type":[{"value":"2077-0383","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,25]]}}}