{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"Research Square"}],"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T03:34:25Z","timestamp":1787024065050,"version":"build-2736575974"},"posted":{"date-parts":[[2026,2,18]]},"group-title":"In Review","reference-count":49,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T00:00:00Z","timestamp":1771372800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100006108","name":"National Center for Advancing Translational Sciences","doi-asserted-by":"publisher","award":["UL1TR004419"],"award-info":[{"award-number":["UL1TR004419"]}],"id":[{"id":"10.13039\/100006108","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["S10OD026880"],"award-info":[{"award-number":["S10OD026880"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["S10OD030463"],"award-info":[{"award-number":["S10OD030463"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,2,17]]},"abstract":"<title>Abstract<\/title>\n                <p>Background\n LLMs are promising for clinical workflows but hallucinations limit deployment. Neuro-symbolic systems pair LLMs with explicit rules, ontologies, or knowledge graphs to constrain outputs, yet integration patterns are described inconsistently and their deployment trade-offs remain unclear.\nMethods\n We conducted a PRISMA 2020 systematic review (PROSPERO CRD420261296004) of peer-reviewed studies (PubMed\/MEDLINE, CENTRAL, Web of Science, Scopus; English; January 1, 2022 to January 30, 2026) and assessed risk of bias using PROBAST adapted to neuro-symbolic evaluations.\nResults\n We identified 3,166 records; after screening, 21 studies were included. Studies spanned 18 clinical settings (N\u2009=\u200920 to 197,761; median 2,398) with external validation in 9\/21 (42.9%). Four integration patterns were identified, ordered by increasing symbolic authority: structured output, rule-guided generation, knowledge retrieval, and iterative validation (symbolic veto or regeneration). Among studies reporting quantitative comparisons (n\u2009=\u200914), performance improvements ranged from +\u20093.1% to +\u2009125.6% (median\u2009+\u200921.9%) and increased with symbolic authority (median gains: 9%, 16%, 26%, 40%). All studies reported explicit hallucination mitigation, and 17\/21 (81.0%) reported guideline alignment. All studies were rated high risk of bias, driven mainly by analysis limitations (high in 19\/21). Iterative validation approaches reported 2\u201388 s latency and up to 100x cost increases.\nConclusion\n Across four neuro-symbolic LLM integration approaches, gains increase with symbolic authority: the strongest results come from iterative validation where the symbolic layer can veto, not just add context. That improves safety and auditability but raises latency, cost, and symbolic-stack failure risk.\nPrimary Funding Source:\n This work was supported in part through the computational and data resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai and supported by the Clinical and Translational Science Awards (CTSA) grant UL1TR004419 from the National Center for Advancing Translational Sciences. Research reported in this publication was also supported by the Office of Research Infrastructure of the National Institutes of Health under award number S10OD026880 and S10OD030463. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.\nRegistration:\n PROSPERO CRD420251120318<\/p>","DOI":"10.21203\/rs.3.rs-8902022\/v1","type":"posted-content","created":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T14:12:40Z","timestamp":1771423960000},"source":"Crossref","is-referenced-by-count":5,"title":["Neuro-symbolic LLM Integration in Clinical Medicine: A Systematic Review and Taxonomy"],"prefix":"10.21203","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-7542-8608","authenticated-orcid":false,"given":"Alon","family":"Gorenshtein","sequence":"first","affiliation":[{"name":"Department of Neurology, Harvard Medical School, Boston, MA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-0438-0827","authenticated-orcid":false,"given":"Mahmud","family":"Omar","sequence":"additional","affiliation":[{"name":"The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Medical Center, NY, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiftach","family":"Barash","sequence":"additional","affiliation":[{"name":"Division of Vascular and Interventional Radiology, Department of Radiology, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6319-4314","authenticated-orcid":false,"given":"Girish N","family":"Nadkarni","sequence":"additional","affiliation":[{"name":"The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Medical Center, NY, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4567-3108","authenticated-orcid":false,"given":"Eyal","family":"Klang","sequence":"additional","affiliation":[{"name":"The Windreich Department of Artificial Intelligence and Human Health, Mount Sinai Medical Center, NY, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","reference":[{"key":"ref1","doi-asserted-by":"crossref","unstructured":"1. Singhal K, Tu T, Gottweis J, et al. Toward expert-level medical question answering with large language models. Nat Med. 2025;31(3):943\u2013950. doi:10.1038\/s41591-024-03423-7","DOI":"10.1038\/s41591-024-03423-7"},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"2. Gorenshtein A, Weisblat Y, Khateb M, et al. AI-Based EMG Reporting: A Randomized Controlled Trial. J Neurol. 2025;272(9):586. doi:10.1007\/s00415-025-13261-3","DOI":"10.1007\/s00415-025-13261-3"},{"key":"ref3","doi-asserted-by":"crossref","unstructured":"3. Goh E, Gallo RJ, Strong E, et al. GPT-4 assistance for improvement of physician performance on patient care tasks: a randomized controlled trial. Nat Med. 2025;31(4):1233\u20131238. doi:10.1038\/s41591-024-03456-y","DOI":"10.1038\/s41591-024-03456-y"},{"key":"ref4","doi-asserted-by":"crossref","unstructured":"4. Katz U, Cohen E, Shachar E, et al. GPT versus Resident Physicians \u2014 A Benchmark Based on Official Board Scores. NEJM AI. 2024;1(5):AIdbp2300192. doi:10.1056\/AIdbp2300192","DOI":"10.1056\/AIdbp2300192"},{"key":"ref5","unstructured":"5. Advancing Claude in healthcare and the life sciences. Accessed January 14, 2026. https:\/\/www.anthropic.com\/news\/healthcare-life-sciences"},{"key":"ref6","unstructured":"6. Introducing ChatGPT Health. January 8, 2026. Accessed January 13, 2026. https:\/\/openai.com\/index\/introducing-chatgpt-health\/"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"7. Gorenshtein A, Omar M, Glicksberg BS, Nadkarni GN, Klang E. AI Agents in Clinical Medicine: A Systematic Review. medRxiv. Preprint posted online August 26, 2025:2025.08.22.25334232. doi:10.1101\/2025.08.22.25334232","DOI":"10.1101\/2025.08.22.25334232"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"8. Roustan D, Bastardot F. The Clinicians\u2019 Guide to Large Language Models: A General Perspective With a Focus on Hallucinations. Interact J Med Res. 2025;14:e59823. doi:10.2196\/59823","DOI":"10.2196\/59823"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"9. Chang Z, Li M, Jia X, et al. One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems. In: Christodoulopoulos C, Chakraborty T, Rose C, Peng V, eds. Findings of the Association for Computational Linguistics: EMNLP 2025. Association for Computational Linguistics; 2025:18811\u201318825. doi:10.18653\/v1\/2025.findings-emnlp.1023","DOI":"10.18653\/v1\/2025.findings-emnlp.1023"},{"key":"ref10","unstructured":"10. Kirchner JH, Chen Y, Edwards H, Leike J, McAleese N, Burda Y. Prover-Verifier Games improve legibility of LLM outputs. arXiv. Preprint posted online August 1, 2024:arXiv:2407.13692. doi:10.48550\/arXiv.2407.13692"},{"key":"ref11","unstructured":"11. Garcez A d\u2019Avila, Lamb LC. Neurosymbolic AI: The 3rd Wave. arXiv. Preprint posted online December 16, 2020:arXiv:2012.05876. doi:10.48550\/arXiv.2012.05876"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"12. Vidal ME, Chudasama Y, Huang H, Purohit D, Torrente M. Integrating Knowledge Graphs with Symbolic AI: The Path to Interpretable Hybrid AI Systems in Medicine. J Web Semant. 2025;84:100856. doi:10.1016\/j.websem.2024.100856","DOI":"10.1016\/j.websem.2024.100856"},{"key":"ref13","unstructured":"13. IOS Press Ebooks - Neuro-Symbolic Artificial Intelligence: The State of the Art. Accessed January 30, 2026. https:\/\/ebooks.iospress.nl\/volume\/neuro-symbolic-artificial-intelligence-the-state-of-the-art"},{"key":"ref14","unstructured":"14. Marcus G. The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence. arXiv. Preprint posted online February 19, 2020:arXiv:2002.06177. doi:10.48550\/arXiv.2002.06177"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"15. Bender EM, Koller A. Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. In: Jurafsky D, Chai J, Schluter N, Tetreault J, eds. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics; 2020:5185\u20135198. doi:10.18653\/v1\/2020.acl-main.463","DOI":"10.18653\/v1\/2020.acl-main.463"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"16. Kautz H. The Third AI Summer: AAAI Robert S. Engelmore Memorial Lecture. AI Mag. 2022;43(1):105\u2013125. doi:10.1002\/aaai.12036","DOI":"10.1002\/aaai.12036"},{"key":"ref17","doi-asserted-by":"crossref","unstructured":"17. Liu S, McCoy AB, Chen Q, Wright A. Integrating rule-based NLP and large language models for statin information extraction from clinical notes. Int J Med Inf. 2026;205:106104. doi:10.1016\/j.ijmedinf.2025.106104","DOI":"10.1016\/j.ijmedinf.2025.106104"},{"key":"ref18","unstructured":"18. Wu J, Zhu J, Qi Y, et al. Medical Graph RAG: Evidence-based Medical Large Language Model via Graph Retrieval-Augmented Generation."},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"19. Wu X, Kreider MS, Empey PE, Li C, Wang Y. Automated extraction of fluoropyrimidine treatment and treatment-related toxicities from clinical notes using natural language processing. Int J Med Inf. 2026;209:106276. doi:10.1016\/j.ijmedinf.2026.106276","DOI":"10.1016\/j.ijmedinf.2026.106276"},{"key":"ref20","unstructured":"20. Jia M, Duan J, Song Y, Wang J. medIKAL: Integrating Knowledge Graphs as Assistants of LLMs for Enhanced Clinical Diagnosis on EMRs."},{"key":"ref21","unstructured":"21. Rezaei MR, Fard RS, Parker JL, Krishnan RG, Lankarany M. Agentic Medical Knowledge Graphs Enhance Medical Question Answering: Bridging the Gap Between LLMs and Evolving Medical Knowledge."},{"key":"ref22","unstructured":"22. Tan X, Li B, Xu W, et al. Prolog-Driven Rule-Based Diagnostics with Large Language Models for Precise Clinical Decision Support."},{"key":"ref23","doi-asserted-by":"crossref","unstructured":"23. Oniani D, Wu X, Visweswaran S, et al. Enhancing Large Language Models for Clinical Decision Support by Incorporating Clinical Practice Guidelines. In: 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI). IEEE; 2024:694\u2013702. doi:10.1109\/ICHI61247.2024.00111","DOI":"10.1109\/ICHI61247.2024.00111"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"24. He Y, Chai Y, Liu Y, et al. RSA-KG: A Graph\u2010Based Rag Enhanced AI Knowledge Graph for Recurrent Spontaneous Abortions Diagnosis and Clinical Decision Support. Med Res. 2025;1(3):412\u2013423. doi:10.1002\/mdr2.70039","DOI":"10.1002\/mdr2.70039"},{"key":"ref25","doi-asserted-by":"crossref","unstructured":"25. Gao Y, Li R, Croxford E, et al. Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Study. JMIR AI. 2025;4:e58670. doi:10.2196\/58670","DOI":"10.2196\/58670"},{"key":"ref26","doi-asserted-by":"crossref","unstructured":"26. Wang MH. An Explainable AI Framework for Corneal Imaging Interpretation and Refractive Surgery Decision Support. Bioengineering. 2025;12(11):1174. doi:10.3390\/bioengineering12111174","DOI":"10.3390\/bioengineering12111174"},{"key":"ref27","doi-asserted-by":"crossref","unstructured":"27. Evangelista E, Ruba F, Bukhari S, Nazir A, Sharma R. GraphRAG-Enabled Local Large Language Model for Gestational Diabetes Mellitus: Development of a Proof-of-Concept. JMIR Diabetes. 2026;11:e76454-e76454. doi:10.2196\/76454","DOI":"10.2196\/76454"},{"key":"ref28","doi-asserted-by":"crossref","unstructured":"28. Hu Y, Xuan W, Zhou Q, et al. A self-correcting Agentic Graph RAG for clinical decision support in hepatology. Front Med. 2025;12:1716327. doi:10.3389\/fmed.2025.1716327","DOI":"10.3389\/fmed.2025.1716327"},{"key":"ref29","doi-asserted-by":"crossref","unstructured":"29. Yang H, Li J, Zhang C, Sierra AP, Shen B. Large Language Model\u2013Driven Knowledge Graph Construction in Sepsis Care Using Multicenter Clinical Databases: Development and Usability Study. J Med Internet Res. 2025;27:e65537. doi:10.2196\/65537","DOI":"10.2196\/65537"},{"key":"ref30","unstructured":"30. Lee T, Kim H, Lee S, Chae S, Kahn CE. Large Language Model-Generated Expansion of the RadLex Ontology: Application to Multinational Datasets of Chest CT Reports."},{"key":"ref31","doi-asserted-by":"crossref","unstructured":"31. Liu Y, Chen J, Lin X, Song J, Chen S. Development of a large language model\u2013based knowledge graph for chemotherapy-induced nausea and vomiting in breast cancer and its implications for nursing. Int J Nurs Sci. 2025;12(6):524\u2013531. doi:10.1016\/j.ijnss.2025.10.010","DOI":"10.1016\/j.ijnss.2025.10.010"},{"key":"ref32","doi-asserted-by":"crossref","unstructured":"32. Liu S, Wright AP, McCoy AB, Huang SS, Steitz B, Wright A. Detecting emergencies in patient portal messages using large language models and knowledge graph-based retrieval-augmented generation. J Am Med Inform Assoc. 2025;32(6):1032\u20131039. doi:10.1093\/jamia\/ocaf059","DOI":"10.1093\/jamia\/ocaf059"},{"key":"ref33","doi-asserted-by":"crossref","unstructured":"33. Cao S, Li R, Wu R, Liu R, Duprey A, Zhao J. Real-time clinical analytics at scale: a platform built on large language models-powered knowledge graphs. JAMIA Open. 2026;9(1):ooaf167. doi:10.1093\/jamiaopen\/ooaf167","DOI":"10.1093\/jamiaopen\/ooaf167"},{"key":"ref34","doi-asserted-by":"crossref","unstructured":"34. Garc\u00eda-Barrag\u00e1n \u00c1, Sakor A, Vidal ME, et al. NSSC: a neuro-symbolic AI system for enhancing accuracy of named entity recognition and linking from oncologic clinical notes. Med Biol Eng Comput. 2025;63(3):749\u2013772. doi:10.1007\/s11517-024-03227-4","DOI":"10.1007\/s11517-024-03227-4"},{"key":"ref35","doi-asserted-by":"crossref","unstructured":"35. Prenosil GA, Weitzel TK, Bello SC, et al. Neuro-symbolic AI for auditable cognitive information extraction from medical reports. Commun Med. 2025;5(1):491. doi:10.1038\/s43856-025-01194-x","DOI":"10.1038\/s43856-025-01194-x"},{"key":"ref36","doi-asserted-by":"crossref","unstructured":"36. Zhuang Y, Yu L, Jiang N, Ge Y. TCM-KLLaMA: Intelligent generation model for Traditional Chinese Medicine Prescriptions based on knowledge graph and large language model. Comput Biol Med. 2025;189:109887. doi:10.1016\/j.compbiomed.2025.109887","DOI":"10.1016\/j.compbiomed.2025.109887"},{"key":"ref37","doi-asserted-by":"crossref","unstructured":"37. Kant I, Gregor MJ, Kant I. Groundwork of the Metaphysics of Morals. Cambridge Univ. Press; 1998.","DOI":"10.1017\/CBO9780511809590"},{"key":"ref38","doi-asserted-by":"crossref","unstructured":"38. Bhuyan BP, Ramdane-Cherif A, Tomar R, Singh TP. Neuro-symbolic artificial intelligence: a survey. Neural Comput Appl. 2024;36(21):12809\u201312844. doi:10.1007\/s00521-024-09960-z","DOI":"10.1007\/s00521-024-09960-z"},{"key":"ref39","doi-asserted-by":"crossref","unstructured":"39. Agrawal G, Kumarage T, Alghamdi Z, Liu H. Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Survey. In: Duh K, Gomez H, Bethard S, eds. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Association for Computational Linguistics; 2024:3947\u20133960. doi:10.18653\/v1\/2024.naacl-long.219","DOI":"10.18653\/v1\/2024.naacl-long.219"},{"key":"ref40","doi-asserted-by":"crossref","unstructured":"40. Huang L, Yu W, Ma W, et al. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Trans Inf Syst. 2025;43(2):42:1\u201342:55. doi:10.1145\/3703155","DOI":"10.1145\/3703155"},{"key":"ref41","doi-asserted-by":"crossref","unstructured":"41. Prenosil GA, Weitzel TK, Bello SC, et al. Neuro-symbolic AI for auditable cognitive information extraction from medical reports. Commun Med. 2025;5:491. doi:10.1038\/s43856-025-01194-x","DOI":"10.1038\/s43856-025-01194-x"},{"key":"ref42","unstructured":"42. Luo L, Zhao Z, Haffari G, Li YF, Gong C, Pan S. Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models. In: Proceedings of the 42nd International Conference on Machine Learning. PMLR; 2025:41540\u201341565. Accessed January 30, 2026. https:\/\/proceedings.mlr.press\/v267\/luo25t.html"},{"key":"ref43","unstructured":"43. Fanconi C, Schaar M van der. Cascaded Language Models for Cost-effective Human-AI Decision-Making. arXiv. Preprint posted online October 24, 2025:arXiv:2506.11887. doi:10.48550\/arXiv.2506.11887"},{"key":"ref44","doi-asserted-by":"crossref","unstructured":"44. Mero\u00f1o-Pe\u00f1uela A, Simperl E, Kurteva A, Reklos I. KG.GOV: Knowledge graphs as the backbone of data governance in AI. J Web Semant. 2025;85:100847. doi:10.1016\/j.websem.2024.100847","DOI":"10.1016\/j.websem.2024.100847"},{"key":"ref45","doi-asserted-by":"crossref","unstructured":"45. Wilkinson MD, Dumontier M, Aalbersberg IjJ, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3(1):160018. doi:10.1038\/sdata.2016.18","DOI":"10.1038\/sdata.2016.18"},{"key":"ref46","unstructured":"46. SNOBERT: A Benchmark for clinical notes entity linking in the SNOMED CT clinical terminology. Accessed January 30, 2026. https:\/\/arxiv.org\/html\/2405.16115v1?utm_source=chatgpt.com"},{"key":"ref47","doi-asserted-by":"crossref","unstructured":"47. French E, Mclnnes BT. An overview of Biomedical Entity Linking throughout the years. J Biomed Inform. 2023;137:104252. doi:10.1016\/j.jbi.2022.104252","DOI":"10.1016\/j.jbi.2022.104252"},{"key":"ref48","doi-asserted-by":"crossref","unstructured":"48. Faust L, Wilson P, Asai S, et al. Considerations for Quality Control Monitoring of Machine Learning Models in Clinical Practice. JMIR Med Inform. 2024;12(1):e50437. doi:10.2196\/50437","DOI":"10.2196\/50437"},{"key":"ref49","unstructured":"49. Distribution shift detection for the postmarket surveillance of medical AI algorithms: a retrospective simulation study | npj Digital Medicine. Accessed January 30, 2026. https:\/\/www.nature.com\/articles\/s41746-024-01085-w?utm_source=chatgpt.com"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/www.researchsquare.com\/article\/rs-8902022\/v1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.researchsquare.com\/article\/rs-8902022\/v1.html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T14:12:56Z","timestamp":1771423976000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchsquare.com\/article\/rs-8902022\/v1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,18]]},"references-count":49,"URL":"https:\/\/doi.org\/10.21203\/rs.3.rs-8902022\/v1","relation":{},"subject":[],"published":{"date-parts":[[2026,2,18]]},"subtype":"preprint"}}