{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"medRxiv"}],"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T04:00:56Z","timestamp":1793073656812,"version":"4.5.0"},"posted":{"date-parts":[[2026,7,17]]},"group-title":"Neurology","reference-count":0,"publisher":"openRxiv","license":[{"start":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T00:00:00Z","timestamp":1784246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.medrxiv.org\/about\/FAQ#license"}],"funder":[{"award":["R01NS098023"],"award-info":[{"award-number":["R01NS098023"]}],"id":[{"id":"https:\/\/ror.org\/01cwqze88","id-type":"ROR","asserted-by":"publisher"}]},{"award":["UL1TR001857"],"award-info":[{"award-number":["UL1TR001857"]}],"id":[{"id":"https:\/\/ror.org\/01cwqze88","id-type":"ROR","asserted-by":"publisher"}]},{"name":"University of Pittsburgh Clinical Scientist Training Program","award":["NA"],"award-info":[{"award-number":["NA"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2026,7,17]]},"abstract":"<jats:p>Background. Early diagnosis and intervention are crucial in multiple sclerosis (MS), yet diagnostic delays are common. Large language models (LLMs) such as generative pre-trained transformers (GPTs) may help streamline diagnostic workflows by extracting MS diagnostic signals from clinical notes.\nObjective. To derive MS diagnosis status from the first neurology note using a computable algorithm based on the 2017 McDonald criteria and applying GPT-4 for node-level reasoning within a structured decision framework.\nMethods. We analyzed first neurology notes from 125 randomly selected patients (including those with MS, related disorders, and controls) enrolled in a clinic cohort between 2017 and 2023. We included the clinical history and diagnostic testing sections but redacted the assessment and plan. We converted the 2017 McDonald criteria into a decision tree and provided expert-curated clinical knowledge to guide GPT-4 reasoning at each decision node. GPT-4 generated binary decisions at each node to traverse the tree and classified MS diagnoses at terminal nodes. We evaluated performance against neurologist-assessed diagnoses and characterized hallucinations (non-factual, incongruent, irrelevant, over-reliant, and logical reasoning errors).\nResults. In this study cohort (mean age 40\u00b113 years; 81% women) representative of the clinic population, GPT-4 performed well in predicting MS diagnosis (84% accuracy, 79% precision, 74% recall, 91% specificity) using first neurology notes. Hallucinations occurred in 32 cases (26%), most commonly incoherence (75%) and overreliance (47%).\nConclusion. A structured, LLM-guided decision framework can flag MS diagnoses from early clinical documentation. Large-scale studies are needed to mitigate hallucinations, validate this approach, and test implementation in clinical settings.<\/jats:p>","DOI":"10.64898\/2026.07.14.26357416","type":"posted-content","created":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T17:50:18Z","timestamp":1784310618000},"source":"Crossref","is-referenced-by-count":1,"title":["Large Language Model - Enhanced Decision Tree Framework for Identifying Multiple Sclerosis Diagnoses from Clinical Documentation"],"prefix":"10.64898","author":[{"given":"Shruthi","family":"Venkatesh","sequence":"first","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marisa","family":"DelSignore","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xizhi","family":"Wu","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michele","family":"Morris","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wesley T","family":"Kerr","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shyam","family":"Visweswaran","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yanshan","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zongqi","family":"Xia","sequence":"additional","affiliation":[{"name":"University of Pittsburgh"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"54368","container-title":[],"original-title":[],"link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.64898\/2026.07.14.26357416","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T17:50:18Z","timestamp":1784310618000},"score":1,"resource":{"primary":{"URL":"http:\/\/medrxiv.org\/lookup\/doi\/10.64898\/2026.07.14.26357416"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,17]]},"references-count":0,"URL":"https:\/\/doi.org\/10.64898\/2026.07.14.26357416","relation":{},"subject":[],"published":{"date-parts":[[2026,7,17]]},"subtype":"preprint"}}