{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:06:11Z","timestamp":1792785971827,"version":"4.5.0"},"reference-count":24,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T00:00:00Z","timestamp":1790812800000},"content-version":"vor","delay-in-days":22,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Institutes of Health National Center for Complementary and Integrative Health","award":["U54AT012307"],"award-info":[{"award-number":["U54AT012307"]}],"id":[{"id":"https:\/\/ror.org\/00190t495","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,9,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>To produce the first structured longitudinal dataset of symptom severity trajectories in vagus nerve stimulation (VNS) patients using a 2-stage large language model pipeline for automated symptom severity extraction from clinical notes, as part of the NIH-funded research evaluating vagal excitations and anatomical linkages (U54AT012307).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and methods<\/jats:title>\n                    <jats:p>We analyzed 1427 annotated clinical notes from 95 patients across 5 corpora. LLaMA models (3.2-1B, 3.2-3B, and 3.3-70B) were fine-tuned to classify 13 symptoms as present, absent, or negated (stage 1), then assessed severity (indeterminate, mild, moderate, or severe; stage 2). Longitudinal trajectories were tracked quarterly in 23 VNS patients against a 10-patient surgical control cohort (NSQIP).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Severity F1-scores ranged from 0.76 to 0.96, and symptom classification reached 0.99 for high-prevalence symptoms, with the validation corpus (VNS vs NSQIP) as the dominant factor (29.4% of variance; no significant source-corpus effect), supporting cross-domain generalizability. Macro F1-scores were lower than weighted F1-scores for smaller models and NSQIP; suicidal-ideation sensitivity was lower on NSQIP than VNS (0.80 vs 1.00, largest model), limiting generalization for psychiatric risk detection. Equity analyses were exploratory given small subgroups (gender precision 0.95). The longitudinal dataset revealed a median time to clinically meaningful seizure improvement of 0.5 quarters, with 50% responders, 25% non-responders, and 25% worsened. Post-surgical decline was absent in the unmatched NSQIP control cohort, consistent with, but not confirming, VNS specificity.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>To our knowledge, this is the first system producing structured symptom trajectories from free-text VNS documentation, supporting individualized over population-level monitoring.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This dataset supports future treatment-response prediction, personalized device programming, and a reproducible clinical natural langauge processing benchmark for neuromodulation research.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooag195","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T11:26:19Z","timestamp":1790335579000},"source":"Crossref","is-referenced-by-count":0,"title":["Longitudinal symptom severity tracking in vagus nerve stimulation patients: a 2-stage LLM-based pipeline with explainable AI"],"prefix":"10.1093","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-3551-5229","authenticated-orcid":false,"given":"Vedansh","family":"Thakkar","sequence":"first","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Natural Language Processing\/Information Extraction Program, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Greg M","family":"Silverman","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Natural Language Processing\/Information Extraction Program, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maura","family":"Putzer","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhinab","family":"Kc","sequence":"additional","affiliation":[{"name":"University of Minnesota Medical School , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sana","family":"Ikramuddin","sequence":"additional","affiliation":[{"name":"University of Minnesota Medical School , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tanya","family":"Melnik","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Natural Language Processing\/Information Extraction Program, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0292-0594","authenticated-orcid":false,"given":"Nicholas E","family":"Ingraham","sequence":"additional","affiliation":[{"name":"Department of Pulmonary, Allergy, Critical Care, and Sleep Medicine, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Center for Learning Health Systems Sciences, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathan","family":"Keller","sequence":"additional","affiliation":[{"name":"University of Minnesota Medical School , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samantha","family":"King","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Washington , Seattle, WA 98195,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Emma","family":"Jones","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Angela","family":"Bailey","sequence":"additional","affiliation":[{"name":"University of Minnesota Medical School , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5193-1663","authenticated-orcid":false,"given":"Genevieve B","family":"Melton","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Natural Language Processing\/Information Extraction Program, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Center for Learning Health Systems Sciences, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8079-5565","authenticated-orcid":false,"given":"Christopher","family":"Tignanelli","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Natural Language Processing\/Information Extraction Program, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]},{"name":"Center for Learning Health Systems Sciences, University of Minnesota , Minneapolis, MN 55455,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,10,1]]},"reference":[{"key":"2026100119541754200_ooag195-B1","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1007\/s10286-024-01065-w","article-title":"Vagus nerve stimulation (VNS): recent advances and future directions","volume":"34","author":"Austelle","year":"2024","journal-title":"Clin Auton Res"},{"key":"2026100119541754200_ooag195-B2","doi-asserted-by":"publisher","first-page":"780","DOI":"10.3390\/jcm13030780","article-title":"Vagus nerve stimulation therapy for drug-resistant epilepsy in children\u2014a literature review","volume":"13","author":"Fukuda","year":"2024","journal-title":"JCM"},{"key":"2026100119541754200_ooag195-B3","doi-asserted-by":"publisher","first-page":"2679","DOI":"10.3390\/ijms25052679","article-title":"Vagus nerve stimulation modulates inflammation in treatment-resistant depression patients: a pilot study","volume":"25","author":"Lesp\u00e9rance","year":"2024","journal-title":"IJMS"},{"key":"2026100119541754200_ooag195-B4","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1016\/j.cnur.2009.07.011","article-title":"Science, technology, and innovation: nursing responsibilities in clinical research","volume":"44","author":"Grady","year":"2009","journal-title":"Nurs Clin North Am"},{"key":"2026100119541754200_ooag195-B5","first-page":"429","article-title":"NLP methods for UMLS concept extraction of symptoms from unstructured data for use in prognostic COVID-19 analytic models","volume":"26","author":"Silverman","year":"2020","journal-title":"Published Online August"},{"key":"2026100119541754200_ooag195-B6","doi-asserted-by":"publisher","author":"Grattafiori","year":"2024","DOI":"10.48550\/ARXIV.2407.21783"},{"key":"2026100119541754200_ooag195-B7","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1613\/jair.1.12631","article-title":"NLP methods for extraction of symptoms from unstructured data for use in prognostic COVID-19 analytic models","volume":"72","author":"Silverman","year":"2021","journal-title":"J Artif Intell Res"},{"key":"2026100119541754200_ooag195-B8","doi-asserted-by":"publisher","author":"Sahoo","DOI":"10.1093\/jamiaopen\/ooab070"},{"key":"2026100119541754200_ooag195-B9","doi-asserted-by":"publisher","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol Bull"},{"key":"2026100119541754200_ooag195-B10","doi-asserted-by":"publisher","author":"Sahoo","year":"2024","DOI":"10.48550\/ARXIV.2402.07927"},{"key":"2026100119541754200_ooag195-B11","year":"2025"},{"key":"2026100119541754200_ooag195-B12","doi-asserted-by":"publisher","first-page":"e0323535","DOI":"10.1371\/journal.pone.0323535","article-title":"A comparative analysis of large language models versus traditional information extraction methods for real-world evidence of patient symptomatology in acute and post-acute sequelae of SARS-CoV-2","volume":"20","author":"Thakkar","year":"2025","journal-title":"PLoS One"},{"key":"2026100119541754200_ooag195-B13"},{"key":"2026100119541754200_ooag195-B14"},{"key":"2026100119541754200_ooag195-B15","doi-asserted-by":"publisher","first-page":"905","DOI":"10.3390\/children11080905","article-title":"Efficacy and safety of vagus nerve stimulation in Lennox-Gastaut syndrome: a scoping review","volume":"11","author":"Samanta","year":"2024","journal-title":"Children (Basel)"},{"key":"2026100119541754200_ooag195-B16","doi-asserted-by":"publisher","first-page":"ooae075","DOI":"10.1093\/jamiaopen\/ooae075","article-title":"Generalizable clinical note section identification with large language models","volume":"7","author":"Zhou","year":"2024","journal-title":"JAMIA Open"},{"key":"2026100119541754200_ooag195-B17","first-page":"432","author":"Peng"},{"key":"2026100119541754200_ooag195-B18","volume-title":"Deep Medicine: How Artificial Intelligence Can Make Healthcare Human Again","author":"Topol"},{"key":"2026100119541754200_ooag195-B19","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1093\/jamia\/ocy173","article-title":"Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review","volume":"26","author":"Koleck","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2026100119541754200_ooag195-B20","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1136\/amiajnl-2012-001377","article-title":"Understanding differences in electronic health record (EHR) use: linking individual physicians\u2019 perceptions of uncertainty and EHR use patterns in ambulatory care","volume":"21","author":"Lanham","year":"2014","journal-title":"J Am Med Inf Assoc"},{"key":"2026100119541754200_ooag195-B21","first-page":"530","article-title":"Agreement between patient-reported symptoms and their documentation in the medical record","volume":"14","author":"Pakhomov","year":"2008","journal-title":"Am J Manag Care"},{"key":"2026100119541754200_ooag195-B22","doi-asserted-by":"publisher","first-page":"103502","DOI":"10.1016\/j.artint.2021.103502","article-title":"Explaining individual predictions when features are dependent: more accurate approximations to shapley values","volume":"298","author":"Aas","year":"2021","journal-title":"Artif Intell"},{"key":"2026100119541754200_ooag195-B23","doi-asserted-by":"publisher","first-page":"866","DOI":"10.7326\/M18-1990","article-title":"Ensuring fairness in machine learning to advance health equity","volume":"169","author":"Rajkomar","year":"2018","journal-title":"Ann Intern Med"},{"key":"2026100119541754200_ooag195-B24","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","article-title":"Dissecting racial bias in an algorithm used to manage the health of populations","volume":"366","author":"Obermeyer","year":"2019","journal-title":"Science (1979)"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/5\/ooag195\/71562233\/ooag195.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/5\/ooag195\/71562233\/ooag195.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T23:54:25Z","timestamp":1790898865000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag195\/8859355"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,9]]},"references-count":24,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,9,9]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooag195","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,10]]},"published":{"date-parts":[[2026,9,9]]},"article-number":"ooag195"}}