{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T16:10:10Z","timestamp":1792512610056,"version":"4.5.0"},"reference-count":46,"publisher":"American Medical Association (AMA)","issue":"10","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"\n Importance<\/jats:title>\n Presenting signs and symptoms affect the care of patients with possible sepsis. However, signs and symptoms are not incorporated into most large observational studies because they are difficult to extract from clinical notes at scale.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n To assess the use of large language models (LLMs) to extract presenting signs and symptoms from admission notes and characterize their associations with infectious diagnoses, multidrug-resistant infections, and mortality.<\/jats:p>\n <\/jats:sec>\n \n Design, Setting, and Participants<\/jats:title>\n This retrospective cohort study obtained data from 5 Massachusetts hospitals within 1 health care system between June 1, 2015, and August 1, 2022. Participants were hospitalized adult patients with possible infection (determined by blood culture drawn and intravenous antibiotics administered within 24 hours of arrival). An LLM (LLaMA 3 8B; Meta) was used to extract up to 10 presenting signs and symptoms from each patient\u2019s history-and-physical admission notes. LLM-generated labels were validated by blinded review of 303 random admission notes. Data analyses were performed from July 2023 to August 2025.<\/jats:p>\n <\/jats:sec>\n \n Exposures<\/jats:title>\n \n Thirty most common signs and symptoms were retained as exposures, and unsupervised clustering was used to create syndromes, which were compared with infection sources derived from the\n International Statistical Classification of Diseases, Tenth Revision, Clinical Modification<\/jats:italic>\n discharge codes.\n <\/jats:p>\n <\/jats:sec>\n \n Main Outcomes and Measures<\/jats:title>\n \n Outcomes included positive cultures for methicillin-resistant\n Staphylococcus aureus<\/jats:italic>\n (MRSA), positive cultures for multidrug-resistant gram-negative (MDRGN) organisms, and in-hospital mortality. Multivariable logistic regression was used to adjust for demographics, comorbidities, physiologic markers of severity of illness, and time to antibiotics.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Among the 104\u202f248 patients (median [IQR] age, 66 [52-78] years; 54 137 males [51.9%]) included, 23\u202f619 (22.7%) had sepsis without shock, 25\u202f990 (24.9%) had septic shock, and 94\u202f913 (91.0%) had 1 or more admission note within 24 hours. The LLM labeled the notes of 93\u202f674 of 94\u202f913 patients (98.7%). On manual validation, LLM labels had an accuracy of 99.3% (95% CI, 99.2%-99.3%), balanced accuracy of 84.6% (95% CI, 83.5%-85.8%), positive predictive value of 68.4% (95% CI, 66.0%-70.7%), sensitivity of 69.7% (95% CI, 67.3%-72.0%), and specificity of 99.6% (95% CI, 99.6%-99.6%) compared with the physician medical record reviewer. The 30 most common signs and symptoms were clustered into syndromes that correlated with infection sources. Presence of skin and soft tissue symptoms (adjusted odds ratio [AOR], 1.73; 95% CI, 1.49-2.00) and absence of gastrointestinal (AOR, 0.63; 95% CI, 0.54-0.73) or urinary tract symptoms (AOR, 0.34; 95% CI, 0.22-0.50) were associated with MRSA culture positivity; inverse associations were seen for MDRGN organisms. Cardiopulmonary symptoms were associated with increased mortality (AOR, 1.30; 95% CI, 1.17-1.45).<\/jats:p>\n <\/jats:sec>\n \n Conclusions and Relevance<\/jats:title>\n This cohort study found that an LLM accurately extracted presenting signs and symptoms from admission notes that clustered into syndromes differentially correlated with infection sources, multidrug-resistant infections, and mortality. Further research is warranted to evaluate the value of large-scale sign-and-symptom data in models of antibiotic choice, effectiveness, and outcomes in patients with possible sepsis.<\/jats:p>\n <\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2025.39267","type":"journal-article","created":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T15:30:42Z","timestamp":1761319842000},"page":"e2539267","source":"Crossref","is-referenced-by-count":7,"title":["Syndromic Analysis of Sepsis Cohorts Using Large Language Models"],"prefix":"10.1001","volume":"8","author":[{"given":"Theodore R.","family":"Pak","sequence":"first","affiliation":[{"name":"Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts"},{"name":"Division of Infectious Diseases, Department of Medicine, Massachusetts General Hospital, Boston"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sanjat","family":"Kanjilal","sequence":"additional","affiliation":[{"name":"Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts"},{"name":"Division of Infectious Diseases, Department of Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Caroline S.","family":"McKenna","sequence":"additional","affiliation":[{"name":"Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Hoffner-Heinike","sequence":"additional","affiliation":[{"name":"Department of Medicine, Massachusetts General Hospital, Boston"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chanu","family":"Rhee","sequence":"additional","affiliation":[{"name":"Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts"},{"name":"Division of Infectious Diseases, Department of Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Klompas","sequence":"additional","affiliation":[{"name":"Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute, Boston, Massachusetts"},{"name":"Division of Infectious Diseases, Department of Medicine, Brigham and Women\u2019s Hospital, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2025,10,24]]},"reference":[{"issue":"10219","key":"zoi251086r1","doi-asserted-by":"publisher","first-page":"200","DOI":"10.1016\/S0140-6736(19)32989-7","article-title":"Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the Global Burden of Disease Study.","volume":"395","author":"Rudd","year":"2020","journal-title":"Lancet"},{"issue":"4","key":"zoi251086r3","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1093\/cid\/ciaa059","article-title":"Infectious Diseases Society of America position paper: recommended revisions to the National Severe Sepsis and Septic Shock Early Management Bundle (SEP-1) sepsis quality measure.","volume":"72","author":"Rhee","year":"2021","journal-title":"Clin Infect Dis"},{"issue":"8","key":"zoi251086r4","doi-asserted-by":"publisher","first-page":"609","DOI":"10.7326\/L18-0139","article-title":"Evidence underpinning the Centers for Medicare & Medicaid Services\u2019 Severe Sepsis and Septic Shock Management Bundle (SEP-1).","volume":"168","author":"Townsend","year":"2018","journal-title":"Ann Intern Med"},{"issue":"4","key":"zoi251086r5","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.2899","article-title":"Prevalence of antibiotic-resistant pathogens in culture-proven sepsis and outcomes associated with inadequate and broad-spectrum empiric antibiotic use.","volume":"3","author":"Rhee","year":"2020","journal-title":"JAMA Netw Open"},{"issue":"15","key":"zoi251086r6","doi-asserted-by":"publisher","first-page":"1478","DOI":"10.1001\/jama.2020.2717","article-title":"Prevalence and outcomes of infection among patients in intensive care units in 2017.","volume":"323","author":"Vincent","year":"2020","journal-title":"JAMA"},{"issue":"6","key":"zoi251086r7","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.18923","article-title":"Trends in empiric broad-spectrum antibiotic use for suspected community-onset sepsis in US hospitals.","volume":"7","author":"Rhee","year":"2024","journal-title":"JAMA Netw Open"},{"issue":"9","key":"zoi251086r8","doi-asserted-by":"publisher","first-page":"1308","DOI":"10.1001\/jamainternmed.2017.1938","article-title":"Association of adverse events with antibiotic use in hospitalized patients.","volume":"177","author":"Tamma","year":"2017","journal-title":"JAMA Intern Med"},{"issue":"4","key":"zoi251086r9","doi-asserted-by":"publisher","first-page":"426","DOI":"10.1513\/AnnalsATS.201809-646PS","article-title":"Improving sepsis treatment by embracing diagnostic uncertainty.","volume":"16","author":"Prescott","year":"2019","journal-title":"Ann Am Thorac Soc"},{"issue":"6","key":"zoi251086r10","doi-asserted-by":"publisher","first-page":"1589","DOI":"10.1097\/01.CCM.0000217961.75225.E9","article-title":"Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock.","volume":"34","author":"Kumar","year":"2006","journal-title":"Crit Care Med"},{"issue":"23","key":"zoi251086r11","doi-asserted-by":"publisher","first-page":"2235","DOI":"10.1056\/NEJMoa1703058","article-title":"Time to treatment and mortality during mandated emergency care for sepsis.","volume":"376","author":"Seymour","year":"2017","journal-title":"N Engl J Med"},{"issue":"4","key":"zoi251086r12","doi-asserted-by":"publisher","first-page":"623","DOI":"10.1097\/CCM.0000000000002262","article-title":"Increased time to initial antimicrobial administration is associated with progression to septic shock in severe sepsis patients.","volume":"45","author":"Whiles","year":"2017","journal-title":"Crit Care Med"},{"issue":"7","key":"zoi251086r13","doi-asserted-by":"publisher","first-page":"856","DOI":"10.1164\/rccm.201609-1848OC","article-title":"The timing of early antibiotics and hospital mortality in sepsis.","volume":"196","author":"Liu","year":"2017","journal-title":"Am J Respir Crit Care Med"},{"issue":"5","key":"zoi251086r14","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1016\/j.chest.2019.02.008","article-title":"ED door-to-antibiotic time and long-term mortality in sepsis.","volume":"155","author":"Peltan","year":"2019","journal-title":"Chest"},{"issue":"4","key":"zoi251086r15","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1097\/CCM.0b013e3181cc4824","article-title":"Impact of time to antibiotics on survival in patients with severe sepsis or septic shock in whom early goal-directed therapy was initiated in the emergency department.","volume":"38","author":"Gaieski","year":"2010","journal-title":"Crit Care Med"},{"issue":"8","key":"zoi251086r16","doi-asserted-by":"publisher","first-page":"1749","DOI":"10.1097\/CCM.0000000000000330","article-title":"Empiric antibiotic treatment reduces mortality in severe sepsis and septic shock from the first hour: results from a guideline-based performance improvement program.","volume":"42","author":"Ferrer","year":"2014","journal-title":"Crit Care Med"},{"issue":"11","key":"zoi251086r17","doi-asserted-by":"publisher","first-page":"1534","DOI":"10.1093\/cid\/ciad450","article-title":"Risk of misleading conclusions in observational studies of time-to-antibiotics and mortality in suspected sepsis.","volume":"77","author":"Pak","year":"2023","journal-title":"Clin Infect Dis"},{"issue":"12","key":"zoi251086r18","doi-asserted-by":"publisher","first-page":"1803","DOI":"10.1093\/cid\/ciy378","article-title":"Difficult-to-treat resistance in gram-negative bacteremia at 173 US hospitals: retrospective cohort analysis of prevalence, predictors, and outcome of resistance to all first-line agents.","volume":"67","author":"Kadri","year":"2018","journal-title":"Clin Infect Dis"},{"key":"zoi251086r19","doi-asserted-by":"publisher","first-page":"S110","DOI":"10.1093\/infdis\/jiaa146","article-title":"A critical analysis of the literature on time-to-antibiotics in suspected sepsis.","volume":"222","author":"Weinberger","year":"2020","journal-title":"J Infect Dis"},{"issue":"10","key":"zoi251086r20","doi-asserted-by":"publisher","first-page":"1585","DOI":"10.1097\/CCM.0000000000003261","article-title":"Compliance with the national SEP-1 quality measure and association with sepsis outcomes: a multicenter retrospective cohort study.","volume":"46","author":"Rhee","year":"2018","journal-title":"Crit Care Med"},{"issue":"10","key":"zoi251086r21","doi-asserted-by":"publisher","first-page":"1592","DOI":"10.1097\/CCM.0000000000003260","article-title":"Presenting symptoms independently predict mortality in septic shock: importance of a previously unmeasured confounder.","volume":"46","author":"Filbin","year":"2018","journal-title":"Crit Care Med"},{"issue":"20","key":"zoi251086r22","doi-asserted-by":"publisher","first-page":"2003","DOI":"10.1001\/jama.2019.5791","article-title":"Derivation, validation, and potential treatment implications of novel clinical phenotypes for sepsis.","volume":"321","author":"Seymour","year":"2019","journal-title":"JAMA"},{"issue":"1","key":"zoi251086r23","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1186\/s13054-018-2279-3","article-title":"Identification of subclasses of sepsis that showed different clinical outcomes and responses to amount of fluid resuscitation: a latent profile analysis.","volume":"22","author":"Zhang","year":"2018","journal-title":"Crit Care"},{"issue":"5","key":"zoi251086r24","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1007\/s00134-015-3764-7","article-title":"Phenotypic clusters within sepsis-associated multiple organ dysfunction syndrome.","volume":"41","author":"Knox","year":"2015","journal-title":"Intensive Care Med"},{"issue":"10","key":"zoi251086r25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0267517","article-title":"Clustering ICU patients with sepsis based on the patterns of their circulating biomarkers: a secondary analysis of the CAPTAIN prospective multicenter cohort study.","volume":"17","author":"Misset","year":"2022","journal-title":"PLoS One"},{"key":"zoi251086r26","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1016\/j.jcrc.2018.06.012","article-title":"Six subphenotypes in septic shock: latent class analysis of the PROWESS Shock Study.","volume":"47","author":"G\u00e5rdlund","year":"2018","journal-title":"J Crit Care"},{"issue":"2","key":"zoi251086r27","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models.","volume":"2","author":"Kung","year":"2023","journal-title":"PLOS Digit Health"},{"issue":"7972","key":"zoi251086r28","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge.","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"issue":"10","key":"zoi251086r29","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.40969","article-title":"Large language model influence on diagnostic reasoning: a randomized clinical trial.","volume":"7","author":"Goh","year":"2024","journal-title":"JAMA Netw Open"},{"issue":"1","key":"zoi251086r30","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1001\/jama.2023.8288","article-title":"Accuracy of a generative artificial intelligence model in a complex diagnostic challenge.","volume":"330","author":"Kanjee","year":"2023","journal-title":"JAMA"},{"issue":"1","key":"zoi251086r32","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1038\/s41746-024-01233-2","article-title":"Privacy-preserving large language models for structured medical information retrieval.","volume":"7","author":"Wiest","year":"2024","journal-title":"NPJ Digit Med"},{"issue":"5","key":"zoi251086r33","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2025.12032","article-title":"Identification of long-term care facility residence from admission notes using large language models.","volume":"8","author":"Goodman","year":"2025","journal-title":"JAMA Netw Open"},{"issue":"8","key":"zoi251086r35","doi-asserted-by":"publisher","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine.","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat Med"},{"issue":"4","key":"zoi251086r36","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1093\/cid\/ciad633","article-title":"Black box warning: large language models and the future of infectious diseases consultation.","volume":"78","author":"Schwartz","year":"2024","journal-title":"Clin Infect Dis"},{"issue":"8","key":"zoi251086r37","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1001\/jama.2024.0555","article-title":"AI-generated clinical summaries require more than accuracy.","volume":"331","author":"Goodman","year":"2024","journal-title":"JAMA"},{"issue":"5","key":"zoi251086r41","doi-asserted-by":"publisher","first-page":"4961","DOI":"10.1007\/s10489-021-02635-5","article-title":"Confidence interval for micro-averaged F1 and macro-averaged F1 scores.","volume":"52","author":"Takahashi","year":"2022","journal-title":"Appl Intell (Dordr)"},{"issue":"12","key":"zoi251086r42","doi-asserted-by":"publisher","first-page":"1928","DOI":"10.1097\/CCM.0000000000006428","article-title":"Assessment of racial, ethnic, and sex-based disparities in time-to-antibiotics and sepsis outcomes in a large multihospital cohort.","volume":"52","author":"Pak","year":"2024","journal-title":"Crit Care Med"},{"issue":"12","key":"zoi251086r43","doi-asserted-by":"publisher","first-page":"1084","DOI":"10.1001\/jama.2024.28047","article-title":"Manual vs AI-assisted prescreening for trial eligibility using large language models: a randomized clinical trial.","volume":"333","author":"Unlu","year":"2025","journal-title":"JAMA"},{"issue":"11","key":"zoi251086r44","doi-asserted-by":"publisher","DOI":"10.1056\/AIcs2400420","article-title":"Large language models for more efficient reporting of hospital quality measures.","volume":"1","author":"Boussina","year":"2024","journal-title":"NEJM AI"},{"issue":"5","key":"zoi251086r45","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.8895","article-title":"Use of a large language model to assess clinical acuity of adults in the emergency department.","volume":"7","author":"Williams","year":"2024","journal-title":"JAMA Netw Open"},{"issue":"12","key":"zoi251086r46","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.48723","article-title":"Developing and evaluating large language model-generated emergency medicine handoff notes.","volume":"7","author":"Hartman","year":"2024","journal-title":"JAMA Netw Open"},{"key":"zoi251086r34","first-page":"82","volume-title":"Proceedings of the 3rd Machine Learning for Health Symposium","author":"Goel","year":"2023"},{"key":"zoi251086r2","unstructured":"Dwyer? J. One boy\u2019s death moves state to action to prevent others. The New York Times. December 20, 2012. Accessed April 5, 2023. https:\/\/www.nytimes.com\/2012\/12\/21\/nyregion\/one-boys-death-moves-state-to-action-to-prevent-others.html"},{"key":"zoi251086r31","doi-asserted-by":"crossref","unstructured":"Agrawal? M, Hegselmann? S, Lang? H, Kim? Y, Sontag? D. Large language models are few-shot clinical information extractors.? arXiv. Preprint posted online May 25, 2022. doi:10.18653\/v1\/2022.emnlp-main.130","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"key":"zoi251086r38","doi-asserted-by":"crossref","unstructured":"Kwon? W, Li? Z, Zhuang? S, . Efficient memory management for large language model serving with PagedAttention.? arXiv. Preprint posted online September 12, 2023. doi:10.1145\/3600006.3613165","DOI":"10.1145\/3600006.3613165"},{"key":"zoi251086r39","unstructured":"Grattafiori? A, Dubey? A, Jauhri? A, . The Llama 3 herd of models.? arXiv. Preprint posted online July 31, 2024. doi:10.48550\/arXiv.2407.21783"},{"key":"zoi251086r40","unstructured":"Willard? BT, Louf? R. Efficient guided generation for large language models.? arXiv. Preprint posted online July 18, 2023. doi:10.48550\/arXiv.2307.09702"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2840524\/pak_2025_oi_251086_1760718018.14066.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T15:30:43Z","timestamp":1761319843000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2840524"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,24]]},"references-count":46,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2025,10,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2025.39267","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,24]]}}}