{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T08:36:31Z","timestamp":1784709391694,"version":"3.55.0"},"reference-count":21,"publisher":"American Medical Association (AMA)","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"Importance<\/jats:title>An estimated half of all long-term care facility (LTCF) residents are colonized with antimicrobial-resistant organisms, and early identification of these patients on admission to acute care hospitals is a core strategy for preventing intrahospital spread. However, because LTCF exposure is not reliably captured in structured electronic health record data, LTCF-exposed patients routinely go undetected. Large language models (LLMs) offer a promising, but untested, opportunity for extracting this information from patient admission histories.<\/jats:p><\/jats:sec>Objective<\/jats:title>To evaluate the performance of an LLM against human review for identifying recent LTCF exposure from identifiable patient admission histories.<\/jats:p><\/jats:sec>Design, Setting, and Participants<\/jats:title>This cross-sectional, multicenter study used the history and physical (H&amp;amp;P) notes from unique, randomly sampled adult admissions occurring between January 1, 2016, and December 31, 2021, at 13 hospitals in the University of Maryland Medical System (UMMS) and the John Hopkins (Hopkins) health care system to compare the performance of an LLM (GPT-4-Turbo) using zero-shot learning and prompting against humans in identifying patients with recent LTCF exposure. LLM analyses were conducted from August to September 2024.<\/jats:p><\/jats:sec>Exposure<\/jats:title>Recent (\u226412 months) LTCF exposure documented in the H&amp;amp;P note, as adjudicated by (1) humans and (2) an LLM.<\/jats:p><\/jats:sec>Main Outcomes and Measures<\/jats:title>LLM sensitivity and specificity with Clopper-Pearson 95% CIs. Secondary outcomes were note review time and cost. The LLM was also prompted to provide a rationale and supporting note-text for each classification.<\/jats:p><\/jats:sec>Results<\/jats:title>The study included 359\u202f601 eligible adult admissions, of which 2087 randomly sampled H&amp;amp;P notes were manually reviewed at UMMS (1020 individuals; median [IQR] age, 58 [41-71] years; 493 [48%] male) and Hopkins (1067 individuals; median [IQR] age, 58 [48-67] years; 561 [53%] male) for LTCF residence. Compared with human review, the LLM achieved a sensitivity of 97% (95% CI, 91%-100%) and a specificity of 98% (95% CI, 97%-99%) at UMMS, and 96% (95% CI, 86%-100%) and 93% (95% CI, 92%-95%) sensitivity and specificity, respectively, at Hopkins; specificity at Hopkins improved with prompt revision (96% [95% CI, 95%-97%]). Of 117 manually reviewed LLM rationales, all were factually correct and quoted note-text accurately, and some demonstrated inferential logic and external knowledge. The LLM identified 37 (1.8%) human errors. Human review time had a mean of 2.5 minutes and cost $0.63 to $0.83 per note vs a mean of 4 to 6 seconds and $0.03 per note for LLM review.<\/jats:p><\/jats:sec>Conclusions and Relevance<\/jats:title>In this 13-hospital study of 2087 adult admissions, an LLM accurately identified LTCF residence from H&amp;amp;P notes and was more than 25 times faster and 20 times less expensive than human review.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2025.12032","type":"journal-article","created":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T15:31:14Z","timestamp":1747927874000},"page":"e2512032","source":"Crossref","is-referenced-by-count":6,"title":["Identification of Long-Term Care Facility Residence From Admission Notes Using Large Language Models"],"prefix":"10.1001","volume":"8","author":[{"given":"Katherine E.","family":"Goodman","sequence":"first","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"},{"name":"The University of Maryland Institute for Health Computing, North Bethesda"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew L.","family":"Robinson","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Seyed M.","family":"Shams","sequence":"additional","affiliation":[{"name":"The University of Maryland Institute for Health Computing, North Bethesda"},{"name":"University of Maryland Medical System, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pilar","family":"Beccar-Varela","sequence":"additional","affiliation":[{"name":"The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Suiyini","family":"Fiawoo","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathan","family":"Kwon","sequence":"additional","affiliation":[{"name":"The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jae Hyoung","family":"Lee","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abigail H.","family":"Vorsteg","sequence":"additional","affiliation":[{"name":"The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Monica","family":"Taneja","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laurence S.","family":"Magder","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Sutherland","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Scott","family":"Sorongon","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pranita D.","family":"Tamma","sequence":"additional","affiliation":[{"name":"Division of Infectious Diseases, The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel J.","family":"Morgan","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip","family":"Resnik","sequence":"additional","affiliation":[{"name":"The University of Maryland, College Park"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anthony D.","family":"Harris","sequence":"additional","affiliation":[{"name":"The University of Maryland School of Medicine, Baltimore"},{"name":"The University of Maryland Institute for Health Computing, North Bethesda"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eili Y.","family":"Klein","sequence":"additional","affiliation":[{"name":"The Johns Hopkins School of Medicine, Baltimore, Maryland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2025,5,22]]},"reference":[{"issue":"4","key":"zoi250403r2","doi-asserted-by":"publisher","DOI":"10.3390\/antibiotics9040196","article-title":"Trends, epidemiology, and management of multi-drug resistant gram-negative bacterial infections in the hospitalized setting.","volume":"9","author":"Morris","year":"2020","journal-title":"Antibiotics (Basel)"},{"issue":"5","key":"zoi250403r4","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1017\/ice.2019.42","article-title":"Predicting probability of perirectal colonization with carbapenem-resistant Enterobacteriaceae (CRE) and other carbapenem-resistant organisms (CROs) at hospital unit admission.","volume":"40","author":"Goodman","year":"2019","journal-title":"Infect Control Hosp Epidemiol"},{"issue":"2","key":"zoi250403r5","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1016\/S1473-3099(20)30477-1","article-title":"Inappropriate empirical antibiotic therapy for bloodstream infections based on discordant in-vitro susceptibilities: a retrospective cohort analysis of prevalence, predictors, and mortality risk in US hospitals.","volume":"21","author":"Kadri","year":"2021","journal-title":"Lancet Infect Dis"},{"issue":"2","key":"zoi250403r6","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.44959","article-title":"Association of exposure to high-risk antibiotics in acute care hospitals with multidrug-resistant organism burden in nursing homes.","volume":"5","author":"Gontjes","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"1","key":"zoi250403r7","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1186\/s13073-016-0353-5","article-title":"Transmission of methicillin-resistant Staphylococcus aureus in long-term care facilities and their related healthcare networks.","volume":"8","author":"Harrison","year":"2016","journal-title":"Genome Med"},{"issue":"12","key":"zoi250403r8","doi-asserted-by":"publisher","first-page":"1937","DOI":"10.1016\/j.jamda.2020.04.007","article-title":"High prevalence of multidrug-resistant organism colonization in 28 nursing homes: an \u201ciceberg effect\u201d.","volume":"21","author":"McKinnell","year":"2020","journal-title":"J Am Med Dir Assoc"},{"issue":"7","key":"zoi250403r9","doi-asserted-by":"publisher","first-page":"889","DOI":"10.1093\/cid\/ciw461","article-title":"Spread of carbapenem-resistant enterobacteriaceae among Illinois healthcare facilities: the role of patient sharing.","volume":"63","author":"Ray","year":"2016","journal-title":"Clin Infect Dis"},{"issue":"7","key":"zoi250403r10","doi-asserted-by":"publisher","first-page":"659","DOI":"10.1016\/j.ajic.2011.08.013","article-title":"Use of the point of origin code from a universal billing form, UB-04, to efficiently identify hospitalized patients admitted from other health care facilities.","volume":"40","author":"Prabaker","year":"2012","journal-title":"Am J Infect Control"},{"issue":"8","key":"zoi250403r12","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1001\/jama.2024.0555","article-title":"AI-generated clinical summaries require more than accuracy.","volume":"331","author":"Goodman","year":"2024","journal-title":"JAMA"},{"issue":"5","key":"zoi250403r13","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000503","article-title":"Addressing 6 challenges in generative AI for digital health: a scoping review.","volume":"3","author":"Templin","year":"2024","journal-title":"PLOS Digit Health"},{"issue":"4","key":"zoi250403r14","doi-asserted-by":"publisher","first-page":"860","DOI":"10.1093\/cid\/ciad633","article-title":"Black box warning: large language models and the future of infectious diseases consultation.","volume":"78","author":"Schwartz","year":"2024","journal-title":"Clin Infect Dis"},{"issue":"1","key":"zoi250403r16","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1177\/001316446002000104","article-title":"A coefficient of agreement for nominal scales.","volume":"20","author":"Cohen","year":"1960","journal-title":"Educ Psychol Meas"},{"issue":"3","key":"zoi250403r17","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1016\/j.sapharm.2012.04.004","article-title":"Interrater agreement and interrater reliability: key concepts, approaches, and applications.","volume":"9","author":"Gisev","year":"2013","journal-title":"Res Social Adm Pharm"},{"issue":"3","key":"zoi250403r18","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic.","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"issue":"1","key":"zoi250403r20","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1038\/s44271-024-00091-8","article-title":"Studying and improving reasoning in humans and machines.","volume":"2","author":"Yax","year":"2024","journal-title":"Commun Psychol"},{"key":"zoi250403r1","doi-asserted-by":"crossref","DOI":"10.15620\/cdc:82532","volume-title":"Antibiotic Resistance Threats in the United States, 2019","author":"Centers for Disease Control and Prevention (U.S.)","year":"2019"},{"key":"zoi250403r3","unstructured":"CDC. Public health strategies to prevent the spread of novel and targeted multidrug-resistant organisms (MDROs). Accessed January 14, 2024. https:\/\/www.cdc.gov\/healthcare-associated-infections\/media\/pdfs\/health-response-prevent-mdro-508.pdf?CDC_AAref_Val=https:\/\/www.cdc.gov\/hai\/pdfs\/mdro-guides\/Health-Response-Prevent-MDRO-508.pdfhttps:\/\/www.cdc.gov\/hai\/pdfs\/mdro-guides\/Health-Response-Prevent-MDRO-508.pdf"},{"key":"zoi250403r11","doi-asserted-by":"crossref","unstructured":"Goodman? KE, Taneja? M, Magder? LS, . A multi-center validation of the electronic health record admission source and discharge location fields against the clinical notes for identifying inpatients with long-term care facility exposure. Infect Control Hosp Epidemiol. Published online April 18, 2024. doi:10.1017\/ice.2024.37","DOI":"10.1017\/ice.2024.37"},{"key":"zoi250403r15","unstructured":"The National Healthcare Safety Network. Long-term care facility component manual. Published January 2023. Accessed April 16, 2025. https:\/\/www.cdc.gov\/nhsn\/pdfs\/ltc\/ltcf-manual-508.pdf"},{"key":"zoi250403r19","doi-asserted-by":"crossref","unstructured":"Wu? Z, Qiu? L, Ross? A, . Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks.? arXiv. Preprint posted online March 28, 2024. doi:10.48550\/arXiv.2307.02477","DOI":"10.18653\/v1\/2024.naacl-long.102"},{"key":"zoi250403r21","unstructured":"Lewis? P, Perez? E, Piktus? A, . Retrieval-augmented generation for knowledge-intensive NLP tasks.? arXiv. Preprint posted April 12, 2021. doi:10.48550\/arXiv.2005.11401"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2834360\/goodman_2025_oi_250403_1747250674.25899.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,22]],"date-time":"2025-05-22T15:31:16Z","timestamp":1747927876000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2834360"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,22]]},"references-count":21,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,5,1]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2025.12032","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,22]]}}}