{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T15:23:00Z","timestamp":1786116180390,"version":"build-2736575974"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"4","funder":[{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["R01MD016037"],"award-info":[{"award-number":["R01MD016037"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006545","name":"National Institute on Minority Health and Health Disparities","doi-asserted-by":"publisher","award":["R01MD015716"],"award-info":[{"award-number":["R01MD015716"]}],"id":[{"id":"10.13039\/100006545","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Intramural Research Program of the National Institutes of Health","award":["ZIA NR000043"],"award-info":[{"award-number":["ZIA NR000043"]}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000016","name":"U.S. Department of Health and Human Services","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000016","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,7,7]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Social determinants of health (SDOH) are incompletely captured in structured electronic health records (EHRs) but are frequently documented in unstructured clinical notes. We evaluated large language models (LLMs) for extracting SDOH from clinical text.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We constructed an adult sepsis cohort from the Medical Information Mart for Intensive Care-IV (Sequential Organ Failure Assessment \u22652) and analyzed clinical notes from 1 year prior to 30 days following suspected infection. Three instruction\u2011tuned, decoder\u2011only LLMs (Mistral\u2011Instruct\u20117B-v0.2, DeepSeek\u2011R1\u2011Distill\u2011Qwen\u201114B, and GPT\u2011oss\u201120B) were evaluated using structured prompts with predefined label schemas and few\u2011shot examples. Performance was benchmarked against a clinically validated annotated dataset and compared with a fine\u2011tuned encoder\u2011decoder baseline. Macro\u2011F1 scores were reported. A gold\u2011standard Intensive Care Unit (ICU) sepsis subset was independently annotated by 3 reviewers to assess domain\u2011level performance and ensemble strategies.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Decoder\u2011only models outperformed the fine\u2011tuned encoder\u2011decoder baseline across SDOH domains. GPT\u2011oss achieved the highest macro\u2011F1 score (0.79) compared with Flan\u2011T5\u2011XXL (0.57). Prompt refinement substantially improved extraction accuracy. Ensemble majority voting increased robustness across domains, while unanimous agreement yielded high precision but limited coverage. In a subsequent mortality analysis, extracted SDOH did not independently predict 30-day mortality, which was instead associated with established clinical and demographic risk factors.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>Instruction\u2011tuned decoder\u2011only LLMs can reliably extract multiclass SDOH from unstructured clinical notes without task\u2011specific fine\u2011tuning. Ensemble and agreement\u2011based strategies provide practical operating points for high\u2011precision clinical deployment.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>These findings support the feasibility of leveraging LLMs to enrich EHRs with structured SDOH data, providing a scalable approach for incorporating social context into downstream risk stratification and health outcome prediction.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooag135","type":"journal-article","created":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T16:30:43Z","timestamp":1784133043000},"source":"Crossref","is-referenced-by-count":0,"title":["Scalable extraction of social determinants of health from clinical notes in a sepsis cohort using instruction-tuned language models"],"prefix":"10.1093","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6013-0008","authenticated-orcid":false,"given":"Diego","family":"Salazar","sequence":"first","affiliation":[{"name":"National Institutes of Health National Institute of Nursing Research, , 9000 Rockville Pike , Bethesda, MD 20814,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9002-4679","authenticated-orcid":false,"given":"Pankaj","family":"Dipankar","sequence":"additional","affiliation":[{"name":"National Institutes of Health National Institute of Nursing Research, , 9000 Rockville Pike , Bethesda, MD 20814,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3860-2181","authenticated-orcid":false,"given":"Daniel","family":"Smolyak","sequence":"additional","affiliation":[{"name":"National Institutes of Health National Institute of Nursing Research, , 9000 Rockville Pike , Bethesda, MD 20814,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4745-6681","authenticated-orcid":false,"given":"Quynh C","family":"Nguyen","sequence":"additional","affiliation":[{"name":"National Institutes of Health National Institute of Nursing Research, , 9000 Rockville Pike , Bethesda, MD 20814,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,7,15]]},"reference":[{"key":"2026080604264146000_ooag135-B1","first-page":"328","article-title":"Social determinants of health screening and management: lessons at a large, urban academic health system","volume":"49","author":"Peretz","year":"2023","journal-title":"Jt Comm J Qual Patient Saf."},{"key":"2026080604264146000_ooag135-B2","doi-asserted-by":"publisher","first-page":"67","author":"Consoli","DOI":"10.1093\/jamia\/ocaf094"},{"key":"2026080604264146000_ooag135-B3","first-page":"e1099","article-title":"Prediction of readmission following sepsis using social determinants of health","volume":"6","author":"Amrollahi","year":"2024","journal-title":"Crit Care Explor."},{"key":"2026080604264146000_ooag135-B4","first-page":"e0731","article-title":"Social determinants of health associated with the development of sepsis in adults: a scoping review","volume":"4","author":"Sheikh","year":"2022","journal-title":"Crit Care Explor."},{"key":"2026080604264146000_ooag135-B5","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1186\/s13054-023-04565-9","article-title":"Neighborhood socioeconomic deprivation, healthcare access, and 30-day mortality and readmission after sepsis or critical illness: findings from a nationwide study","volume":"27","author":"Lusk","year":"2023","journal-title":"Crit Care."},{"key":"2026080604264146000_ooag135-B6","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1002\/wmh3.70043","article-title":"Social determinants of sepsis mortality in the United States: a retrospective, epidemiologic analysis","volume":"17","author":"Ardabili","year":"2025","journal-title":"World Med & Health Policy."},{"key":"2026080604264146000_ooag135-B7","doi-asserted-by":"crossref","first-page":"342","DOI":"10.1186\/s12913-025-12494-2","article-title":"Impact of hospital and health system initiatives to address social determinants of health (SDOH) in the United States: a scoping review of the peer-reviewed literature","volume":"25","author":"Rangachari","year":"2025","journal-title":"BMC Health Serv Res."},{"key":"2026080604264146000_ooag135-B8","first-page":"391","volume-title":"Machine Learning for Healthcare Conference","author":"Ahsan","year":"2021"},{"key":"2026080604264146000_ooag135-B9","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1007\/s12630-024-02790-6","article-title":"Social determinants of health and sepsis: a case-control study","volume":"71","author":"Sheikh","year":"2024","journal-title":"Can J Anaesth"},{"key":"2026080604264146000_ooag135-B10","doi-asserted-by":"crossref","first-page":"2074","DOI":"10.3390\/app14052074","article-title":"A review of current trends, techniques, and challenges in large language models (LLMs)","volume":"14","author":"Patil","year":"2024","journal-title":"Appl Sci"},{"key":"2026080604264146000_ooag135-B11","doi-asserted-by":"publisher","author":"Liu","DOI":"10.48550\/arxiv.240603712"},{"key":"2026080604264146000_ooag135-B12","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1038\/s41746-023-00970-0","article-title":"Large language models to identify social determinants of health in electronic health records","volume":"7","author":"Guevara","year":"2024","journal-title":"NPJ Digit Med."},{"key":"2026080604264146000_ooag135-B13","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1038\/s41746-025-02025-y","article-title":"A large language model-based approach to quantifying the effects of social determinants in liver transplant decisions","volume":"8","author":"Robitschek","year":"2025","journal-title":"NPJ Digit Med."},{"key":"2026080604264146000_ooag135-B14","doi-asserted-by":"crossref","first-page":"1379","DOI":"10.1093\/jamia\/ocad046","article-title":"Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition","volume":"30","author":"Richie","year":"2023","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604264146000_ooag135-B15","author":"Jiang"},{"key":"2026080604264146000_ooag135-B16","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1038\/s41597-023-02136-9","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci Data."},{"key":"2026080604264146000_ooag135-B17","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s13054-022-04290-9","article-title":"The sequential organ failure assessment (SOFA) score: has the time come for an update?","volume":"27","author":"Moreno","year":"2023","journal-title":"Crit Care."},{"key":"2026080604264146000_ooag135-B18","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1001\/jama.2016.0288","article-title":"Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (sepsis-3)","volume":"315","author":"Seymour","year":"2016","journal-title":"JAMA."},{"key":"2026080604264146000_ooag135-B19","author":"Boonstra","year":"2025"},{"key":"2026080604264146000_ooag135-B20","doi-asserted-by":"publisher","first-page":"e83793","DOI":"10.2196\/83793","article-title":"Applications of natural language processing and large language models for social determinants of health: systematic review","volume":"28","author":"Rajwal","year":"2026","journal-title":"J Med Internet Res"},{"key":"2026080604264146000_ooag135-B21","doi-asserted-by":"publisher","author":"Gu Z, Chen X, Shi X, et al. GAPO: learning preferential prompt through generative adversarial policy optimization. In: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Vienna, Austria: Association for Computational Linguistics; 2025;282-296.","DOI":"10.18653\/v1\/2025.acl-long.13"},{"key":"2026080604264146000_ooag135-B22","author":"Shazeer","year":"2017"},{"key":"2026080604264146000_ooag135-B23","author":"Agarwal","year":"2025"},{"key":"2026080604264146000_ooag135-B24","doi-asserted-by":"publisher","first-page":"633","DOI":"10.1038\/s41586-025-09422-z","article-title":"DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning","volume":"645","author":"Guo","year":"2025","journal-title":"Nature."},{"key":"2026080604264146000_ooag135-B25","doi-asserted-by":"publisher","first-page":"100190","DOI":"10.1016\/j.metrad.2025.100190","article-title":"Prompt engineering for healthcare: methodologies and applications","volume":"4","author":"Wang","year":"2026","journal-title":"Meta Radiol."},{"key":"2026080604264146000_ooag135-B26","doi-asserted-by":"crossref","first-page":"e60501","DOI":"10.2196\/60501","article-title":"Prompt engineering paradigms for medical applications: scoping review","volume":"26","author":"Zaghir","year":"2024","journal-title":"J Med Internet Res."},{"key":"2026080604264146000_ooag135-B27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3729219","article-title":"Privacy preserving prompt engineering: a survey","volume":"57","author":"Edemacu","year":"2025","journal-title":"ACM Comput Surv."},{"key":"2026080604264146000_ooag135-B28","doi-asserted-by":"crossref","first-page":"986","DOI":"10.1093\/jamia\/ocx039","article-title":"Challenges in adapting existing clinical natural language processing systems to multiple, diverse health care settings","volume":"24","author":"Carrell","year":"2017","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604264146000_ooag135-B29","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1038\/s41746-025-01686-z","article-title":"Iterative refinement and goal articulation to optimize large language models for clinical information extraction","volume":"8","author":"Hein","year":"2025","journal-title":"NPJ Digit Med."},{"key":"2026080604264146000_ooag135-B30","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1023\/A:1018054314350","article-title":"Bagging predictors","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach Learn."},{"key":"2026080604264146000_ooag135-B31","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn."},{"key":"2026080604264146000_ooag135-B32","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw"},{"key":"2026080604264146000_ooag135-B33","first-page":"1","volume-title":"International Workshop on Multiple Classifier Systems","author":"Dietterich","year":"2000"},{"key":"2026080604264146000_ooag135-B34","doi-asserted-by":"publisher","first-page":"ooad085","DOI":"10.1093\/jamiaopen\/ooad085","article-title":"Application of natural language processing to identify social needs from patient medical notes: development and assessment of a scalable, performant, and rule-based model in an integrated healthcare delivery system","volume":"6","author":"Gray","year":"2023","journal-title":"JAMIA Open."},{"key":"2026080604264146000_ooag135-B35","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1093\/jamia\/ocab199","article-title":"The quality of social determinants data in the electronic health record: a systematic review","volume":"29","author":"Cook","year":"2021","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604264146000_ooag135-B36","doi-asserted-by":"publisher","first-page":"2716","DOI":"10.1093\/jamia\/ocab170","article-title":"Extracting social determinants of health from electronic health records using natural language processing: a systematic review","volume":"28","author":"Patra","year":"2021","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604264146000_ooag135-B37","doi-asserted-by":"publisher","first-page":"2608","DOI":"10.1093\/jamia\/ocab194","article-title":"Documentation and review of social determinants of health data in the EHR: measures and associated insights","volume":"28","author":"Wang","year":"2021","journal-title":"J Am Med Inform Assoc."},{"key":"2026080604264146000_ooag135-B38","doi-asserted-by":"publisher","first-page":"ooaa069","DOI":"10.1093\/jamiaopen\/ooaa069","article-title":"Identification of social determinants of health using multi-label classification of electronic health record clinical notes","volume":"4","author":"Stemerman","year":"2021","journal-title":"JAMIA Open."}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag135\/68906384\/ooag135.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/9\/4\/ooag135\/68906384\/ooag135.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T08:26:53Z","timestamp":1786004813000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooag135\/8734988"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,7]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,7,7]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooag135","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,8]]},"published":{"date-parts":[[2026,7,7]]},"article-number":"ooag135"}}