{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T01:07:00Z","timestamp":1773796020298,"version":"3.50.1"},"reference-count":31,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2023,3,21]],"date-time":"2023-03-21T00:00:00Z","timestamp":1679356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["R21MH130853-01"],"award-info":[{"award-number":["R21MH130853-01"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,7,19]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Social determinants of health (SDOH) are nonclinical, socioeconomic conditions that influence patient health and quality of life. Identifying SDOH may help clinicians target interventions. However, SDOH are more frequently available in narrative notes compared to structured electronic health records. The 2022 n2c2 Track 2 competition released clinical notes annotated for SDOH to promote development of NLP systems for extracting SDOH. We developed a system addressing 3 limitations in state-of-the-art SDOH extraction: the inability to identify multiple SDOH events of the same type per sentence, overlapping SDOH attributes within text spans, and SDOH spanning multiple sentences.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We developed and evaluated a 2-stage architecture. In stage 1, we trained a BioClinical-BERT-based named entity recognition system to extract SDOH event triggers, that is, text spans indicating substance use, employment, or living status. In stage 2, we trained a multitask, multilabel NER to extract arguments (eg, alcohol \u201ctype\u201d) for events extracted in stage 1. Evaluation was performed across 3 subtasks differing by provenance of training and validation data using precision, recall, and F1 scores.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>When trained and validated on data from the same site, we achieved 0.87 precision, 0.89 recall, and 0.88 F1. Across all subtasks, we ranked between second and fourth place in the competition and always within 0.02 F1 from first.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Our 2-stage, deep-learning-based NLP system effectively extracted SDOH events from clinical notes. This was achieved with a novel classification framework that leveraged simpler architectures compared to state-of-the-art systems. Improved SDOH extraction may help clinicians improve health outcomes.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocad046","type":"journal-article","created":{"date-parts":[[2023,4,1]],"date-time":"2023-04-01T21:21:10Z","timestamp":1680384070000},"page":"1379-1388","source":"Crossref","is-referenced-by-count":22,"title":["Extracting social determinants of health events with transformer-based multitask, multilabel named entity recognition"],"prefix":"10.1093","volume":"30","author":[{"given":"Russell","family":"Richie","sequence":"first","affiliation":[{"name":"Tsui Laboratory, Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, Pennsylvania, USA"},{"name":"MindCORE and Cognitive Science, University of Pennsylvania , Philadelphia, Pennsylvania, USA"}]},{"given":"Victor M","family":"Ruiz","sequence":"additional","affiliation":[{"name":"Tsui Laboratory, Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3281-5955","authenticated-orcid":false,"given":"Sifei","family":"Han","sequence":"additional","affiliation":[{"name":"Tsui Laboratory, Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, Pennsylvania, USA"}]},{"given":"Lingyun","family":"Shi","sequence":"additional","affiliation":[{"name":"Tsui Laboratory, Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6383-8471","authenticated-orcid":false,"given":"Fuchiang (Rich)","family":"Tsui","sequence":"additional","affiliation":[{"name":"Tsui Laboratory, Department of Biomedical and Health Informatics, Children\u2019s Hospital of Philadelphia , Philadelphia, Pennsylvania, USA"},{"name":"Department of Anesthesiology and Critical Care, University of Pennsylvania Perelman School of Medicine , Philadelphia, Pennsylvania, USA"}]}],"member":"286","published-online":{"date-parts":[[2023,3,21]]},"reference":[{"issue":"10","key":"2023071909391614100_ocad046-B1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.31478\/201710c","article-title":"Social determinants of health 101 for health care: five plus five","volume":"7","author":"Magnan","year":"2017","journal-title":"NAM Perspect"},{"issue":"25","key":"2023071909391614100_ocad046-B2","first-page":"625","article-title":"Annual smoking-attributable mortality, years of potential life lost, and productivity losses \u2013 United States, 1997\u20132001","volume":"54","author":"Centers for Disease Control and Prevention (CDC)","year":"2005","journal-title":"MMWR Morb Mortal Wkly Rep"},{"key":"2023071909391614100_ocad046-B3"},{"issue":"9810","key":"2023071909391614100_ocad046-B4","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/S0140-6736(11)61138-0","article-title":"Extent of illicit drug use and dependence, and their contribution to the global burden of disease","volume":"379","author":"Degenhardt","year":"2012","journal-title":"Lancet"},{"issue":"2","key":"2023071909391614100_ocad046-B5","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1111\/1475-6773.12670","article-title":"Hospital readmission and social risk factors identified from physician notes","volume":"53","author":"Navathe","year":"2018","journal-title":"Health Serv Res"},{"key":"2023071909391614100_ocad046-B6","doi-asserted-by":"crossref","first-page":"103429","DOI":"10.1016\/j.jbi.2020.103429","article-title":"Maximizing the use of social and behavioural information from secondary care mental health electronic health records","volume":"107","author":"Goodday","year":"2020","journal-title":"J Biomed Inform"},{"issue":"12","key":"2023071909391614100_ocad046-B7","doi-asserted-by":"crossref","first-page":"2716","DOI":"10.1093\/jamia\/ocab170","article-title":"Extracting social determinants of health from electronic health records using natural language processing: a systematic review","volume":"28","author":"Patra","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2023071909391614100_ocad046-B8","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1186\/s13326-019-0198-0","article-title":"Moonstone: a novel natural language processing system for inferring social risk from clinical narratives","volume":"10","author":"Conway","year":"2019","journal-title":"J Biomed Semantics"},{"issue":"1","key":"2023071909391614100_ocad046-B9","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1177\/1460458218824742","article-title":"Natural language processing of lifestyle modification documentation","volume":"26","author":"Shoenbill","year":"2020","journal-title":"Health Informatics J"},{"issue":"1","key":"2023071909391614100_ocad046-B10","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1055\/s-0040-1702214","article-title":"Detecting social and behavioral determinants of health with structured and free-text clinical data","volume":"11","author":"Feller","year":"2020","journal-title":"Appl Clin Inform"},{"key":"2023071909391614100_ocad046-B11","article-title":"Automatic extraction of social determinants of health from medical notes of chronic lower back pain patients","author":"Lituiev","journal-title":"medRxiv"},{"key":"2023071909391614100_ocad046-B12","doi-asserted-by":"crossref","first-page":"103984","DOI":"10.1016\/j.jbi.2021.103984","article-title":"Classifying social determinants of health from unstructured electronic health records using deep learning-based natural language processing","volume":"127","author":"Han","year":"2022","journal-title":"J Biomed Inform"},{"issue":"3","key":"2023071909391614100_ocad046-B13","doi-asserted-by":"crossref","first-page":"ooaa069","DOI":"10.1093\/jamiaopen\/ooaa069","article-title":"Identification of social determinants of health using multi-label classification of electronic health record clinical notes","volume":"4","author":"Stemerman","year":"2021","journal-title":"JAMIA Open"},{"key":"2023071909391614100_ocad046-B14","doi-asserted-by":"publisher","first-page":"ocad012","DOI":"10.1093\/jamia\/ocad012","article-title":"The 2022 n2c2\/UW shared task on extracting social determinants of health","author":"Lybarger","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2023071909391614100_ocad046-B15","doi-asserted-by":"crossref","first-page":"103631","DOI":"10.1016\/j.jbi.2020.103631","article-title":"Annotating social determinants of health using active learning, and characterizing determinants using neural event extraction","volume":"113","author":"Lybarger","year":"2021","journal-title":"J Biomed Inform"},{"key":"2023071909391614100_ocad046-B16"},{"key":"2023071909391614100_ocad046-B17","doi-asserted-by":"publisher","author":"Alsentzer","year":"2019","DOI":"10.48550\/arXiv.1904.03323"},{"key":"2023071909391614100_ocad046-B18","doi-asserted-by":"publisher","author":"Devlin","year":"2019","DOI":"10.48550\/arXiv.1810.04805"},{"key":"2023071909391614100_ocad046-B19","doi-asserted-by":"publisher","author":"Wolf","year":"2020","DOI":"10.48550\/arXiv.1910.03771"},{"key":"2023071909391614100_ocad046-B20","first-page":"1","article-title":"Desmaison A. Pytorch: An imperative style, high-performance deep learning library","volume":"32","author":"Paszke","year":"2019","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023071909391614100_ocad046-B21","doi-asserted-by":"publisher","author":"Honnibal","year":"2020","DOI":"10.5281\/zenodo.1212303"},{"key":"2023071909391614100_ocad046-B22","doi-asserted-by":"publisher","first-page":"5284","DOI":"10.18653\/v1\/P19-1522","author":"Yang","year":"2019"},{"key":"2023071909391614100_ocad046-B23","author":"Lybarger","year":"2022"},{"key":"2023071909391614100_ocad046-B24","doi-asserted-by":"publisher","author":"Li","year":"2022","DOI":"10.48550\/arXiv.2107.02126"},{"key":"2023071909391614100_ocad046-B25","volume-title":"State Innovation Models (SIM) Round 2: Model Test Annual Report Two","author":"Coughlin","year":"2018"},{"issue":"14","key":"2023071909391614100_ocad046-B26","doi-asserted-by":"crossref","first-page":"1416","DOI":"10.1001\/jama.2021.12825","article-title":"Screening and interventions for social risk factors: technical brief to support the US Preventive Services Task Force","volume":"326","author":"Eder","year":"2021","journal-title":"JAMA"},{"issue":"3","key":"2023071909391614100_ocad046-B27","doi-asserted-by":"crossref","first-page":"130","DOI":"10.6000\/1929-4247.2014.03.03.3","article-title":"The effectiveness of food insecurity screening in pediatric primary care","volume":"3","author":"Lane","year":"2014","journal-title":"Int J Child Health Nutr"},{"key":"2023071909391614100_ocad046-B28","doi-asserted-by":"crossref","first-page":"18","DOI":"10.7812\/TPP\/18-093","article-title":"Lessons learned from implementation of the food insecurity screening and referral program at Kaiser Permanente Colorado","volume":"22","author":"Stenmark","year":"2018","journal-title":"Perm J"},{"issue":"2","key":"2023071909391614100_ocad046-B29","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1097\/QAI.0000000000001580","article-title":"Using clinical notes and natural language processing for automated HIV risk assessment","volume":"77","author":"Feller","year":"2018","journal-title":"J Acquir Immune Defic Syndr"},{"key":"2023071909391614100_ocad046-B30","doi-asserted-by":"publisher","author":"Liu","year":"2019","DOI":"10.48550\/arXiv.1907.11692"},{"key":"2023071909391614100_ocad046-B31","doi-asserted-by":"publisher","author":"Li","year":"2022","DOI":"10.48550\/arXiv.2201.11838"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/8\/1379\/50908626\/ocad046.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/30\/8\/1379\/50908626\/ocad046.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T10:08:33Z","timestamp":1689761313000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/30\/8\/1379\/7099518"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,21]]},"references-count":31,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,3,21]]},"published-print":{"date-parts":[[2023,7,19]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocad046","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,8,1]]},"published":{"date-parts":[[2023,3,21]]}}}