{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T19:15:13Z","timestamp":1775157313182,"version":"3.50.1"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2024,12,26]],"date-time":"2024-12-26T00:00:00Z","timestamp":1735171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["T32DA0505552"],"award-info":[{"award-number":["T32DA0505552"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA057599"],"award-info":[{"award-number":["R01DA057599"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000026","name":"NIDA","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>To detect and classify features of stigmatizing and biased language in intensive care electronic health records (EHRs) using natural language processing techniques.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We first created a lexicon and regular expression lists from literature-driven stem words for linguistic features of stigmatizing patient labels, doubt markers, and scare quotes within EHRs. The lexicon was further extended using Word2Vec and GPT 3.5, and refined through human evaluation. These lexicons were used to search for matches across 18 million sentences from the de-identified Medical Information Mart for Intensive Care-III (MIMIC-III) dataset. For each linguistic bias feature, 1000 sentence matches were sampled, labeled by expert clinical and public health annotators, and used to supervised learning classifiers.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Lexicon development from expanded literature stem-word lists resulted in a doubt marker lexicon containing 58 expressions, and a stigmatizing labels lexicon containing 127 expressions. Classifiers for doubt markers and stigmatizing labels had the highest performance, with macro F1-scores of 0.84 and 0.79, positive-label recall and precision values ranging from 0.71 to 0.86, and accuracies aligning closely with human annotator agreement (0.87).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>This study demonstrated the feasibility of supervised classifiers in automatically identifying stigmatizing labels and doubt markers in medical text and identified trends in stigmatizing language use in an EHR setting. Additional labeled data may help improve lower scare quote model performance.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>Classifiers developed in this study showed high model performance and can be applied to identify patterns and target interventions to reduce stigmatizing labels and doubt markers in healthcare systems.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocae310","type":"journal-article","created":{"date-parts":[[2024,12,26]],"date-time":"2024-12-26T23:42:22Z","timestamp":1735256542000},"page":"365-374","source":"Crossref","is-referenced-by-count":11,"title":["CARE-SD: classifier-based analysis for recognizing provider stigmatizing and doubt marker labels in electronic health records: model development and validation"],"prefix":"10.1093","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4216-2396","authenticated-orcid":false,"given":"Andrew","family":"Walker","sequence":"first","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, Emory University , Atlanta, GA 30322,","place":["United States"]}]},{"given":"Annie","family":"Thorne","sequence":"additional","affiliation":[{"name":"Department of Infectious Disease, Children\u2019s Healthcare of Atlanta , Atlanta, GA 30329,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2112-6986","authenticated-orcid":false,"given":"Sudeshna","family":"Das","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, Emory University , Atlanta, GA 30322,","place":["United States"]}]},{"given":"Jennifer","family":"Love","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Mount Sinai , New York, NY 10029,","place":["United States"]}]},{"given":"Hannah L F","family":"Cooper","sequence":"additional","affiliation":[{"name":"Department of Behavioral, Social, Health Education Sciences, Rollins School of Public Health, Emory University , Atlanta, GA 30322,","place":["United States"]}]},{"suffix":"III","given":"Melvin","family":"Livingston","sequence":"additional","affiliation":[{"name":"Department of Behavioral, Social, Health Education Sciences, Rollins School of Public Health, Emory University , Atlanta, GA 30322,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7358-544X","authenticated-orcid":false,"given":"Abeed","family":"Sarker","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, School of Medicine, Emory University , Atlanta, GA 30322,","place":["United States"]},{"name":"Department of Biomedical Engineering, Georgia Institute of Technology and Emory University , Atlanta, GA 30332,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2024,12,26]]},"reference":[{"key":"2025012119043988400_ocae310-B1","doi-asserted-by":"publisher","first-page":"813","DOI":"10.2105\/AJPH.2012.301069","article-title":"Stigma as a fundamental cause of population health inequalities","volume":"103","author":"Hatzenbuehler","year":"2013","journal-title":"Am J Public Health"},{"key":"2025012119043988400_ocae310-B2","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1016\/j.socscimed.2017.05.009","article-title":"A decade of studying implicit racial\/ethnic bias in healthcare providers using the implicit association test","volume":"199","author":"Maina","year":"2018","journal-title":"Soc Sci Med"},{"key":"2025012119043988400_ocae310-B3","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/s12910-017-0179-8","article-title":"Implicit bias in healthcare professionals: a systematic review","volume":"18","author":"FitzGerald","year":"2017","journal-title":"BMC Med Ethics"},{"key":"2025012119043988400_ocae310-B4","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1146\/annurev.soc.27.1.363","article-title":"Conceptualizing stigma","volume":"27","author":"Link","year":"2001","journal-title":"Annu Rev Sociol"},{"key":"2025012119043988400_ocae310-B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.12840\/issn.2255-4165.017","article-title":"How stereotypes are shared through language\u2014a review and introduction of the social categories and stereotypes communication (SCSC) framework","volume":"7","author":"Beukeboom","year":"2019","journal-title":"Rev Commun Res"},{"key":"2025012119043988400_ocae310-B6","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.7326\/M21-2449","article-title":"Quoting patients in clinical notes: first, do no harm","volume":"174","author":"Beach","year":"2021","journal-title":"Ann Intern Med"},{"key":"2025012119043988400_ocae310-B7","doi-asserted-by":"publisher","first-page":"685","DOI":"10.1007\/s11606-017-4289-2","article-title":"Do words matter? Stigmatizing language and the transmission of bias in the medical record","volume":"33","author":"Goddu","year":"2018","journal-title":"J Gen Intern Med"},{"key":"2025012119043988400_ocae310-B8","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1145\/3368555.3384448","author":"Zhang","year":"2020"},{"key":"2025012119043988400_ocae310-B9","author":"Abuse NI on D","year":"2021"},{"key":"2025012119043988400_ocae310-B10","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1002\/ajh.23441","article-title":"Among emergency physicians, use of the term \u201cSickler\u201d is associated with negative attitudes toward people with sickle cell disease","volume":"88","author":"Glassberg","year":"2013","journal-title":"Am J Hematol"},{"key":"2025012119043988400_ocae310-B11","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1377\/hlthaff.2021.01423","article-title":"Negative patient descriptors: documenting racial bias in the electronic health record","volume":"41","author":"Sun","year":"2022","journal-title":"Health Aff (Millwood)"},{"key":"2025012119043988400_ocae310-B12","doi-asserted-by":"publisher","DOI":"10.1093\/acrefore\/9780190228613.013.439","volume-title":"Oxford Research Encyclopedia of Communication","author":"Beukeboom","year":"2017"},{"key":"2025012119043988400_ocae310-B13","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1162\/COLI_a_00278","article-title":"Evaluative language beyond bags of words: linguistic insights and computational applications","volume":"43","author":"Benamara","year":"2017","journal-title":"Comput Linguist"},{"key":"2025012119043988400_ocae310-B14","doi-asserted-by":"publisher","first-page":"1708","DOI":"10.1007\/s11606-021-06682-z","article-title":"Testimonial injustice: linguistic bias in the medical records of Black patients and women","volume":"36","author":"Beach","year":"2021","journal-title":"J Gen Intern Med"},{"key":"2025012119043988400_ocae310-B15","doi-asserted-by":"publisher","DOI":"10.1037\/sah0000497","article-title":"A scoping review of methodological approaches to detect bias in the electronic health record [published online ahead of print]","author":"Kelly","year":"2023","journal-title":"Stigma Health"},{"key":"2025012119043988400_ocae310-B16","doi-asserted-by":"publisher","first-page":"e2144967","DOI":"10.1001\/jamanetworkopen.2021.44967","article-title":"Examination of stigmatizing language in the electronic health record","volume":"5","author":"Himmelstein","year":"2022","journal-title":"JAMA Netw Open"},{"key":"2025012119043988400_ocae310-B17","doi-asserted-by":"publisher","first-page":"312","DOI":"10.18653\/v1\/2023.acl-short.28","author":"Harrigian","year":"2023"},{"key":"2025012119043988400_ocae310-B18","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1007\/s10995-023-03857-4","article-title":"Using natural language processing to identify stigmatizing language in labor and birth clinical notes","volume":"28","author":"Barcelona","year":"2024","journal-title":"Matern Child Health J"},{"key":"2025012119043988400_ocae310-B19","author":"Walker","year":"2024"},{"key":"2025012119043988400_ocae310-B20","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"2025012119043988400_ocae310-B21","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1038\/s41597-019-0055-0","article-title":"BioWordVec, improving biomedical word embeddings with subword information and MeSH","volume":"6","author":"Zhang","year":"2019","journal-title":"Sci Data"},{"key":"2025012119043988400_ocae310-B22","year":"2022"},{"key":"2025012119043988400_ocae310-B23","year":"2023"},{"key":"2025012119043988400_ocae310-B24","doi-asserted-by":"publisher","first-page":"e2117052","DOI":"10.1001\/jamanetworkopen.2021.17052","article-title":"Physician use of stigmatizing language in patient medical records","volume":"4","author":"Park","year":"2021","journal-title":"JAMA Netw Open"},{"key":"2025012119043988400_ocae310-B25","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1007\/s40615-020-00776-w","article-title":"Health care providers\u2019 negative implicit attitudes and stereotypes of American Indians","volume":"8","author":"Zestcott","year":"2021","journal-title":"J Racial Ethn Health Disparities"},{"key":"2025012119043988400_ocae310-B26","author":"Liu","year":"2019"},{"key":"2025012119043988400_ocae310-B27","year":"2024"},{"key":"2025012119043988400_ocae310-B28","doi-asserted-by":"publisher","author":"Dettmers","year":"2023","DOI":"10.48550\/arXiv.2305.14314"},{"key":"2025012119043988400_ocae310-B29","doi-asserted-by":"publisher","first-page":"3711","DOI":"10.1093\/bioinformatics\/bty373","article-title":"The revival of the Gini importance?","volume":"34","author":"Nembrini","year":"2018","journal-title":"Bioinformatics"},{"key":"2025012119043988400_ocae310-B30","year":"2024"},{"key":"2025012119043988400_ocae310-B31","year":"2024"},{"key":"2025012119043988400_ocae310-B32","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1176\/appi.ps.201200289","article-title":"Service use and barriers to mental health care among adults with major depression and comorbid substance dependence","volume":"64","author":"Chen","year":"2013","journal-title":"Psychiatr Serv"},{"key":"2025012119043988400_ocae310-B33","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1186\/s12875-015-0353-3","article-title":"Barriers and facilitators to primary care for people with mental health and\/or substance use issues: a qualitative study","volume":"16","author":"Ross","year":"2015","journal-title":"BMC Fam Pract"},{"key":"2025012119043988400_ocae310-B34","doi-asserted-by":"publisher","first-page":"1141093","DOI":"10.3389\/fpubh.2023.1141093","article-title":"Barriers to opioid use disorder treatment: a comparison of self-reported information from social media with barriers found in literature","volume":"11","author":"Bremer","year":"2023","journal-title":"Front Public Health"},{"key":"2025012119043988400_ocae310-B35","doi-asserted-by":"publisher","first-page":"2381","DOI":"10.1001\/jama.2020.7858","article-title":"Digital health equity as a necessity in the 21st century cures act era","volume":"323","author":"Rodriguez","year":"2020","journal-title":"JAMA"},{"key":"2025012119043988400_ocae310-B36","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1111\/sdi.12934","article-title":"Healthcare in the new age of transparency","volume":"33","author":"DesRoches","year":"2020","journal-title":"Semin Dial"},{"key":"2025012119043988400_ocae310-B37","doi-asserted-by":"publisher","first-page":"462","DOI":"10.1111\/j.1468-2885.2007.00307.x","article-title":"Language of the lost: an explication of stigma communication","volume":"17","author":"Smith","year":"2007","journal-title":"Commun Theory"},{"key":"2025012119043988400_ocae310-B38","doi-asserted-by":"publisher","first-page":"528","DOI":"10.1016\/S0140-6736(06)68184-1","article-title":"Stigma and its public health implications","volume":"367","author":"Link","year":"2006","journal-title":"Lancet"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/2\/365\/61281694\/ocae310.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/2\/365\/61281694\/ocae310.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T19:04:50Z","timestamp":1737486290000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/2\/365\/7933303"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,26]]},"references-count":38,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,12,26]]},"published-print":{"date-parts":[[2025,2,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae310","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,2]]},"published":{"date-parts":[[2024,12,26]]}}}