{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T21:59:52Z","timestamp":1790114392238,"version":"4.0.1"},"reference-count":48,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Natural language processing can extract meaningful information from clinical notes. However, human annotation is time-consuming and costly, and scarce data poses a challenge.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to explore a semantic similarity search approach to extract exemplars of stigmatizing and positive language in obstetric clinical notes.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We used electronic health record data from labor and birth admissions at 2 hospitals in the United States from 2017 to 2019. We used a semantic similarity search approach, which used 200 randomly selected true exemplars, stratified by language categories, as queries to search for similar exemplar candidates. We extracted the top 5 candidates with the highest cosine similarities, which were assessed for accuracy.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n We retrieved 1000 candidates. An average precision of 0.69 was achieved when candidates with cosine similarity thresholds of 0.75 or higher were included, at which point 68.8% (64\/93) of exemplar candidates accurately represented true cases. At the 0.75 threshold, the proportion of true cases was higher for preferred language (41\/56, 73.2%) and unilateral\/authoritarian decisions (5\/7, 71.4%). The proportion of true cases was lower for difficult patients (2\/7, 28.6%) and marginalized identities (3\/9, 33.3%).<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The semantic similarity search approach shows promise in efficiently extracting exemplars while reducing the annotation burden, laying the groundwork for future applications in other domains.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/85088","type":"journal-article","created":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T21:01:41Z","timestamp":1790110901000},"page":"e85088-e85088","source":"Crossref","is-referenced-by-count":0,"title":["Semantic Similarity Search Approach to Extract Exemplars of Stigmatizing and Positive Language in Obstetric Clinical Notes: Exploratory Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0864-7867","authenticated-orcid":false,"given":"Jihye Kim","family":"Scroggins","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5504-4646","authenticated-orcid":false,"given":"Ismael Ibrahim","family":"Hulchafo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3070-1716","authenticated-orcid":false,"given":"Veronica","family":"Barcelona","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4345-3889","authenticated-orcid":false,"given":"Anahita","family":"Davoudi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1418-7892","authenticated-orcid":false,"given":"Hans","family":"Moen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2528-7186","authenticated-orcid":false,"given":"Sarah","family":"Harkins","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3265-6667","authenticated-orcid":false,"given":"Danielle","family":"Scharp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0704-3826","authenticated-orcid":false,"given":"Kenrick","family":"Cato","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5351-095X","authenticated-orcid":false,"given":"Michele","family":"Tadiello","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2358-9837","authenticated-orcid":false,"given":"Maxim","family":"Topaz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,22]]},"reference":[{"key":"R1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2023.102701","article-title":"Natural language processing with machine learning methods to analyze unstructured patient-reported outcomes derived from electronic health records: a systematic review","volume":"146","author":"Sim","journal-title":"Artif Intell Med"},{"key":"R2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tacc.2021.02.007","article-title":"Natural language processing in medicine: a review","volume":"38","author":"Locke","journal-title":"Trends Anaesth Crit Care"},{"key":"R3","volume":"2018","author":"Wei","journal-title":"AMIA Annu Symp Proc"},{"issue":"9","key":"R4","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"Garcia","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"R5","unstructured":"Krishnakumar A . Active learning literature survey. University of California; 2007. URL: https:\/\/www.researchgate.net\/publication\/228971426 [Accessed 26-08-2026]"},{"key":"R6","unstructured":"Settles B . Active learning literature survey. University of Wisconsin\u2013Madison; 2009. URL: https:\/\/minds.wisc.edu\/server\/api\/core\/bitstreams\/8a78cf83-0702-4157-aeff-3cacb62f9ad5\/content [Accessed 20-08-2026]"},{"key":"R7","unstructured":"Chen Y Mani S . Active learning for unbalanced data in the challenge with multiple models and biasing. Presented at: Active Learning and Experimental Design Workshop in Conjunction with AISTATS 2010; May 26, 2010; Sardinia, Italy. URL: https:\/\/proceedings.mlr.press\/v16\/chen11a\/chen11a.pdf [Accessed 20-08-2026]"},{"key":"R8","doi-asserted-by":"crossref","unstructured":"Wei J Zou K . EDA: easy data augmentation techniques for boosting performance on text classification tasks. In: Inui K Jiang J Ng V Wan X , editors. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics; 2019:6382-6388. [doi: 10.18653\/v1\/D19-1670]","DOI":"10.18653\/v1\/D19-1670"},{"key":"R9","doi-asserted-by":"crossref","unstructured":"Li Z Zhu H Lu Z Yin M . Synthetic data generation with large language models for text classification: potential and limitations. In: Bouamor H Pino J Bali K , editors. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2023:10443-10461. [doi: 10.18653\/v1\/2023.emnlp-main.647]","DOI":"10.18653\/v1\/2023.emnlp-main.647"},{"key":"R10","doi-asserted-by":"crossref","unstructured":"Feng SY Gangal V Wei J et al. A survey of data augmentation approaches for NLP. In: Zong C Xia F Li W Navigli R , editors. Findings of the Association for Computational Linguistics. Association for Computational Linguistics; 2021:968-988. [doi: 10.18653\/v1\/2021.findings-acl.84]","DOI":"10.18653\/v1\/2021.findings-acl.84"},{"issue":"1","key":"R11","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1162\/coli_a_00425","article-title":"To augment or not to augment? A comparative study on text augmentation techniques for low-resource NLP","volume":"48","author":"\u015eahin","journal-title":"Computational Linguistics"},{"issue":"3","key":"R12","doi-asserted-by":"publisher","first-page":"102757","DOI":"10.1016\/j.outlook.2026.102757","article-title":"Assessing the quality and performance of synthetic data augmentation to identify stigmatizing language in obstetric clinical notes","volume":"74","author":"Scroggins","journal-title":"Nurs Outlook"},{"key":"R13","unstructured":"Mikolov T Chen K Corrado G Dean J . Efficient estimation of word representations in vector space. arXiv. Preprint posted online on Jan 16, 2013. [doi: 10.48550\/arXiv.1301.3781]"},{"key":"R14","unstructured":"Salton G . Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley; 1989. ISBN: 978-0-201-12227-5"},{"key":"R15","doi-asserted-by":"crossref","unstructured":"Reimers N Gurevych I . Sentence-BERT: sentence embeddings using siamese BERT-networks. In: Inui K Jiang J Ng V Wan X , editors. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics; 2019:3982-3992. [doi: 10.18653\/v1\/D19-1410]","DOI":"10.18653\/v1\/D19-1410"},{"issue":"11","key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/22508","article-title":"Identification of semantically similar sentences in clinical notes: iterative intermediate training using multi-task learning","volume":"8","author":"Mahajan","journal-title":"JMIR Med Inform"},{"issue":"5","key":"R17","doi-asserted-by":"publisher","DOI":"10.2196\/23099","article-title":"Predicting semantic similarity between clinical sentence pairs using transformer models: evaluation and representational analysis","volume":"9","author":"Ormerod","journal-title":"JMIR Med Inform"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01377-1","article-title":"Clinical entity augmented retrieval for clinical information extraction","volume":"8","author":"Lopez","journal-title":"NPJ Digit Med"},{"key":"R19","doi-asserted-by":"publisher","DOI":"10.2196\/82997","article-title":"Improving retrieval augmented generation for health care by fine-tuning clinical embedding models: development and evaluation study","volume":"28","author":"Arzideh","journal-title":"J Med Internet Res"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Zhang B Rouhizadeh H Chen Y et al. Retrieval-augmented generation in biomedicine: a survey of technologies, datasets, and clinical applications. Research Square. Preprint posted online on Dec 15, 2025. [doi: 10.21203\/rs.3.rs-8330917\/v1]","DOI":"10.21203\/rs.3.rs-8330917\/v1"},{"issue":"3","key":"R21","doi-asserted-by":"publisher","DOI":"10.1080\/01612840802694668","article-title":"Stigmatizing language with unintended meanings: \u201cpersons with mental illness\u201d or \u201cmentally ill persons\u201d?","volume":"30","author":"Shattell","journal-title":"Issues Ment Health Nurs"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","first-page":"203","DOI":"10.1377\/hlthaff.2021.01423","article-title":"Negative patient descriptors: documenting racial bias in the electronic health record","volume":"41","author":"Sun","journal-title":"Health Aff (Millwood)"},{"issue":"2","key":"R23","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1080\/08964289.2019.1588220","article-title":"Ubiquitous yet unclear: a systematic review of medical mistrust","volume":"45","author":"Benkert","journal-title":"Behav Med"},{"key":"R24","unstructured":"Martin JA Hamilton BE Osterman MJK . Births in the United States, 2023. National Center for Health Statistics; 2022. URL: https:\/\/www.cdc.gov\/nchs\/data\/databriefs\/db507.pdf [Accessed 20-08-2026]"},{"issue":"1","key":"R25","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.44967","article-title":"Examination of stigmatizing language in the electronic health record","volume":"5","author":"Himmelstein","journal-title":"JAMA Netw Open"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"37073504","DOI":"10.1111\/nin.12557","article-title":"A qualitative analysis of stigmatizing language in birth admission clinical notes","volume":"30","author":"Barcelona","journal-title":"Nurs Inq"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1097\/AOG.0000000000005333","article-title":"The power of language in hospital care for pregnant and birthing people: a vision for change","volume":"142","author":"Barcelona","journal-title":"Obstet Gynecol"},{"issue":"3","key":"R28","doi-asserted-by":"publisher","first-page":"578","DOI":"10.1007\/s10995-023-03857-4","article-title":"Using natural language processing to identify stigmatizing language in labor and birth clinical notes","volume":"28","author":"Barcelona","journal-title":"Matern Child Health J"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.jogn.2024.10.003","article-title":"Secondary qualitative analysis of stigmatizing and nonstigmatizing language used in hospital birth settings","volume":"54","author":"Barcelona","journal-title":"J Obstet Gynecol Neonatal Nurs"},{"issue":"7","key":"R30","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2021.17052","article-title":"Physician use of stigmatizing language in patient medical records","volume":"4","author":"Park","journal-title":"JAMA Netw Open"},{"issue":"2","key":"R31","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1093\/jamia\/ocae290","article-title":"Identifying stigmatizing and positive\/preferred language in obstetric clinical notes using natural language processing","volume":"32","author":"Scroggins","journal-title":"J Am Med Inform Assoc"},{"key":"R32","unstructured":"Sentence-transformers\/multi-qa-distilbert-cos-v1. Hugging Face. 2024. URL: https:\/\/huggingface.co\/sentence-transformers\/multi-qa-distilbert-cos-v1 [Accessed 20-08-2026]"},{"key":"R33","unstructured":"Excoffier JB Roehr T Figueroa A Papaioannou JM Bressem K Ortala M . Generalist embedding models are better at short-context clinical semantic search than specialized embedding models. arXiv. Preprint posted online on Jan 3, 2024. [doi: 10.48550\/arXiv.2401.01943]"},{"issue":"3","key":"R34","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1109\/TBDATA.2019.2921572","article-title":"Billion-scale similarity search with GPUs","volume":"7","author":"Johnson","journal-title":"IEEE Trans Big Data"},{"key":"R35","unstructured":"Manning CD Raghavan P Sch\u00fctze H . Introduction to Information Retrieval. Cambridge University Press; 2009. URL: https:\/\/nlp.stanford.edu\/IR-book\/pdf\/irbookprint.pdf [Accessed 20-08-2026]"},{"issue":"3","key":"R36","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1136\/amiajnl-2013-001837","article-title":"Evaluating the impact of pre-annotation on annotation speed and potential bias: natural language processing gold standard development for clinical named entity recognition in clinical trial announcements","volume":"21","author":"Lingren","journal-title":"J Am Med Inform Assoc"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2014.05.002","article-title":"Evaluating the effects of machine pre-annotation and an interactive annotation interface on manual de-identification of clinical text","volume":"50","author":"South","journal-title":"J Biomed Inform"},{"key":"R38","doi-asserted-by":"publisher","DOI":"10.1609\/icwsm.v19i1.35883","article-title":"Keeping humans in the loop: human-centered automated annotation with generative AI","volume":"19","author":"Pangakis","journal-title":"ICWSM"},{"key":"R39","doi-asserted-by":"crossref","unstructured":"Pennington J Socher R Manning C . Glove: global vectors for word representation. Presented at: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP); Oct 25-29, 2014; Doha, Qatar. [doi: 10.3115\/v1\/D14-1162]","DOI":"10.3115\/v1\/D14-1162"},{"key":"R40","unstructured":"Devlin J Chang MW Lee K Toutanova K . BERT: pre-training of deep bidirectional transformers for language understanding. arXiv. Preprint posted online on Oct 11, 2018. [doi: 10.48550\/arXiv.1810.04805]"},{"key":"R41","unstructured":"Arora S Liang Y Ma T . A simple but tough-to-beat baseline for sentence embeddings. Presented at: 5th International Conference on Learning Representations (ICLR 2017); Apr 24-26, 2017; Toulon, France. URL: https:\/\/openreview.net\/pdf?id=SyK00v5xx [Accessed 20-08-2026]"},{"key":"R42","doi-asserted-by":"crossref","unstructured":"Ethayarajh K . How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In: Inui K Jiang J Ng V Wan X , editors. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP. Association for Computational Linguistics; 2019:55-65. [doi: 10.18653\/v1\/D19-1006]","DOI":"10.18653\/v1\/D19-1006"},{"key":"R43","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2017.08.001","article-title":"Active learning reduces annotation time for clinical concept extraction","volume":"106","author":"Kholghi","journal-title":"Int J Med Inform"},{"issue":"11","key":"R44","doi-asserted-by":"publisher","first-page":"1314","DOI":"10.1093\/jamia\/ocz102","article-title":"Cost-aware active learning for named entity recognition in clinical text","volume":"26","author":"Wei","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"R45","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0292356","article-title":"A study of deep active learning methods to reduce labelling efforts in biomedical relation extraction","volume":"18","author":"Nachtegael","journal-title":"PLoS ONE"},{"key":"R46","doi-asserted-by":"crossref","unstructured":"Alsentzer E Murphy J Boag W et al. Publicly available clinical. In: Rumshisky A Roberts K Bethard S Naumann T , editors. Proceedings of the 2nd Clinical Natural Language Processing Workshop. Association for Computational Linguistics; 2019:72-78. [doi: 10.18653\/v1\/W19-1909]","DOI":"10.18653\/v1\/W19-1909"},{"issue":"4","key":"R47","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","journal-title":"Inf Process Manag"},{"key":"R48","doi-asserted-by":"crossref","unstructured":"Ward N Marco D . A collection of pragmatic-similarity judgments over spoken dialog utterances. In: Calzolari N Kan MY Hoste V Lenci A Sakti S Xue N , editors. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING). ELRA and ICCL; 154-163. [doi: 10.63317\/5iqq22vfx8n2]","DOI":"10.63317\/5iqq22vfx8n2"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T21:01:46Z","timestamp":1790110906000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e85088"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,22]]},"references-count":48,"URL":"https:\/\/doi.org\/10.2196\/85088","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,22]]},"article-number":"v14i9e85088"}}