{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T17:33:27Z","timestamp":1775928807072,"version":"3.50.1"},"reference-count":33,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T00:00:00Z","timestamp":1762905600000},"content-version":"vor","delay-in-days":9,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Blueprint Medicines"},{"DOI":"10.13039\/100030924","name":"Kaiser Permanente Southern California","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100030924","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,11,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Systemic mastocytosis (SM) is a rare disorder with heterogeneous, multisystem symptoms that often lead to diagnostic delays. Real-world symptom documentation in unstructured clinical notes may hold untapped potential for earlier recognition, but scalable extraction methods are lacking.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To develop and validate a natural language processing (NLP) algorithm to identify 23 potentially SM-related symptoms from unstructured electronic health record (EHR) documentation and assess its application across SM and comparison groups.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We conducted a retrospective study using EHR data from Kaiser Permanente Southern California (2008-2023). SM patients (n\u2009=\u2009135) were matched 1:2:3 to chronic spontaneous urticaria (CSU) patients (n\u2009=\u2009270) and non-SM\/non-CSU controls (n\u2009=\u2009405). A rule-based NLP algorithm was developed using manually annotated training data (339 patients, 57\u2009495 notes) and validated against double-annotated notes from 16 SM patients (818 notes). Performance was measured by precision, recall, and F1 score. The algorithm was then applied to 118\u2009252 notes from the full cohort.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The algorithm achieved strong overall performance, with precision and recall exceeding 90% for most symptoms. Lower precision was observed for \u201cepigastric or abdominal bloating\u201d (77%) and \u201cswelling\u201d (80%) due to ambiguous or non-patient references. Notes documenting at least one symptom occurred in 15.9% of the full dataset, including 19.6% of CSU notes, followed by SM (15.5%) and non-SM\/non-CSU (11.0%). SM notes had a higher frequency of gastrointestinal (6.8%) and systemic symptoms (3.9%), while CSU notes more often included cutaneous symptoms (13.3%). SM notes also had the greatest proportion documenting 5 or more distinct symptoms (0.7%), suggesting more complex symptom patterns.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>This study demonstrates the feasibility of using rule-based NLP to identify SM-related symptoms from unstructured EHR narratives. The approach achieved strong performance and revealed meaningful patterns in symptom documentation. These findings support the broader utility of NLP for characterizing rare disease symptoms, informing early recognition, and enhancing data-driven care strategies.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooaf154","type":"journal-article","created":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T09:12:09Z","timestamp":1762938729000},"source":"Crossref","is-referenced-by-count":1,"title":["Using natural language processing to identify symptoms in systemic mastocytosis"],"prefix":"10.1093","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1565-0490","authenticated-orcid":false,"given":"Fagen","family":"Xie","sequence":"first","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin Y","family":"Tse","sequence":"additional","affiliation":[{"name":"Department of Allergy, Kaiser Permanente Medical Center , San Diego, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chantal C","family":"Avila","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matt","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mary","family":"Saparudin","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hiba","family":"Atif","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robert S","family":"Zeiger","sequence":"additional","affiliation":[{"name":"Department of Allergy, Kaiser Permanente Medical Center , San Diego, CA,","place":["United States"]},{"name":"Department of Clinical Science, Kaiser Permanente Bernard J. Tyson School of Medicine , San Diego, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kerri","family":"Miller","sequence":"additional","affiliation":[{"name":"Blueprint Medicines Corporation , Cambridge, MA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dakota","family":"Powell","sequence":"additional","affiliation":[{"name":"Blueprint Medicines Corporation , Cambridge, MA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Erin","family":"Sullivan","sequence":"additional","affiliation":[{"name":"Blueprint Medicines Corporation , Cambridge, MA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ben","family":"Lampson","sequence":"additional","affiliation":[{"name":"Blueprint Medicines Corporation , Cambridge, MA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eric J","family":"Puttock","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chris","family":"Yuen","sequence":"additional","affiliation":[{"name":"Blueprint Medicines Corporation , Cambridge, MA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wansu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Research and Evaluation, Kaiser Permanente Southern California , Pasadena, CA,","place":["United States"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2025,11,11]]},"reference":[{"key":"2025111304463235800_ooaf154-B1","doi-asserted-by":"publisher","first-page":"508","DOI":"10.1002\/ajh.26118","article-title":"Systemic mastocytosis in adults: 2021 update on diagnosis, risk stratification and management","volume":"96","author":"Pardanani","year":"2021","journal-title":"Am J Hematol"},{"key":"2025111304463235800_ooaf154-B2","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1016\/j.hoc.2011.09.012","article-title":"Systemic mastocytosis","volume":"25","author":"George","year":"2011","journal-title":"Hematol Oncol Clin North Am"},{"key":"2025111304463235800_ooaf154-B3","doi-asserted-by":"publisher","first-page":"e646","DOI":"10.1097\/HS9.0000000000000646","article-title":"Updated diagnostic criteria and classification of mast cell disorders: a consensus proposal","volume":"5","author":"Valent","year":"2021","journal-title":"Hemasphere"},{"key":"2025111304463235800_ooaf154-B4","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1111\/bjh.12916","article-title":"Epidemiology of systemic mastocytosis in Denmark","volume":"166","author":"Cohen","year":"2014","journal-title":"Br J Haematol"},{"key":"2025111304463235800_ooaf154-B5","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1111\/bjh.12916","article-title":"Epidemiology, prognosis, and risk factors in mastocytosis","volume":"34","author":"Brockow","year":"2014","journal-title":"Immunol Allergy Clin North Am"},{"key":"2025111304463235800_ooaf154-B6","doi-asserted-by":"publisher","first-page":"1830","DOI":"10.1182\/blood-2018-99-120192","article-title":"Systemic mastocytosis in United States: a population based study","volume":"132","author":"Bista","year":"2018","journal-title":"Blood"},{"key":"2025111304463235800_ooaf154-B7","doi-asserted-by":"publisher","first-page":"6380","DOI":"10.3390\/cancers13246380","article-title":"A multidisciplinary diagnostic approach reveals a higher prevalence of indolent systemic mastocytosis: 15-years\u2019 experience of the GISM network","volume":"13","author":"Zanotti","year":"2021","journal-title":"Cancers (Basel)"},{"key":"2025111304463235800_ooaf154-B8","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1159\/000048188","article-title":"Mastocytosis: mediator-related signs and symptoms","volume":"127","author":"Castells","year":"2002","journal-title":"Int Arch Allergy Immunol"},{"key":"2025111304463235800_ooaf154-B9","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1186\/s12885-022-10498-3","article-title":"Comprehensive mastocytosis data analysis from a single center","volume":"23","author":"Tiryaki","year":"2023","journal-title":"BMC Cancer"},{"key":"2025111304463235800_ooaf154-B10","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1186\/s13023-021-02035-5","article-title":"Development of symptom-focused outcome measures for advanced and indolent systemic mastocytosis: the AdvSM-SAF and ISM-SAF\u00a9","volume":"16","author":"Taylor","year":"2021","journal-title":"Orphanet J Rare Dis"},{"key":"2025111304463235800_ooaf154-B11","doi-asserted-by":"publisher","first-page":"e12319","DOI":"10.1002\/clt2.12319","article-title":"Cognitive, neuropsychiatric and neurological alterations in mastocytosis: a systematic review","volume":"13","author":"Sag\u00fces-Ses\u00e9","year":"2023","journal-title":"Clin Transl Allergy"},{"key":"2025111304463235800_ooaf154-B12","doi-asserted-by":"publisher","first-page":"3316","DOI":"10.3390\/cancers13133316","article-title":"Gastrointestinal manifestations in systemic mastocytosis: the need of a multidisciplinary approach","volume":"13","author":"Zanelli","year":"2021","journal-title":"Cancers (Basel)"},{"key":"2025111304463235800_ooaf154-B13","doi-asserted-by":"publisher","first-page":"100316","DOI":"10.1016\/j.jacig.2024.100316","article-title":"MASTering systemic mastocytosis: lessons learned from a large patient cohort","volume":"3","author":"Tse","year":"2024","journal-title":"J Allergy Clin Immunol Glob"},{"key":"2025111304463235800_ooaf154-B14","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.jaip.2024.10.021","article-title":"Patient-reported burden of indolent systemic mastocytosis in a managed care organization","volume":"13","author":"Zeiger","year":"2025","journal-title":"J Allergy Clin Immunol Pract"},{"key":"2025111304463235800_ooaf154-B15","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1186\/s13023-021-02037-3","article-title":"Psychometric evaluation of the Indolent Systemic Mastocytosis Symptom Assessment Form (ISM-SAF) in a phase 2 clinical study","volume":"16","author":"Padilla","year":"2021","journal-title":"Orphanet J Rare Dis"},{"key":"2025111304463235800_ooaf154-B16","doi-asserted-by":"publisher","first-page":"3691","DOI":"10.1002\/cncr.34420","article-title":"Patient-reported outcomes among patients with systemic mastocytosis in routine clinical practice: results of the TouchStone SM Patient Survey","volume":"128","author":"Mesa","year":"2022","journal-title":"Cancer"},{"key":"2025111304463235800_ooaf154-B17","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1136\/jamia.1994.95236146","article-title":"A general natural-language text processor for clinical radiology","volume":"1","author":"Friedman","year":"1994","journal-title":"J Am Med Inform Assoc"},{"key":"2025111304463235800_ooaf154-B18","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1093\/jamia\/ocy173","article-title":"Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review","volume":"26","author":"Koleck","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2025111304463235800_ooaf154-B19","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1016\/j.anai.2013.07.022","article-title":"Automated chart review for asthma cohort identification using natural language processing: an exploratory study","volume":"111","author":"Wu","year":"2013","journal-title":"Ann Allergy Asthma Immunol"},{"key":"2025111304463235800_ooaf154-B20","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1093\/jamia\/ocab015","article-title":"COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model","volume":"28","author":"Wang","year":"2021","journal-title":"J Am Med Inform Assoc"},{"key":"2025111304463235800_ooaf154-B21","doi-asserted-by":"publisher","first-page":"1737","DOI":"10.1109\/JBHI.2021.3123192","article-title":"A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media","volume":"26","author":"Luo","year":"2022","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025111304463235800_ooaf154-B22","doi-asserted-by":"publisher","first-page":"e69132","DOI":"10.2196\/69132","article-title":"Identifying asthma-related symptoms from electronic health records using a hybrid natural language processing approach within a large integrated health care system: retrospective study","volume":"4","author":"Xie","year":"2025","journal-title":"JMIR AI"},{"key":"2025111304463235800_ooaf154-B23","doi-asserted-by":"publisher","first-page":"e41529","DOI":"10.2196\/41529","article-title":"Natural language processing for improved characterization of COVID-19 symptoms: observational study of 350,000 patients in a large integrated health care system","volume":"8","author":"Malden","year":"2022","journal-title":"JMIR Public Health Surveill"},{"key":"2025111304463235800_ooaf154-B24","doi-asserted-by":"publisher","first-page":"e51240","DOI":"10.2196\/51240","article-title":"Identifying symptoms prior to pancreatic ductal adenocarcinoma diagnosis in real-world care settings: natural language processing approach","volume":"3","author":"Xie","year":"2024","journal-title":"JMIR AI"},{"key":"2025111304463235800_ooaf154-B25","doi-asserted-by":"publisher","first-page":"37","DOI":"10.7812\/tpp\/12-031","article-title":"Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data","volume":"16","author":"Koebnick","year":"2012","journal-title":"Perm J"},{"key":"2025111304463235800_ooaf154-B26","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1186\/1472-6947-12-12","article-title":"Performance evaluation of Unified Medical Language System\u2019s synonyms expansion to query PubMed","volume":"12","author":"Griffon","year":"2012","journal-title":"BMC Med Inform Decis Mak"},{"key":"2025111304463235800_ooaf154-B27","author":"Mikolov","year":"2013"},{"key":"2025111304463235800_ooaf154-B28","doi-asserted-by":"publisher","author":"Loper","year":"2022","DOI":"10.3115\/1118108.1118117"},{"key":"2025111304463235800_ooaf154-B29","first-page":"360","article-title":"Understanding interobserver agreement: the kappa statistic","volume":"37","author":"Viera","year":"2005","journal-title":"Fam Med"},{"key":"2025111304463235800_ooaf154-B30","doi-asserted-by":"publisher","first-page":"e26777","DOI":"10.2196\/26777","article-title":"Natural language processing and machine learning methods to characterize unstructured patient-reported outcomes: validation study","volume":"23","author":"Lu","year":"2021","journal-title":"J Med Internet Res"},{"key":"2025111304463235800_ooaf154-B31","doi-asserted-by":"publisher","author":"Alsentzer","year":"2019","DOI":"10.18653\/v1\/w19-1909"},{"key":"2025111304463235800_ooaf154-B32","doi-asserted-by":"publisher","author":"Rijcken","year":"2024","DOI":"10.21203\/rs.3.rs-2320804\/v2"},{"key":"2025111304463235800_ooaf154-B33","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1093\/jamia\/ocac219","article-title":"Automated deidentification of radiology reports combining transformer and \u201chide in plain sight\u201d rule-based methods","volume":"30","author":"Chambon","year":"2023","journal-title":"J Am Med Inform Assoc"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/6\/ooaf154\/65274200\/ooaf154.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/6\/ooaf154\/65274200\/ooaf154.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T09:46:41Z","timestamp":1763027201000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooaf154\/8321497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,3]]},"references-count":33,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11,3]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooaf154","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,12]]},"published":{"date-parts":[[2025,11,3]]},"article-number":"ooaf154"}}