{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T03:47:27Z","timestamp":1787024847296,"version":"build-2736575974"},"reference-count":49,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Asthma-related symptoms are significant predictors of asthma exacerbation. Most of these symptoms are documented in clinical notes in a free-text format, and effective methods for capturing asthma-related symptoms from unstructured data are lacking.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The study aims to develop a natural language processing (NLP) algorithm for identifying symptoms associated with asthma from clinical notes within a large integrated health care system.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We analyzed unstructured clinical notes within 2 years before a visit with asthma diagnosis in 2013\u20102018 and 2021\u20102022 to identify 4 common asthma-related symptoms. Related terms and phrases were initially compiled from publicly available resources and then refined through clinician input and chart review. A rule-based NLP algorithm was iteratively developed and refined via multiple rounds of chart review followed by adjudication. Subsequently, transformer-based deep learning algorithms were trained using the same manually annotated datasets. A hybrid NLP algorithm was then generated by combining rule-based and transformer-based algorithms. The hybrid NLP algorithm was finally applied to the implementation notes.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n A total of 11,374,552 eligible clinical notes with 128,211,793 sentences were analyzed. After applying the hybrid algorithm to implementation notes, at least 1 asthma-related symptom was identified in 1,663,450 out of 127,763,086 (1.3%) sentences and 858,350 out of 11,364,952 (7.55%) notes, respectively. Cough was the most frequently identified at both the sentence (1,363,713\/127,763,086, 1.07%) and note (660,685\/11,364,952, 5.81%) levels, while chest tightness was the least frequent at both the sentence (141,733\/127,763,086, 0.11%) and note (64,251\/11,364,952, 0.57%) levels. The frequency of multiple symptoms ranged from 0.03% (36,057\/127,763,086) to 0.38% (484,050\/127,763,086) at the sentence level and 0.10% (10,954\/11,364,952) to 1.85% (209,805\/11,364,952) at the note level. Validation against 1600 manually annotated clinical notes yielded a positive predictive value ranging from 96.53% (wheezing) to 97.42% (chest tightness) at the sentence level and 96.76% (wheezing) to 97.42% (chest tightness) at the note level. Sensitivity ranged from 93.9% (dyspnea) to 95.95% (cough) at the sentence level and 96% (chest tightness) to 99.07% (cough) at the note level. All 4 symptoms had\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores greater than 0.95 at both the sentence and note levels, regardless of NLP algorithms.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n The developed NLP algorithms could effectively capture asthma-related symptoms from unstructured clinical notes. These algorithms could be used to facilitate early asthma detection and predict exacerbation risk.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/69132","type":"journal-article","created":{"date-parts":[[2025,3,15]],"date-time":"2025-03-15T16:00:07Z","timestamp":1742054407000},"page":"e69132-e69132","source":"Crossref","is-referenced-by-count":7,"title":["Identifying Asthma-Related Symptoms From Electronic Health Records Using a Hybrid Natural Language Processing Approach Within a Large Integrated Health Care System: Retrospective Study"],"prefix":"10.2196","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1565-0490","authenticated-orcid":false,"given":"Fagen","family":"Xie","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5788-5063","authenticated-orcid":false,"given":"Robert S","family":"Zeiger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5627-4726","authenticated-orcid":false,"given":"Mary Marycania","family":"Saparudin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0864-1621","authenticated-orcid":false,"given":"Sahar","family":"Al-Salman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3963-314X","authenticated-orcid":false,"given":"Eric","family":"Puttock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8447-7532","authenticated-orcid":false,"given":"William","family":"Crawford","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7640-5560","authenticated-orcid":false,"given":"Michael","family":"Schatz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4750-7672","authenticated-orcid":false,"given":"Stanley","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1216-323X","authenticated-orcid":false,"given":"William M","family":"Vollmer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3486-7468","authenticated-orcid":false,"given":"Wansu","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,5,2]]},"reference":[{"issue":"11","key":"R1","doi-asserted-by":"publisher","first-page":"992","DOI":"10.1136\/thx.53.11.992","article-title":"Airway hyperresponsiveness in asthma: not just a matter of airway inflammation","volume":"53","author":"Brusasco","journal-title":"Thorax"},{"issue":"10258","key":"R2","doi-asserted-by":"publisher","first-page":"1204","DOI":"10.1016\/S0140-6736(20)30925-9","article-title":"Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019","volume":"396","author":"GBD 2019 Diseases and Injuries Collaborators","journal-title":"Lancet"},{"key":"R3","unstructured":"Asthma prevalence in the United States, 2001\u20132021. Centers for Disease Control and Prevention. URL: https:\/\/www.cdc.gov\/asthma\/Asthma-Prevalence-US-2023-508.pdf [Accessed 21-11-2024]"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1159\/000338998","article-title":"The cost of persistent asthma in Europe: an international population-based study in adults","volume":"160","author":"Accordini","journal-title":"Int Arch Allergy Immunol"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1016\/j.jaip.2013.07.016","article-title":"Change in asthma control over time: predictors and outcomes","volume":"2","author":"Schatz","journal-title":"J Allergy Clin Immunol Pract"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","first-page":"252","DOI":"10.4187\/respcare.06714","article-title":"Frequency of signs and symptoms in persons with asthma","volume":"65","author":"He","journal-title":"Respir Care"},{"issue":"2","key":"R7","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1136\/jamia.1994.95236146","article-title":"A general natural-language text processor for clinical radiology","volume":"1","author":"Friedman","journal-title":"J Am Med Inform Assoc"},{"issue":"11","key":"R8","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0187121","article-title":"ADEPt, a semantically-enriched pipeline for extracting adverse drug events from free-text electronic health records","volume":"12","author":"Iqbal","journal-title":"PLoS ONE"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1093\/jamia\/ocy173","article-title":"Natural language processing of symptoms documented in free-text narratives of electronic health records: a systematic review","volume":"26","author":"Koleck","journal-title":"J Am Med Inform Assoc"},{"issue":"3","key":"R10","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1016\/j.ijmedinf.2011.11.005","article-title":"Detection of infectious symptoms from VA emergency department and primary care clinical documentation","volume":"81","author":"Matheny","journal-title":"Int J Med Inform"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.7812\/TPP\/20.022","article-title":"Prevalence and characteristics of chronic cough in adults identified by administrative data","volume":"24","author":"Zeiger","journal-title":"Perm J"},{"issue":"12","key":"R12","doi-asserted-by":"publisher","DOI":"10.2196\/41529","article-title":"Natural language processing for improved characterization of COVID-19 symptoms: observational study of 350,000 patients in a large integrated health care system","volume":"8","author":"Malden","journal-title":"JMIR Public Health Surveill"},{"key":"R13","doi-asserted-by":"publisher","DOI":"10.2196\/51240","article-title":"Identifying symptoms prior to pancreatic ductal adenocarcinoma diagnosis in real-world care settings: natural language processing approach","volume":"3","author":"Xie","journal-title":"JMIR AI"},{"issue":"6","key":"R14","doi-asserted-by":"publisher","first-page":"1275","DOI":"10.1093\/jamia\/ocab015","article-title":"COVID-19 SignSym: a fast adaptation of a general clinical NLP tool to identify and normalize COVID-19 signs and symptoms to OMOP common data model","volume":"28","author":"Wang","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"R15","doi-asserted-by":"publisher","first-page":"1737","DOI":"10.1109\/JBHI.2021.3123192","article-title":"A deep language model for symptom extraction from clinical text and its application to extract COVID-19 symptoms from social media","volume":"26","author":"Luo","journal-title":"IEEE J Biomed Health Inform"},{"key":"R16","doi-asserted-by":"publisher","DOI":"10.1016\/j.ymeth.2019.07.009","article-title":"A disease inference method based on symptom extraction and bidirectional long short term memory networks","volume":"173","author":"Guo","journal-title":"Methods"},{"key":"R17","doi-asserted-by":"crossref","unstructured":"Devlin J Chang M Lee K et al. BERT: pre-training of deep bidirectional transformers for language understanding. Presented at: 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies; Jun 3-5, 2019; Minneapolis, MA, United States. [doi: 10.18653\/v1\/N19-1423]","DOI":"10.18653\/v1\/N19-1423"},{"key":"R18","unstructured":"Pang C Jiang XZ Kalluri KS et al. CEHR-BERT: incorporating temporal information from structured EHR data to improve prediction tasks. arXiv. Preprint posted online onNov10, 2021. [doi: 10.48550\/arXiv.2111.08585]"},{"issue":"5","key":"R19","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1016\/j.anai.2013.07.022","article-title":"Automated chart review for asthma cohort identification using natural language processing: an exploratory study","volume":"111","author":"Wu","journal-title":"Ann Allergy Asthma Immunol"},{"issue":"4","key":"R20","doi-asserted-by":"publisher","first-page":"430","DOI":"10.1164\/rccm.201610-2006OC","article-title":"Application of a natural language processing algorithm to asthma ascertainment. An automated chart review","volume":"196","author":"Wi","journal-title":"Am J Respir Crit Care Med"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1016\/j.jaip.2017.04.041","article-title":"Natural language processing for asthma ascertainment in different practice settings","volume":"6","author":"Wi","journal-title":"J Allergy Clin Immunol Pract"},{"issue":"3","key":"R22","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1093\/jamia\/ocx138","article-title":"Clinical documentation variations and NLP system portability: a case study in asthma birth cohorts across institutions","volume":"25","author":"Sohn","journal-title":"J Am Med Inform Assoc"},{"issue":"6","key":"R23","doi-asserted-by":"publisher","first-page":"2292","DOI":"10.1016\/j.jaci.2017.12.1003","article-title":"Ascertainment of asthma prognosis using natural language processing from electronic medical records","volume":"141","author":"Sohn","journal-title":"J Allergy Clin Immunol"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.1186\/s12890-018-0593-9","article-title":"Automated chart review utilizing natural language processing algorithm for asthma predictive index","volume":"18","author":"Kaur","journal-title":"BMC Pulm Med"},{"issue":"Suppl 7","key":"R25","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-021-01633-4","article-title":"Identification of asthma control factor in clinical notes using a hybrid deep learning model","volume":"21","author":"Agnikula Kshatriya","journal-title":"BMC Med Inform Decis Mak"},{"issue":"4","key":"R26","doi-asserted-by":"publisher","first-page":"1047","DOI":"10.1016\/j.jaip.2021.11.004","article-title":"Artificial intelligence assesses clinicians\u2019 adherence to asthma guidelines using electronic health records","volume":"10","author":"Sagheb","journal-title":"J Allergy Clin Immunol Pract"},{"issue":"3","key":"R27","doi-asserted-by":"publisher","first-page":"37","DOI":"10.7812\/TPP\/12-031","article-title":"Sociodemographic characteristics of members of a large, integrated health care system: comparison with US Census Bureau data","volume":"16","author":"Koebnick","journal-title":"Perm J"},{"issue":"4","key":"R28","doi-asserted-by":"publisher","DOI":"10.2196\/26558","article-title":"Impact of the COVID-19 pandemic on health care utilization in a large Integrated health care system: retrospective cohort study","volume":"23","author":"Xu","journal-title":"J Med Internet Res"},{"issue":"10","key":"R29","doi-asserted-by":"publisher","first-page":"2705","DOI":"10.1016\/j.jaip.2024.05.034","article-title":"Risk factors for acute asthma exacerbations in adults with mild asthma","volume":"12","author":"Chen","journal-title":"J Allergy Clin Immunol Pract"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1186\/1472-6947-12-12","article-title":"Performance evaluation of Unified Medical Language System\u00ae\u2019s synonyms expansion to query PubMed","volume":"12","author":"Griffon","journal-title":"BMC Med Inform Decis Mak"},{"key":"R31","unstructured":"Mikolov T Chen K Corrado G et al. Efficient estimation of word representations in vector space. arXiv. Preprint posted online onSep7, 2013. [doi: 10.48550\/arXiv.1301.3781]"},{"key":"R32","doi-asserted-by":"crossref","unstructured":"Loper E Bird S . NLTK: the natural language toolkit. Presented at: ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics; Jul 7, 2002; Philadelphia, PA, United States. [doi: 10.3115\/1118108.1118117]","DOI":"10.3115\/1118108.1118117"},{"issue":"5","key":"R33","first-page":"360","volume":"37","author":"Viera","journal-title":"Fam Med"},{"key":"R34","unstructured":"Kingma DP Ba J . Adam: a method for stochastic optimization. arXiv. Preprint posted online onJan30, 2017. [doi: 10.48550\/arXiv.1412.6980]"},{"key":"R35","unstructured":"Google-research\/bert. GitHub. URL: https:\/\/github.com\/google-research\/bert [Accessed 26-02-2025]"},{"key":"R36","doi-asserted-by":"crossref","unstructured":"Rijcken E Zervanou K Mosteiro P Scheepers F Spruit M Kaymak U . Machine learning vs. rule-based methods for document classification of electronic health records within mental health care - a systematic literature review. Research Square. Preprint posted online onMar21, 2024. [doi: 10.21203\/rs.3.rs-2320804\/v2]","DOI":"10.21203\/rs.3.rs-2320804\/v2"},{"key":"R37","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3436902","article-title":"Large language models and rule-based approaches in domain-specific communication","volume":"12","author":"Halvon\u00edk","journal-title":"IEEE Access"},{"issue":"8","key":"R38","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0255261","article-title":"Artificial intelligence-assisted clinical decision support for childhood asthma management: a randomized clinical trial","volume":"16","author":"Seol","journal-title":"PLoS ONE"},{"issue":"1","key":"R39","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-25773-2","article-title":"Identifying suicide ideation and suicidal attempts in a psychiatric clinical research database using natural language processing","volume":"8","author":"Fernandes","journal-title":"Sci Rep"},{"key":"R40","unstructured":"Gorrell G Jackson R Roberts A et al. Finding negative symptoms of schizophrenia in patient records. Presented at: Proceedings of the Workshop on NLP for Medicine and Biology Associated with RANLP; Sep 3, 2013; Hissar, Bulgaria. URL: https:\/\/aclanthology.org\/W13-51\/ [Accessed 25-04-2025]"},{"issue":"3","key":"R41","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2020-042274","article-title":"Text mining occupations from the mental health electronic health record: a natural language processing approach using records from the Clinical Record Interactive Search (CRIS) platform in south London, UK","volume":"11","author":"Chilman","journal-title":"BMJ Open"},{"issue":"2","key":"R42","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1093\/jamia\/ocac219","article-title":"Automated deidentification of radiology reports combining transformer and \u201chide in plain sight\u201d rule-based methods","volume":"30","author":"Chambon","journal-title":"J Am Med Inform Assoc"},{"key":"R43","unstructured":"Sun X Yang D Li X et al. Interpreting deep learning models in natural language processing: a review. arXiv. Preprint posted online onOct25, 2021. [doi: 10.48550\/arXiv.2110.10470]"},{"issue":"8","key":"R44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3546577","article-title":"Post-hoc interpretability for neural NLP: a survey","volume":"55","author":"Madsen","journal-title":"ACM Comput Surv"},{"key":"R45","doi-asserted-by":"crossref","unstructured":"Danilevsky M Qian K Aharonov R et al. A survey of the state of explainable AI for natural language processing. Presented at: Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing; Dec 4-7, 2020; Suzhou, China. URL: https:\/\/aclanthology.org\/2020.aacl-main.46\/ [Accessed 25-04-2025]","DOI":"10.18653\/v1\/2020.aacl-main.46"},{"key":"R46","unstructured":"Sundararajan M Taly A Yan Q . Axiomatic attribution for deep networks. Presented at: Proceedings of the 34th International Conference on Machine Learning; Aug 6-11, 2017; Sydney, Australia. URL: https:\/\/dl.acm.org\/doi\/10.5555\/3305890.3306024 [Accessed 25-04-2025]"},{"key":"R47","doi-asserted-by":"crossref","unstructured":"Ribeiro MT Singh S Guestrin C . \u201cWhy should I trust you?\u201d: explaining the predictions of any classifier. Presented at: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining; Jun 13-17, 2016; Santa Barbara, CA, United States. [doi: 10.1145\/2939672.2939778]","DOI":"10.1145\/2939672.2939778"},{"key":"R48","unstructured":"Lundberg S Lee SI . A unified approach to interpreting model predictions. arXiv. Preprint posted online onMar22, 2017. [doi: 10.48550\/arXiv.1705.07874]"},{"key":"R49","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.357","article-title":"Empirical evaluation of pre-trained transformers for human-level NLP: the role of sample size and dimensionality","volume":"2021","author":"Ganesan","journal-title":"Proc Conf"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e69132","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:56:34Z","timestamp":1782842194000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e69132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,2]]},"references-count":49,"URL":"https:\/\/doi.org\/10.2196\/69132","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,2]]},"article-number":"v4i11e69132"}}