{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,22]],"date-time":"2026-02-22T08:07:01Z","timestamp":1771747621196,"version":"3.50.1"},"reference-count":39,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2019,5,28]],"date-time":"2019-05-28T00:00:00Z","timestamp":1559001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["5R01LM010681"],"award-info":[{"award-number":["5R01LM010681"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["U24 CA194215"],"award-info":[{"award-number":["U24 CA194215"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["5U01TR002062"],"award-info":[{"award-number":["5U01TR002062"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100004917","name":"Cancer Prevention and Research Institute of Texas","doi-asserted-by":"publisher","award":["RP160015"],"award-info":[{"award-number":["RP160015"]}],"id":[{"id":"10.13039\/100004917","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>This article presents our approaches to extraction of medications and associated adverse drug events (ADEs) from clinical documents, which is the second track of the 2018 National NLP Clinical Challenges (n2c2) shared task.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>The clinical corpus used in this study was from the MIMIC-III database and the organizers annotated 303 documents for training and 202 for testing. Our system consists of 2 components: a named entity recognition (NER) and a relation classification (RC) component. For each component, we implemented deep learning-based approaches (eg, BI-LSTM-CRF) and compared them with traditional machine learning approaches, namely, conditional random fields for NER and support vector machines for RC, respectively. In addition, we developed a deep learning-based joint model that recognizes ADEs and their relations to medications in 1 step using a sequence labeling approach. To further improve the performance, we also investigated different ensemble approaches to generating optimal performance by combining outputs from multiple approaches.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our best-performing systems achieved F1 scores of 93.45% for NER, 96.30% for RC, and 89.05% for end-to-end evaluation, which ranked #2, #1, and #1 among all participants, respectively. Additional evaluations show that the deep learning-based approaches did outperform traditional machine learning algorithms in both NER and RC. The joint model that simultaneously recognizes ADEs and their relations to medications also achieved the best performance on RC, indicating its promise for relation extraction.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>In this study, we developed deep learning approaches for extracting medications and their attributes such as ADEs, and demonstrated its superior performance compared with traditional machine learning algorithms, indicating its uses in broader NER and RC tasks in the medical domain.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocz063","type":"journal-article","created":{"date-parts":[[2019,4,18]],"date-time":"2019-04-18T03:22:55Z","timestamp":1555557775000},"page":"13-21","source":"Crossref","is-referenced-by-count":108,"title":["A study of deep learning approaches for medication and adverse drug event extraction from clinical text"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8665-0201","authenticated-orcid":false,"given":"Qiang","family":"Wei","sequence":"first","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Zongcheng","family":"Ji","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Zhiheng","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Dalian University of Technology, Dalian, China"}]},{"given":"Jingcheng","family":"Du","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Jingqi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Jun","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Yang","family":"Xiang","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Firat","family":"Tiryaki","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Stephen","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Yaoyun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Cui","family":"Tao","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]},{"given":"Hua","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA"}]}],"member":"286","published-online":{"date-parts":[[2019,5,28]]},"reference":[{"issue":"11","key":"2020110613065853000_ocz063-B1","doi-asserted-by":"crossref","first-page":"1075","DOI":"10.1007\/s40264-017-0558-6","article-title":"Natural language processing for EHR-based pharmacovigilance: a structured review","volume":"40","author":"Luo","year":"2017","journal-title":"Drug Saf"},{"issue":"3","key":"2020110613065853000_ocz063-B2","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1007\/s40264-013-0018-x","article-title":"Postmarketing safety surveillance","volume":"36","author":"Coloma","year":"2013","journal-title":"Drug Saf"},{"issue":"10","key":"2020110613065853000_ocz063-B3","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1007\/s40264-014-0218-z","article-title":"Text mining for adverse drug events: the promise, challenges, and state of the art","volume":"37","author":"Harpaz","year":"2014","journal-title":"Drug Saf"},{"issue":"3","key":"2020110613065853000_ocz063-B4","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1197\/jamia.M3028","article-title":"Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study","volume":"16","author":"Wang","year":"2009","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613065853000_ocz063-B5","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1038\/clpt.2012.54","article-title":"Detection of pharmacovigilance-related adverse events using electronic health records and automated methods","volume":"92","author":"Haerian","year":"2012","journal-title":"Clin Pharmacol Ther"},{"issue":"6","key":"2020110613065853000_ocz063-B6","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1136\/jamia.2010.008607","article-title":"Drug safety surveillance using de-identified EMR and claims data: issues and challenges","volume":"17","author":"Nadkarni","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2020110613065853000_ocz063-B7","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1111\/j.1365-2753.2008.00992.x","article-title":"Understanding pharmacist decision making for adverse drug event (ADE) detection","volume":"15","author":"Phansalkar","year":"2009","journal-title":"J Eval Clin Pract"},{"issue":"1","key":"2020110613065853000_ocz063-B8","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1197\/jamia.M3378","article-title":"MedEx: a medication information extraction system for clinical narratives","volume":"17","author":"Xu","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110613065853000_ocz063-B9","doi-asserted-by":"crossref","first-page":"524","DOI":"10.1136\/jamia.2010.003939","article-title":"High accuracy information extraction of medication information from clinical notes: 2009 i2b2 medication extraction challenge","volume":"17","author":"Patrick","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"5","key":"2020110613065853000_ocz063-B10","doi-asserted-by":"crossref","first-page":"858","DOI":"10.1136\/amiajnl-2013-002190","article-title":"MedXN: an open source medication extraction and normalization tool for clinical text","volume":"21","author":"Sohn","year":"2014","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613065853000_ocz063-B11","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1093\/bib\/bbv024","article-title":"Community challenges in biomedical text mining over 10 years: success, failure, and the future","volume":"17","author":"Huang","year":"2016","journal-title":"Brief Bioinform"},{"issue":"5","key":"2020110613065853000_ocz063-B12","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1136\/jamia.2010.003947","article-title":"Extracting medication information from clinical text","volume":"17","author":"Uzuner","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613065853000_ocz063-B13","author":"Roberts","year":"2017"},{"issue":"1","key":"2020110613065853000_ocz063-B14","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s40264-018-0762-z","article-title":"Overview of the first natural language processing challenge for extracting medication, indication, and adverse drug events from electronic health record notes (MADE 1.0)","volume":"42","author":"Jagannatha","year":"2019","journal-title":"Drug Saf"},{"key":"2020110613065853000_ocz063-B15","author":"Uzuner"},{"issue":"1","key":"2020110613065853000_ocz063-B16","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1007\/s40264-018-0763-y","article-title":"Detecting adverse drug events with rapidly trained classification models","volume":"42","author":"Chapman","year":"2019","journal-title":"Drug Saf"},{"issue":"Pt 1","key":"2020110613065853000_ocz063-B17","first-page":"739","article-title":"Extraction of adverse drug effects from clinical records","volume":"160","author":"Aramaki","year":"2010","journal-title":"Stud Health Technol Inform"},{"key":"2020110613065853000_ocz063-B18","first-page":"260","author":"Lample","year":"2016"},{"key":"2020110613065853000_ocz063-B19","author":"Huang","year":"2015"},{"key":"2020110613065853000_ocz063-B20","first-page":"856","article-title":"Structured prediction models for RNN based sequence labeling in clinical text","volume":"2016","author":"Jagannatha","year":"2016","journal-title":"Proc Conf Empir Methods Nat Lang Process"},{"key":"2020110613065853000_ocz063-B21","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1186\/s12911-017-0468-7","article-title":"Entity recognition from clinical texts via recurrent neural network","volume":"17 (Suppl 2)","author":"Liu","year":"2017","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613065853000_ocz063-B22","first-page":"39","author":"Dandala","year":"2018"},{"issue":"1","key":"2020110613065853000_ocz063-B23","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1007\/s40264-018-0764-x","article-title":"Adverse drug events detection in clinical notes by jointly modeling entities and relations using neural networks","volume":"42 (","author":"Dandala","year":"2019","journal-title":"Drug Saf"},{"key":"2020110613065853000_ocz063-B24","first-page":"402","author":"Li","year":"2014"},{"key":"2020110613065853000_ocz063-B25","first-page":"1105","author":"Miwa","year":"2016"},{"key":"2020110613065853000_ocz063-B26","doi-asserted-by":"crossref","first-page":"160035.","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"2020110613065853000_ocz063-B27","article-title":"CLAMP \u2013 a toolkit for efficiently building customized clinical natural language processing pipelines","author":"Soysal","year":"2017","journal-title":"J Am Med Informatics Assoc"},{"key":"2020110613065853000_ocz063-B28","doi-asserted-by":"crossref","first-page":"S34","DOI":"10.1016\/j.jbi.2017.05.023","article-title":"De-identification of clinical notes via recurrent neural network and conditional random field","volume":"75","author":"Liu","year":"2017","journal-title":"J Biomed Inform"},{"key":"2020110613065853000_ocz063-B29","first-page":"3111","author":"Mikolov","year":"2013"},{"key":"2020110613065853000_ocz063-B30","volume-title":"Ensemble Methods: Foundations and Algorithms","author":"Zhi-Hua","year":"2012"},{"key":"2020110613065853000_ocz063-B31","first-page":"1292","author":"Lee","year":"2016"},{"key":"2020110613065853000_ocz063-B32","author":"Zhang","year":"2015"},{"key":"2020110613065853000_ocz063-B33","author":"Zeng"},{"key":"2020110613065853000_ocz063-B34"},{"key":"2020110613065853000_ocz063-B35","first-page":"2012","author":"Lin","year":"2018"},{"key":"2020110613065853000_ocz063-B36","author":"Openai","year":"2018"},{"key":"2020110613065853000_ocz063-B37","author":"Devlin","year":"2018"},{"issue":"5","key":"2020110613065853000_ocz063-B38","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1136\/amiajnl-2012-001431","article-title":"Development and evaluation of an ensemble resource linking medications to their indications","volume":"20","author":"Wei","year":"2013","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613065853000_ocz063-B39","author":"Peters","year":"2017"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/13\/34152042\/ocz063.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/13\/34152042\/ocz063.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,6]],"date-time":"2020-12-06T18:33:31Z","timestamp":1607279611000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/1\/13\/5499225"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,28]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,5,28]]},"published-print":{"date-parts":[[2020,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz063","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,1]]},"published":{"date-parts":[[2019,5,28]]}}}