{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T19:53:15Z","timestamp":1772913195974,"version":"3.50.1"},"reference-count":40,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2019,6,14]],"date-time":"2019-06-14T00:00:00Z","timestamp":1560470400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EMPATHY","award":["BB\/M006891\/1"],"award-info":[{"award-number":["BB\/M006891\/1"]}]},{"name":"MMPathIC","award":["MR\/N00583X\/1"],"award-info":[{"award-number":["MR\/N00583X\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>This article describes an ensembling system to automatically extract adverse drug events and drug related entities from clinical narratives, which was developed for the 2018 n2c2 Shared Task Track 2.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We designed a neural model to tackle both nested (entities embedded in other entities) and polysemous entities (entities annotated with multiple semantic types) based on MIMIC III discharge summaries. To better represent rare and unknown words in entities, we further tokenized the MIMIC III data set by splitting the words into finer-grained subwords. We finally combined all the models to boost the performance. Additionally, we implemented a featured-based conditional random field model and created an ensemble to combine its predictions with those of the neural model.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Our method achieved 92.78% lenient micro F1-score, with 95.99% lenient precision, and 89.79% lenient recall, respectively. Experimental results showed that combining the predictions of either multiple models, or of a single model with different settings can improve performance.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>Analysis of the development set showed that our neural models can detect more informative text regions than feature-based conditional random field models. Furthermore, most entity types significantly benefit from subword representation, which also allows us to extract sparse entities, especially nested entities.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>The overall results have demonstrated that the ensemble method can accurately recognize entities, including nested and polysemous entities. Additionally, our method can recognize sparse entities by reconsidering the clinical narratives at a finer-grained subword level, rather than at the word level.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocz075","type":"journal-article","created":{"date-parts":[[2019,5,7]],"date-time":"2019-05-07T22:22:54Z","timestamp":1557267774000},"page":"22-30","source":"Crossref","is-referenced-by-count":48,"title":["An ensemble of neural models for nested adverse drug events and medication extraction with subwords"],"prefix":"10.1093","volume":"27","author":[{"given":"Meizhi","family":"Ju","sequence":"first","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, UK"},{"name":"Artificial Intelligence Research Centre (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]},{"given":"Nhung T H","family":"Nguyen","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, UK"},{"name":"Artificial Intelligence Research Centre (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]},{"given":"Makoto","family":"Miwa","sequence":"additional","affiliation":[{"name":"Toyota Technological Institute, Nagoya, Japan"},{"name":"Artificial Intelligence Research Centre (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]},{"given":"Sophia","family":"Ananiadou","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, UK"},{"name":"Artificial Intelligence Research Centre (AIRC), National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]}],"member":"286","published-online":{"date-parts":[[2019,6,14]]},"reference":[{"key":"2020110613095790000_ocz075-B1","doi-asserted-by":"crossref","first-page":"46226","DOI":"10.1038\/srep46226","article-title":"Analysis of free text in electronic health records for identification of cancer patient trajectories","volume":"7","author":"Jensen","year":"2017","journal-title":"Sci Rep"},{"issue":"6","key":"2020110613095790000_ocz075-B2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3127881","article-title":"Mining electronic health records (EHRs)","volume":"50","author":"Yadav","year":"2018","journal-title":"ACM Comput Surv"},{"key":"2020110613095790000_ocz075-B3","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.jbi.2018.10.005","article-title":"Using clinical natural language processing for health outcomes research: overview and actionable suggestions for future advances","volume":"88","author":"Velupillai","year":"2018","journal-title":"J Biomed Inform"},{"key":"2020110613095790000_ocz075-B4","volume-title":"International Drug Monitoring: The Role of National Centres, Report of a WHO Meeting","year":"1972"},{"key":"2020110613095790000_ocz075-B5","doi-asserted-by":"crossref","first-page":"160035.","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"issue":"11","key":"2020110613095790000_ocz075-B6","doi-asserted-by":"crossref","first-page":"e0187121","DOI":"10.1371\/journal.pone.0187121","article-title":"ADEPt, a semantically-enriched pipeline for extracting adverse drug events from free-text electronic health records","volume":"12","author":"Iqbal","year":"2017","journal-title":"PLoS One"},{"key":"2020110613095790000_ocz075-B7","doi-asserted-by":"crossref","first-page":"13.","DOI":"10.1186\/1472-6947-14-13","article-title":"A pipeline to extract drug-adverse event pairs from multiple data sources","volume":"14","author":"Yeleswarapu","year":"2014","journal-title":"BMC Med Inform Decis Mak"},{"key":"2020110613095790000_ocz075-B8"},{"key":"2020110613095790000_ocz075-B9"},{"issue":"1","key":"2020110613095790000_ocz075-B10","doi-asserted-by":"crossref","first-page":"19.","DOI":"10.1186\/s13326-018-0184-y","article-title":"Adverse event detection by integrating Twitter data and VAERS","volume":"9","author":"Wang","year":"2018","journal-title":"J Biomed Semantics"},{"issue":"3","key":"2020110613095790000_ocz075-B11","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1093\/jamia\/ocu041","article-title":"Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features","volume":"22","author":"Nikfarjam","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613095790000_ocz075-B12","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.jbi.2016.06.007","article-title":"Analysis of the effect of sentiment analysis on extracting adverse drug reactions from tweets and forum posts","volume":"62","author":"Korkontzelos","year":"2016","journal-title":"J Biomed Inform"},{"key":"2020110613095790000_ocz075-B13","author":"Roberts"},{"key":"2020110613095790000_ocz075-B14","author":"Belousov"},{"key":"2020110613095790000_ocz075-B15","author":"Cocos"},{"key":"2020110613095790000_ocz075-B16","author":"Dandala"},{"key":"2020110613095790000_ocz075-B17","author":"Gu"},{"key":"2020110613095790000_ocz075-B18","author":"Tiftikci"},{"key":"2020110613095790000_ocz075-B19","author":"Xu"},{"key":"2020110613095790000_ocz075-B20","first-page":"57","author":"Xu"},{"key":"2020110613095790000_ocz075-B21","first-page":"7","author":"Florez"},{"key":"2020110613095790000_ocz075-B22","first-page":"1","author":"Yang"},{"key":"2020110613095790000_ocz075-B23","first-page":"48","author":"Wunnava"},{"issue":"4","key":"2020110613095790000_ocz075-B24","doi-asserted-by":"crossref","first-page":"e12159.","DOI":"10.2196\/12159","article-title":"Extraction of information related to adverse drug events from electronic health record notes: design of an end-to-end model based on deep learning","volume":"6","author":"Li","year":"2018","journal-title":"JMIR Med Inform"},{"key":"2020110613095790000_ocz075-B25","author":"Lample"},{"key":"2020110613095790000_ocz075-B26","first-page":"1812","article-title":"Clinical named entity recognition using deep learning models","volume":"2017","author":"Wu","year":"2018","journal-title":"AMIA Annu Symp Proc"},{"issue":"20","key":"2020110613095790000_ocz075-B27","doi-asserted-by":"crossref","first-page":"3539","DOI":"10.1093\/bioinformatics\/bty356","article-title":"D3NER: biomedical named entity recognition using CRF-biLSTM improved with fine-tuned embeddings of various linguistic information","volume":"34","author":"Dang","year":"2018","journal-title":"Bioinformatics"},{"key":"2020110613095790000_ocz075-B28","author":"Ju"},{"key":"2020110613095790000_ocz075-B29","author":"Sennrich"},{"key":"2020110613095790000_ocz075-B30","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1007\/978-94-011-3488-0_5","volume-title":"Automated Reasoning: Essays in Honor of Woody Bledsoe","author":"Boyer","year":"1991"},{"key":"2020110613095790000_ocz075-B31"},{"issue":"1","key":"2020110613095790000_ocz075-B32","doi-asserted-by":"crossref","first-page":"41.","DOI":"10.1186\/1758-2946-3-41","article-title":"OSCAR4: a flexible architecture for chemical text-mining","volume":"3","author":"Jessop","year":"2011","journal-title":"J Cheminform"},{"issue":"4","key":"2020110613095790000_ocz075-B33","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1016\/j.ipm.2013.03.002","article-title":"Named entity recognition with multiple segment representations","volume":"49","author":"Cho","year":"2013","journal-title":"Inf Process Manag"},{"key":"2020110613095790000_ocz075-B34","first-page":"382","volume-title":"Proceedings of the 10th Panhellenic Conference on Advances in Informatics.","author":"Tsuruoka","year":"2005"},{"key":"2020110613095790000_ocz075-B35"},{"issue":"3","key":"2020110613095790000_ocz075-B36","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s40495-016-0055-0","article-title":"Ontology-based vaccine and drug adverse event representation and theory-guided systematic causal network analysis toward integrative pharmacovigilance research","volume":"2","author":"He","year":"2016","journal-title":"Curr Pharmacol Rep"},{"key":"2020110613095790000_ocz075-B37","author":"Lance"},{"key":"2020110613095790000_ocz075-B38","article-title":"Efficient estimation of word representations in vector space","author":"Mikolov","year":"2013","journal-title":"CoRR"},{"key":"2020110613095790000_ocz075-B39","volume-title":"Computer-Intensive Methods for Testing Hypotheses: An Introduction","author":"Noreen","year":"1989"},{"key":"2020110613095790000_ocz075-B40","first-page":"2951","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems (Vol. 2)","author":"Snoek","year":"2012"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/22\/34152338\/ocz075.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/22\/34152338\/ocz075.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T19:30:37Z","timestamp":1604691037000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/1\/22\/5518594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,14]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,6,14]]},"published-print":{"date-parts":[[2020,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz075","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,1]]},"published":{"date-parts":[[2019,6,14]]}}}