{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T04:56:22Z","timestamp":1775278582053,"version":"3.50.1"},"reference-count":42,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2019,8,7]],"date-time":"2019-08-07T00:00:00Z","timestamp":1565136000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Biotechnology and Biological Services Research Council EMPATHY","award":["BB\/M006891\/1"],"award-info":[{"award-number":["BB\/M006891\/1"]}]},{"name":"Manchester Molecular Pathology Innovation Centre","award":["MR\/N00583X\/1"],"award-info":[{"award-number":["MR\/N00583X\/1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,1,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Identification of drugs, associated medication entities, and interactions among them are crucial to prevent unwanted effects of drug therapy, known as adverse drug events. This article describes our participation to the n2c2 shared-task in extracting relations between medication-related entities in electronic health records.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>We proposed an ensemble approach for relation extraction and classification between drugs and medication-related entities. We incorporated state-of-the-art named-entity recognition (NER) models based on bidirectional long short-term memory (BiLSTM) networks and conditional random fields (CRF) for end-to-end extraction. We additionally developed separate models for intra- and inter-sentence relation extraction and combined them using an ensemble method. The intra-sentence models rely on bidirectional long short-term memory networks and attention mechanisms and are able to capture dependencies between multiple related pairs in the same sentence. For the inter-sentence relations, we adopted a neural architecture that utilizes the Transformer network to improve performance in longer sequences.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our team ranked third with a micro-averaged F1 score of 94.72% and 87.65% for relation and end-to-end relation extraction, respectively (Tracks 2 and 3). Our ensemble effectively takes advantages from our proposed models. Analysis of the reported results indicated that our proposed approach is more generalizable than the top-performing system, which employs additional training data- and corpus-driven processing techniques.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>We proposed a relation extraction system to identify relations between drugs and medication-related entities. The proposed approach is independent of external syntactic tools. Analysis showed that by using latent Drug-Drug interactions we were able to significantly improve the performance of non\u2013Drug-Drug pairs in EHRs.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocz101","type":"journal-article","created":{"date-parts":[[2019,5,24]],"date-time":"2019-05-24T19:11:29Z","timestamp":1558725089000},"page":"39-46","source":"Crossref","is-referenced-by-count":94,"title":["Adverse drug events and medication relation extraction in electronic health records with ensemble deep learning methods"],"prefix":"10.1093","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5217-9848","authenticated-orcid":false,"given":"Fenia","family":"Christopoulou","sequence":"first","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, United Kingdom"},{"name":"Artificial Intelligence Research Centre, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]},{"given":"Thy Thy","family":"Tran","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, United Kingdom"},{"name":"Artificial Intelligence Research Centre, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]},{"given":"Sunil Kumar","family":"Sahu","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, United Kingdom"}]},{"given":"Makoto","family":"Miwa","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Research Centre, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"},{"name":"Toyota Technological Institute, Nagoya, Japan"}]},{"given":"Sophia","family":"Ananiadou","sequence":"additional","affiliation":[{"name":"National Centre for Text Mining, School of Computer Science, The University of Manchester, Manchester, United Kingdom"},{"name":"Artificial Intelligence Research Centre, National Institute of Advanced Industrial Science and Technology (AIST), Tokyo, Japan"}]}],"member":"286","published-online":{"date-parts":[[2019,8,7]]},"reference":[{"issue":"1","key":"2020110613093410300_ocz101-B1","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1001\/jama.1995.03530010043033","article-title":"Incidence of adverse drug events and potential adverse drug events: implications for prevention","volume":"274","author":"Bates","year":"1995","journal-title":"JAMA"},{"issue":"3","key":"2020110613093410300_ocz101-B2","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1197\/jamia.M3028","article-title":"Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study","volume":"16","author":"Wang","year":"2009","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2020110613093410300_ocz101-B3","doi-asserted-by":"crossref","first-page":"64.","DOI":"10.1186\/1471-2105-15-64","article-title":"Knowledge-based extraction of adverse drug events from biomedical text","volume":"15","author":"Kang","year":"2014","journal-title":"BMC Bioinformatics"},{"key":"2020110613093410300_ocz101-B4","first-page":"463","author":"Dasgupta","year":"2017"},{"issue":"5","key":"2020110613093410300_ocz101-B5","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1136\/amiajnl-2011-000203","article-title":"2010 i2b2\/VA challenge on concepts, assertions, and relations in clinical text","volume":"18","author":"Uzuner","year":"2011","journal-title":"J Am Med Inform Assoc"},{"key":"2020110613093410300_ocz101-B6","author":"Roberts","year":"2010"},{"key":"2020110613093410300_ocz101-B7","author":"de Bruijn","year":"2010"},{"key":"2020110613093410300_ocz101-B8","author":"Sahu","year":"2016"},{"issue":"5","key":"2020110613093410300_ocz101-B9","doi-asserted-by":"crossref","first-page":"885","DOI":"10.1016\/j.jbi.2012.04.008","article-title":"Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports","volume":"45","author":"Gurulingappa","year":"2012","journal-title":"J Biomed Inform"},{"issue":"8","key":"2020110613093410300_ocz101-B10","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput"},{"issue":"4","key":"2020110613093410300_ocz101-B11","doi-asserted-by":"crossref","first-page":"e12159","DOI":"10.2196\/12159","article-title":"Extraction of information related to adverse drug events from electronic health record notes: design of an end-to-end model based on deep learning","volume":"6","author":"Li","year":"2018","journal-title":"JMIR Med Inform"},{"key":"2020110613093410300_ocz101-B12","author":"Ramamoorthy"},{"key":"2020110613093410300_ocz101-B13","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.eswa.2018.07.032","article-title":"Joint entity recognition and relation extraction as a multi-head selection problem","volume":"114","author":"Bekoulis","year":"2018","journal-title":"Expert Syst Appl"},{"key":"2020110613093410300_ocz101-B14","author":"Bekoulis"},{"key":"2020110613093410300_ocz101-B15","first-page":"1","author":"Zhao","year":"2018"},{"key":"2020110613093410300_ocz101-B16","author":"Roberts","year":"2017"},{"key":"2020110613093410300_ocz101-B17","author":"Xu","year":"2017"},{"key":"2020110613093410300_ocz101-B18","author":"Dandala"},{"issue":"2","key":"2020110613093410300_ocz101-B19","doi-asserted-by":"crossref","first-page":"e29","DOI":"10.2196\/publichealth.9361","article-title":"Clinical relation extraction toward drug safety surveillance using electronic health record narratives: classical learning versus deep learning","volume":"4","author":"Munkhdalai","year":"2018","journal-title":"JMIR Public Health Surveill"},{"issue":"1","key":"2020110613093410300_ocz101-B20","first-page":"99\u2013111","article-title":"Overview of the first natural language processing challenge for extracting medication, indication, and adverse drug events from electronic health record notes (MADE 1.0","volume":"42","author":"Jagannatha","year":"2018","journal-title":"Drug Saf"},{"key":"2020110613093410300_ocz101-B21","first-page":"39","author":"Dandala","year":"2018"},{"issue":"1","key":"2020110613093410300_ocz101-B22","doi-asserted-by":"crossref","first-page":"198.","DOI":"10.1186\/s12859-017-1609-9","article-title":"A neural joint model for entity and relation extraction from biomedical text","volume":"18","author":"Li","year":"2018","journal-title":"BMC Bioinform"},{"key":"2020110613093410300_ocz101-B23","doi-asserted-by":"crossref","article-title":"A walk-based model on entity graphs for relation extraction","author":"Christopoulou","DOI":"10.18653\/v1\/P18-2014"},{"key":"2020110613093410300_ocz101-B24","first-page":"554","author":"Yi","year":"2017"},{"key":"2020110613093410300_ocz101-B25","first-page":"98","author":"Bj\u00f6rne","year":"2018"},{"key":"2020110613093410300_ocz101-B26","first-page":"2335","author":"Zeng","year":"2014"},{"key":"2020110613093410300_ocz101-B27","first-page":"5998","article-title":"Attention is all you need","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"issue":"1","key":"2020110613093410300_ocz101-B28","doi-asserted-by":"crossref","first-page":"44.","DOI":"10.1186\/s40360-017-0153-6","article-title":"Data-driven prediction of adverse drug reactions induced by drug-drug interactions","volume":"18","author":"Liu","year":"2017","journal-title":"BMC Pharmacol Toxicol"},{"key":"2020110613093410300_ocz101-B29","author":"Verga"},{"key":"2020110613093410300_ocz101-B30","author":"He"},{"key":"2020110613093410300_ocz101-B31","author":"Ju"},{"key":"2020110613093410300_ocz101-B32","first-page":"22","author":"Ju","year":"2020"},{"key":"2020110613093410300_ocz101-B33","doi-asserted-by":"crossref","first-page":"160035.","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"issue":"1","key":"2020110613093410300_ocz101-B34","doi-asserted-by":"crossref","first-page":"41.","DOI":"10.1186\/1758-2946-3-41","article-title":"OSCAR4: a flexible architecture for chemical text-mining","volume":"3","author":"Jessop","year":"2011","journal-title":"J Cheminform"},{"key":"2020110613093410300_ocz101-B35","first-page":"166","author":"Chiu","year":"2016"},{"issue":"5","key":"2020110613093410300_ocz101-B36","first-page":"553","article-title":"Application of majority voting to pattern recognition: an analysis of its behavior and performance","volume":"27","author":"Lam","year":"1997","journal-title":"Trans of SMC"},{"key":"2020110613093410300_ocz101-B37","first-page":"2168","author":"Mu","year":"2009"},{"key":"2020110613093410300_ocz101-B38","volume-title":"Computer-Intensive Methods for Testing Hypotheses","author":"Noreen","year":"1989"},{"key":"2020110613093410300_ocz101-B39","author":"Zhou"},{"issue":"1","key":"2020110613093410300_ocz101-B40","first-page":"13","volume":"27","author":"Wei","year":"2020"},{"issue":"D1","key":"2020110613093410300_ocz101-B41","doi-asserted-by":"crossref","first-page":"D1075","DOI":"10.1093\/nar\/gkv1075","article-title":"The SIDER database of drugs and side effects","volume":"44","author":"Kuhn","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"2020110613093410300_ocz101-B42","author":"Miwa"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/39\/34152750\/ocz101.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/27\/1\/39\/34152750\/ocz101.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,15]],"date-time":"2020-12-15T17:26:18Z","timestamp":1608053178000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/27\/1\/39\/5544735"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,7]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,8,7]]},"published-print":{"date-parts":[[2020,1,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocz101","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,1]]},"published":{"date-parts":[[2019,8,7]]}}}