{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T19:55:17Z","timestamp":1791575717325,"version":"4.3.4"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"S21","license":[{"start":{"date-parts":[[2019,12,1]],"date-time":"2019-12-01T00:00:00Z","timestamp":1575158400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2019,12,23]],"date-time":"2019-12-23T00:00:00Z","timestamp":1577059200000},"content-version":"vor","delay-in-days":22,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>Use of medication can cause adverse drug reactions (ADRs), unwanted or unexpected events, which are a major safety concern. Drug labels, or prescribing information or package inserts, describe ADRs. Therefore, systematically identifying ADR information from drug labels is critical in multiple aspects; however, this task is challenging due to the nature of the natural language of drug labels.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>In this paper, we present a machine learning- and rule-based system for the identification of ADR entity mentions in the text of drug labels and their normalization through the Medical Dictionary for Regulatory Activities (MedDRA) dictionary. The machine learning approach is based on a recently proposed deep learning architecture, which integrates bi-directional Long Short-Term Memory (Bi-LSTM), Convolutional Neural Network (CNN), and Conditional Random Fields (CRF) for entity recognition. The rule-based approach, used for normalizing the identified ADR mentions to MedDRA terms, is based on an extension of our in-house text-mining system, SciMiner. We evaluated our system on the Text Analysis Conference (TAC) Adverse Drug Reaction 2017 challenge test data set, consisting of 200 manually curated US FDA drug labels. Our ML-based system achieved 77.0% F1 score on the task of ADR mention recognition and 82.6% micro-averaged F1 score on the task of ADR normalization, while rule-based system achieved 67.4 and 77.6% F1 scores, respectively.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Our study demonstrates that a system composed of a deep learning architecture for entity recognition and a rule-based model for entity normalization is a promising approach for ADR extraction from drug labels.<\/jats:p><\/jats:sec>","DOI":"10.1186\/s12859-019-3195-5","type":"journal-article","created":{"date-parts":[[2019,12,23]],"date-time":"2019-12-23T01:02:30Z","timestamp":1577062950000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Machine learning-based identification and rule-based normalization of adverse drug reactions in drug labels"],"prefix":"10.1186","volume":"20","author":[{"given":"Mert","family":"Tiftikci","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arzucan","family":"\u00d6zg\u00fcr","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongqun","family":"He","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junguk","family":"Hur","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,12,23]]},"reference":[{"key":"3195_CR1","volume-title":"The importance of pharmacovigilance","author":"World Health Organization and others","year":"2002","unstructured":"World Health Organization and others. The importance of pharmacovigilance; 2002."},{"key":"3195_CR2","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1046\/j.1525-1497.2003.20130.x","volume":"18","author":"SR Ahmad","year":"2003","unstructured":"Ahmad SR. Adverse drug event monitoring at the Food and Drug Administration. J Gen Intern Med. 2003;18:57\u201360.","journal-title":"J Gen Intern Med"},{"key":"3195_CR3","first-page":"16","volume-title":"First international workshop on knowledge discovery and health care management (KD-HCM), European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD)","author":"H Gurulingappa","year":"2011","unstructured":"Gurulingappa H, Fluck J, Hofmann-Apitius M, Toldo L. Identification of adverse drug event assertive sentences in medical case reports. In: First international workshop on knowledge discovery and health care management (KD-HCM), European conference on machine learning and principles and practice of knowledge discovery in databases (ECML PKDD); 2011. p. 16\u201327."},{"key":"3195_CR4","first-page":"117","volume-title":"Proceedings of the 2010 Workshop on biomedical natural language processing","author":"R Leaman","year":"2010","unstructured":"Leaman R, Wojtulewicz L, Sullivan R, Skariah A, Yang J, Gonzales G. Towards Internet-age Pharmacovigilance: Extracting Adverse Drug Reactions from User Posts to Health-related Social Networks. In: Proceedings of the 2010 Workshop on biomedical natural language processing. Stroudsburg: Association for Computational Linguistics; 2010. p. 117\u201325."},{"key":"3195_CR5","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1016\/j.jbi.2014.11.002","volume":"53","author":"A Sarker","year":"2015","unstructured":"Sarker A, Gonzales G. Portable automatic text classification for adverse drug reaction detection via multi-corpus training. J Biomed Inform. 2015;53:196\u2013207.","journal-title":"J Biomed Inform"},{"key":"3195_CR6","first-page":"1019","volume":"2011","author":"A Nikfarjam","year":"2011","unstructured":"Nikfarjam A, Gonzalez GH. Pattern Mining for Extraction of mentions of adverse drug reactions from user comments. AMIA Ann Symp Proc. 2011;2011:1019\u201326.","journal-title":"AMIA Ann Symp Proc"},{"issue":"10","key":"3195_CR7","doi-asserted-by":"publisher","first-page":"777","DOI":"10.1007\/s40264-014-0218-z","volume":"37","author":"R Harpaz","year":"2014","unstructured":"Harpaz R, Callahan A, Tamang S, Low Y, Odgers D, Finlayson S, et al. Text Mining for Adverse Drug Events: the Promise, Challenges, and State of the Art. Drug Safety. 2014;37(10):777\u201390.","journal-title":"Drug Safety"},{"issue":"4","key":"3195_CR8","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1145\/2719920","volume":"47","author":"S Karimi","year":"2015","unstructured":"Karimi S, Wang C, Metke-Jimanez A, Gaire R, Paris C. Text and Data Mining Techniques in Adverse Drug Reaction Detection. ACM Comput Surv. 2015;47(4):56.","journal-title":"ACM Comput Surv"},{"key":"3195_CR9","doi-asserted-by":"publisher","first-page":"109","DOI":"10.2165\/00002018-199920020-00002","volume":"20","author":"EG Brown","year":"1999","unstructured":"Brown EG, Wood L, Wood S. The medical dictionary for regulatory activities (MedDRA). Drug Saf. 1999;20:109\u201317.","journal-title":"Drug Saf"},{"issue":"1","key":"3195_CR10","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1186\/1472-6947-10-66","volume":"10","author":"PM Nadkarni","year":"2010","unstructured":"Nadkarni PM, Nadkarni PM, Darer J. Determining correspondences between high-frequency MedDRA concepts and SNOMED: a case study. BMC Med Inform Decis Mak. 2010;10(1):66.","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"3195_CR11","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1186\/2041-1480-5-29","volume":"5","author":"Y He","year":"2014","unstructured":"He Y, Sarntivijai S, Sarntivijai S, Lin Y, Xiang Z, Guo A, et al. OAE: the ontology of adverse events. J Biomed Semantics. 2014;5(1):29.","journal-title":"J Biomed Semantics"},{"issue":"1","key":"3195_CR12","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1186\/s13326-016-0069-x","volume":"7","author":"A Guo","year":"2016","unstructured":"Guo A, Racz R, Hur J, Lin Y, Xiang Z, Zhao L, et al. Ontology-based collection, representation and analysis of drug-associated neuropathy adverse events. J Biomed Semantics. 2016;7(1):29.","journal-title":"J Biomed Semantics"},{"key":"3195_CR13","volume-title":"Natural language processing with Python: analyzing text with the natural language toolkit","author":"S Bird","year":"2009","unstructured":"Bird S, Loper E, Klein E. Natural language processing with Python: analyzing text with the natural language toolkit; 2009."},{"key":"3195_CR14","doi-asserted-by":"crossref","unstructured":"Sang EFT, Veenstra J. Representing text chunks. In: Proceedings of the ninth conference on European chapter of the Association for Computational Linguistics. Association for Computational Linguistics; 1999. p. 173\u20139. Available from: https:\/\/www.aclweb.org\/anthology\/E99-1023\/.","DOI":"10.3115\/977035.977059"},{"key":"3195_CR15","doi-asserted-by":"crossref","unstructured":"Roberts K, Demner-Fushman D, Tonning JM. Overview of the TAC 2017 Adverse Reaction Extraction from Drug Labels Track. In: Proceedings of the 2017 Text Analysis Conference. NIST; 2017. Available from: https:\/\/tac.nist.gov\/publications\/2017\/additional.papers\/TAC2017.ADRoverview.proceedings.pdf.","DOI":"10.6028\/NIST.SP.500-324.pm-overview"},{"key":"3195_CR16","unstructured":"Xu J, Lee H, Ji Z, Wang J, Wei Q, Xu H. UTH_CCB System for Adverse Drug Reaction Extraction from Drug Labels at TAC-ADR 2017. In: Proceedings of the 2017 Text Analysis Conference. NIST; 2017. Available from: https:\/\/tac.nist.gov\/publications\/2017\/participant.papers\/TAC2017.UTHCCB.proceedings.pdf."},{"key":"3195_CR17","unstructured":"IBM Research System at TAC 2017: Adverse Drug Reactions Extraction from Drug Labels. In: Proceedings of the 2017 Text Analysis Conference. NIST; 2017. Available from: https:\/\/tac.nist.gov\/publications\/2017\/participant.papers\/TAC2017.IBMResearch.proceedings.pdf."},{"key":"3195_CR18","first-page":"166","volume-title":"Proceedings of BioNLP16","author":"B Chiu","year":"2016","unstructured":"Chiu B, Crichton G, Korhonen A, Pyysalo S. How to train good word embeddings for biomedical NLP. In: Proceedings of BioNLP16; 2016. p. 166."},{"issue":"6","key":"3195_CR19","doi-asserted-by":"publisher","first-page":"838","DOI":"10.1093\/bioinformatics\/btp049","volume":"25","author":"J Hur","year":"2009","unstructured":"Hur J, Schuyler AD, States DJ, Feldman EL. SciMiner: web-based literature mining tool for target identification and functional enrichment analysis. Bioinformatics. 2009;25(6):838\u201340.","journal-title":"Bioinformatics"},{"key":"3195_CR20","doi-asserted-by":"crossref","unstructured":"Ma X, Hovy E. End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Berlin: Association for Computational Linguistics; 2016. p. 1064\u201374. Available from: https:\/\/www.aclweb.org\/anthology\/P16-1101.","DOI":"10.18653\/v1\/P16-1101"},{"key":"3195_CR21","doi-asserted-by":"crossref","unstructured":"Reimers N, Gurevych I. Reporting Score Distributions Makes a Difference: Performance Study of LSTM-networks for Sequence Tagging. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark: Association for Computational Linguistics; 2017. p. 338\u201348. Available from: https:\/\/www.aclweb.org\/anthology\/D17-1035.","DOI":"10.18653\/v1\/D17-1035"},{"key":"3195_CR22","unstructured":"Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. In: Advances in neural information processing systems; 2013. p. 3111\u20139."},{"key":"3195_CR23","volume-title":"Distributional Semantics Resources for Biomedical Text Processing","author":"S Pyysalo","year":"2013","unstructured":"Pyysalo S, Ginter F, Moen H, Salakoski T, Ananiadou S. Distributional Semantics Resources for Biomedical Text Processing; 2013 [cited 2017 10 23. Available from: http:\/\/escholar.manchester.ac.uk\/uk-ac-man-scw:267174."},{"issue":"8","key":"3195_CR24","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735\u201380.","journal-title":"Neural Comput"},{"key":"3195_CR25","first-page":"157","volume":"5","author":"Y Bengio","year":"1994","unstructured":"Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE. 1994;5:157\u201366.","journal-title":"IEEE"},{"key":"3195_CR26","volume-title":"On the difficulty of training recurrent neural networks: international conference on machine learning","author":"R Pascanu","year":"2013","unstructured":"Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks: international conference on machine learning; 2013."},{"key":"3195_CR27","doi-asserted-by":"crossref","unstructured":"Lample G, Ballesteros M, Subramanian S, Kawakami K, Dyer C. Neural Architectures for Named Entity Recognition. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego: Association for Computational Linguistics; 2016. p. 260\u201370. Available from: https:\/\/www.aclweb.org\/anthology\/N16-1030.","DOI":"10.18653\/v1\/N16-1030"},{"key":"3195_CR28","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929\u20131958."},{"key":"3195_CR29","unstructured":"Gal Y, Ghahramani Z. A theoretically grounded application of dropout in recurrent neural networks. In: Advances in neural information processing systems; 2016. p. 1019\u201327"},{"key":"3195_CR30","volume-title":"Conditional random fields: probabilistic models for segmenting and labeling sequence data","author":"J Lafferty","year":"2001","unstructured":"Lafferty J, McCallum A, Pereira FC. Conditional random fields: probabilistic models for segmenting and labeling sequence data; 2001."},{"key":"3195_CR31","volume-title":"Incorporating nesterov momentum into adam","author":"T Dozat","year":"2016","unstructured":"Dozat T. Incorporating nesterov momentum into adam; 2016."},{"issue":"1","key":"3195_CR32","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1186\/2041-1480-6-2","volume":"6","author":"J Hur","year":"2015","unstructured":"Hur J, \u00d6zg\u00fcr A, Xiang Z, He Y. Development and application of an interaction network ontology for literature mining of vaccine-associated gene-gene interactions. J Biomed Semantics. 2015;6(1):2.","journal-title":"J Biomed Semantics"},{"issue":"1","key":"3195_CR33","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1186\/1471-2172-12-49","volume":"12","author":"J Hur","year":"2011","unstructured":"Hur J, Xiang Z, Feldman EL, He Y. Ontology-based Brucella vaccine literature indexing and systematic analysis of gene-vaccine association network. BMC Immunol. 2011;12(1):49.","journal-title":"BMC Immunol"},{"issue":"1","key":"3195_CR34","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1186\/s13326-017-0122-4","volume":"8","author":"J Hur","year":"2017","unstructured":"Hur J, \u00d6zg\u00fcr A, He Y. Ontology-based literature mining of E. coli vaccine-associated gene interaction networks. J Biomed Semantics. 2017;8(1):12.","journal-title":"J Biomed Semantics"},{"key":"3195_CR35","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1186\/s13326-018-0185-x","volume":"9","author":"J Hur","year":"2018","unstructured":"Hur J, \u00d6zg\u00fcr A, He Y. Ontology-based literature mining and class effect analysis of adverse drug reactions associated with neuropathy-inducing drugs. J Biomed Semantics. 2018;9:17.","journal-title":"J Biomed Semantics"},{"key":"3195_CR36","doi-asserted-by":"publisher","first-page":"180001","DOI":"10.1038\/sdata.2018.1","volume":"5","author":"D Demner-Fusman","year":"2018","unstructured":"Demner-Fusman D, Shooshan SE, Rodriguez L, Aronson AR, Lang F, Rogers W, et al. A dataset of 200 structured product labels annotated for adverse drug reactions. Scientific Data. 2018;5:180001.","journal-title":"Scientific Data"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-3195-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12859-019-3195-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-3195-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,29]],"date-time":"2024-07-29T01:33:47Z","timestamp":1722216827000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-019-3195-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12]]},"references-count":36,"journal-issue":{"issue":"S21","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["3195"],"URL":"https:\/\/doi.org\/10.1186\/s12859-019-3195-5","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12]]},"assertion":[{"value":"23 December 2019","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"707"}}