{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T18:39:01Z","timestamp":1773081541892,"version":"3.50.1"},"reference-count":29,"publisher":"Oxford University Press (OUP)","issue":"10","license":[{"start":{"date-parts":[[2021,7,16]],"date-time":"2021-07-16T00:00:00Z","timestamp":1626393600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"University of Pennsylvania was supported by the National Institutes of Health (NIH) National Library of Medicine","award":["R01LM011176"],"award-info":[{"award-number":["R01LM011176"]}]},{"name":"Kazan Federal University on BERT-based models and manuscript was supported by the Russian Science Foundation","award":["18-11-00284"],"award-info":[{"award-number":["18-11-00284"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,9,18]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Research on pharmacovigilance from social media data has focused on mining adverse drug events (ADEs) using annotated datasets, with publications generally focusing on 1 of 3 tasks: ADE classification, named entity recognition for identifying the span of ADE mentions, and ADE mention normalization to standardized terminologies. While the common goal of such systems is to detect ADE signals that can be used to inform public policy, it has been impeded largely by limited end-to-end solutions for large-scale analysis of social media reports for different drugs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>We present a dataset for training and evaluation of ADE pipelines where the ADE distribution is closer to the average \u2018natural balance\u2019 with ADEs present in about 7% of the tweets. The deep learning architecture involves an ADE extraction pipeline with individual components for all 3 tasks.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The system presented achieved state-of-the-art performance on comparable datasets and scored a classification performance of F1 = 0.63, span extraction performance of F1 = 0.44 and an end-to-end entity resolution performance of F1 = 0.34 on the presented dataset.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The performance of the models continues to highlight multiple challenges when deploying pharmacovigilance systems that use social media data. We discuss the implications of such models in the downstream tasks of signal detection and suggest future enhancements.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Mining ADEs from Twitter posts using a pipeline architecture requires the different components to be trained and tuned based on input data imbalance in order to ensure optimal performance on the end-to-end resolution task.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocab114","type":"journal-article","created":{"date-parts":[[2021,6,8]],"date-time":"2021-06-08T07:49:30Z","timestamp":1623138570000},"page":"2184-2192","source":"Crossref","is-referenced-by-count":46,"title":["DeepADEMiner: a deep learning pharmacovigilance pipeline for extraction and normalization of adverse drug event mentions on Twitter"],"prefix":"10.1093","volume":"28","author":[{"given":"Arjun","family":"Magge","sequence":"first","affiliation":[{"name":"DBEI, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA"}]},{"given":"Elena","family":"Tutubalina","sequence":"additional","affiliation":[{"name":"Kazan Federal University, Kazan, Russia"}]},{"given":"Zulfat","family":"Miftahutdinov","sequence":"additional","affiliation":[{"name":"Kazan Federal University, Kazan, Russia"}]},{"given":"Ilseyar","family":"Alimova","sequence":"additional","affiliation":[{"name":"Kazan Federal University, Kazan, Russia"}]},{"given":"Anne","family":"Dirkson","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, Leiden, Netherlands"}]},{"given":"Suzan","family":"Verberne","sequence":"additional","affiliation":[{"name":"LIACS, Leiden University, Leiden, Netherlands"}]},{"given":"Davy","family":"Weissenbacher","sequence":"additional","affiliation":[{"name":"DBEI, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA"}]},{"given":"Graciela","family":"Gonzalez-Hernandez","sequence":"additional","affiliation":[{"name":"DBEI, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, USA"}]}],"member":"286","published-online":{"date-parts":[[2021,7,16]]},"reference":[{"issue":"9237","key":"2021091817525397500_ocab114-B1","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1016\/S0140-6736(00)02799-9","article-title":"Adverse drug reactions: definitions, diagnosis, and management","volume":"356","author":"Edwards","year":"2000","journal-title":"Lancet"},{"issue":"3","key":"2021091817525397500_ocab114-B2","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1093\/jamia\/ocu041","article-title":"Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features","volume":"22","author":"Nikfarjam","year":"2015","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817525397500_ocab114-B3","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1016\/j.jbi.2015.02.004","article-title":"Utilizing social media data for pharmacovigilance: a review","volume":"54","author":"Sarker","year":"2015","journal-title":"J Biomed Inform"},{"issue":"4","key":"2021091817525397500_ocab114-B4","doi-asserted-by":"crossref","first-page":"910","DOI":"10.1111\/bcp.12717","article-title":"Social media and pharmacovigilance: a review of the opportunities and challenges","volume":"80","author":"Sloane","year":"2015","journal-title":"Br J Clin Pharmacol"},{"issue":"1","key":"2021091817525397500_ocab114-B5","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1186\/s12911-018-0621-y","article-title":"Utility of social media and crowd-intelligence data for pharmacovigilance: a scoping review","volume":"18","author":"Tricco","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"issue":"2","key":"2021091817525397500_ocab114-B6","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s41060-019-00175-3","article-title":"Harnessing social media data for pharmacovigilance: a review of current state of the art, challenges and future directions","volume":"8","author":"Pappa","year":"2019","journal-title":"Int J Data Sci Anal"},{"key":"2021091817525397500_ocab114-B7","author":"Weissenbacher","year":"31, 2018; ,"},{"key":"2021091817525397500_ocab114-B8","author":"Weissenbacher","year":"2, 2019; ,"},{"key":"2021091817525397500_ocab114-B9","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1016\/j.jbi.2014.11.002","article-title":"Portable automatic text classification for adverse drug reaction detection via multi-corpus training","volume":"53","author":"Sarker","year":"2015","journal-title":"J Biomed Inform"},{"key":"2021091817525397500_ocab114-B10","author":"Wang","year":"30 2018; ,"},{"issue":"10","key":"2021091817525397500_ocab114-B11","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1093\/jamia\/ocy114","article-title":"Data and systems for medication-related text classification and concept normalization from Twitter: insights from the Social Media Mining for Health (SMM4H)-2017 shared task","volume":"25","author":"Sarker","year":"2018","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817525397500_ocab114-B12","volume-title":"Proceedings of the Fifth Social Media Mining for Health Applications Workshop & Shared Task","author":"Gonzalez-Hernandez","year":"2020"},{"issue":"5","key":"2021091817525397500_ocab114-B13","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1007\/s40264-020-00912-9","article-title":"Adverse events in twitter-development of a benchmark reference dataset: results from IMI WEB-RADR","volume":"43","author":"Dietrich","year":"2020","journal-title":"Drug Saf"},{"issue":"6","key":"2021091817525397500_ocab114-B14","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1093\/jamia\/ocz013","article-title":"Comment on: \u201cDeep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts","volume":"26","author":"Magge","year":"2019","journal-title":"J Am Med Inform Assoc"},{"issue":"12","key":"2021091817525397500_ocab114-B15","doi-asserted-by":"crossref","first-page":"1618","DOI":"10.1093\/jamia\/ocz156","article-title":"Deep neural networks ensemble for detecting medication mentions in tweets","volume":"26","author":"Weissenbacher","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2021091817525397500_ocab114-B16","author":"Miftahutdinov","year":"2, 2019; ,"},{"key":"2021091817525397500_ocab114-B17","author":"Limsopatham","year":"7, 2016; ,"},{"key":"2021091817525397500_ocab114-B18","author":"Miftahutdinov","year":"2, 2019; ,"},{"issue":"2","key":"2021091817525397500_ocab114-B19","first-page":"65","article-title":"MedDRA: an overview of the dictionary for regulatory activities","volume":"23","author":"Mozzicato","year":"2009","journal-title":"Pharmaceut Med"},{"key":"2021091817525397500_ocab114-B20","doi-asserted-by":"crossref","DOI":"10.2165\/00002018-199920020-00002","article-title":"The Medical Dictionary for Regulatory Activities (MedDRA)","author":"Brown","year":"1999","journal-title":"Drug Saf"},{"key":"2021091817525397500_ocab114-B21","author":"Liu","year":"2019"},{"key":"2021091817525397500_ocab114-B22","author":"Akbik","year":"20, 2018;  ,  ,"},{"key":"2021091817525397500_ocab114-B23","author":"Devlin","year":"2019"},{"issue":"90001","key":"2021091817525397500_ocab114-B24","doi-asserted-by":"crossref","first-page":"267D","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2021091817525397500_ocab114-B25","first-page":"427","author":"Joulin","year":"2017"},{"key":"2021091817525397500_ocab114-B26","first-page":"1532","article-title":"Glove: global vectors for word representation","author":"Pennington","year":"2014","journal-title":"EMNLP"},{"issue":"11","key":"2021091817525397500_ocab114-B27","doi-asserted-by":"crossref","first-page":"2180","DOI":"10.1007\/s11172-017-2000-8","article-title":"Using semantic analysis of texts for the identification of drugs with similar therapeutic effects","volume":"66","author":"Tutubalina","year":"2017","journal-title":"Russ Chem Bull"},{"key":"2021091817525397500_ocab114-B28","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1162\/tacl_a_00051","article-title":"Enriching word vectors with subword information","volume":"5","author":"Bojanowski","year":"2017","journal-title":"TACL"},{"key":"2021091817525397500_ocab114-B29","author":"Elena","year":"8, 2020; ,  ()"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/10\/2184\/40408928\/ocab114.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/10\/2184\/40408928\/ocab114.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,9,18]],"date-time":"2021-09-18T13:54:53Z","timestamp":1631973293000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/10\/2184\/6322900"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,16]]},"references-count":29,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,7,16]]},"published-print":{"date-parts":[[2021,9,18]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab114","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2020.12.15.20248229","asserted-by":"object"}]},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,10,1]]},"published":{"date-parts":[[2021,7,16]]}}}