{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,25]],"date-time":"2025-10-25T12:22:52Z","timestamp":1761394972749,"version":"3.37.3"},"reference-count":28,"publisher":"Oxford University Press (OUP)","issue":"5","funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["T32 GM007365"],"award-info":[{"award-number":["T32 GM007365"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2017,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Objective: As the US Food and Drug Administration (FDA) receives over a million adverse event reports associated with medication use every year, a system is needed to aid FDA safety evaluators in identifying reports most likely to demonstrate causal relationships to the suspect medications. We combined text mining with machine learning to construct and evaluate such a system to identify medication-related adverse event reports.<\/jats:p>\n               <jats:p>Methods: FDA safety evaluators assessed 326 reports for medication-related causality. We engineered features from these reports and constructed random forest, L1 regularized logistic regression, and support vector machine models. We evaluated model accuracy and further assessed utility by generating report rankings that represented a prioritized report review process.<\/jats:p>\n               <jats:p>Results: Our random forest model showed the best performance in report ranking and accuracy, with an area under the receiver operating characteristic curve of 0.66. The generated report ordering assigns reports with a higher probability of medication-related causality a higher rank and is significantly correlated to a perfect report ordering, with a Kendall\u2019s tau of 0.24 (P\u2009=\u2009.002).<\/jats:p>\n               <jats:p>Conclusion: Our models produced prioritized report orderings that enable FDA safety evaluators to focus on reports that are more likely to contain valuable medication-related adverse event information. Applying our models to all FDA adverse event reports has the potential to streamline the manual review process and greatly reduce reviewer workload.<\/jats:p>","DOI":"10.1093\/jamia\/ocx022","type":"journal-article","created":{"date-parts":[[2017,2,24]],"date-time":"2017-02-24T20:10:20Z","timestamp":1487967020000},"page":"913-920","source":"Crossref","is-referenced-by-count":31,"title":["Development of an automated assessment tool for MedWatch reports in the FDA adverse event reporting system"],"prefix":"10.1093","volume":"24","author":[{"given":"Lichy","family":"Han","sequence":"first","affiliation":[{"name":"Biomedical Informatics Training Program, Stanford University, Stanford, CA, USA"}]},{"given":"Robert","family":"Ball","sequence":"additional","affiliation":[{"name":"Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA"}]},{"given":"Carol A","family":"Pamer","sequence":"additional","affiliation":[{"name":"Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA"}]},{"given":"Russ B","family":"Altman","sequence":"additional","affiliation":[{"name":"Department of Genetics, Stanford University"},{"name":"Department of Bioengineering, Stanford University"}]},{"given":"Scott","family":"Proestel","sequence":"additional","affiliation":[{"name":"Office of Surveillance and Epidemiology, Center for Drug Evaluation and Research, Food and Drug Administration, Silver Spring, MD, USA"}]}],"member":"286","published-online":{"date-parts":[[2017,3,21]]},"reference":[{"key":"2020110612444749100_ocx022-B1","unstructured":"US Food and Drug Administration. FDA Adverse Event Reporting System (FAERS). 2016. http:\/\/www.fda.gov.laneproxy.stanford.edu\/Drugs\/GuidanceComplianceRegulatoryInformation\/Surveillance\/AdverseDrugEffects\/. Accessed November 8, 2016."},{"key":"2020110612444749100_ocx022-B2","doi-asserted-by":"crossref","first-page":"2765","DOI":"10.1001\/jama.1993.03500210065033","article-title":"Introducing MEDWatch. A new approach to reporting medication and device adverse effects and product problems","volume":"269","author":"Kessler","year":"1993","journal-title":"JAMA."},{"key":"2020110612444749100_ocx022-B3","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1056\/NEJMNEJMhpr011493","article-title":"Reporting of adverse events","volume":"347","author":"Leape","year":"2002","journal-title":"N Engl J Med."},{"key":"2020110612444749100_ocx022-B4","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1002\/pds.3395","article-title":"Evaluation of FDA safety-related drug label changes in 2010","volume":"22","author":"Lester","year":"2013","journal-title":"Pharmacoepidemiol Drug Saf."},{"key":"2020110612444749100_ocx022-B5","doi-asserted-by":"crossref","first-page":"981","DOI":"10.2165\/00002018-200528110-00002","article-title":"Perspectives on the use of data mining in pharmaco-vigilance","volume":"28","author":"Almenoff","year":"2005","journal-title":"Drug Saf."},{"key":"2020110612444749100_ocx022-B6","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1093\/jamia\/ocv063","article-title":"Use of data mining at the Food and Drug Administration","volume":"23","author":"Duggirala","year":"2015","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612444749100_ocx022-B7","unstructured":"The Use of the WHO-UMC System for Standardised Case Causality Assessment. 2012. http:\/\/who-umc.org\/Graphics\/26649.pdf. Accessed November 8, 2016."},{"key":"2020110612444749100_ocx022-B8","first-page":"1","author":"World Health Organization Collaborating Centre","year":"2003","journal-title":"Introduction to Drug Utilization Research"},{"key":"2020110612444749100_ocx022-B9","doi-asserted-by":"crossref","first-page":"109","DOI":"10.2165\/00002018-199920020-00002","article-title":"The Medical Dictionary for Regulatory Activities (MedDRA)","volume":"20","author":"Brown","year":"1999","journal-title":"Drug Saf."},{"key":"2020110612444749100_ocx022-B10","doi-asserted-by":"crossref","first-page":"55","DOI":"10.3115\/v1\/P14-5010","article-title":"The Stanford CoreNLP Natural Language Processing Toolkit","author":"Manning","year":"2014","journal-title":"Proc 52nd Annu Meet Assoc Comput Linguist Syst Demonstr."},{"key":"2020110612444749100_ocx022-B11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization Paths for Generalized Linear Models via Coordinate Descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw."},{"key":"2020110612444749100_ocx022-B12","first-page":"18","article-title":"Classification and Regression by randomForest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News."},{"journal-title":"Caret: Classification and Regression Training","author":"Kuhn","key":"2020110612444749100_ocx022-B13"},{"key":"2020110612444749100_ocx022-B14","first-page":"1137","article-title":"A study of cross-validation and bootstrap for accuracy estimation and model selection","volume":"2","author":"Kohavi","year":"1995","journal-title":"Int Jt Conf Artif Intell."},{"key":"2020110612444749100_ocx022-B15","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-84858-7","volume-title":"The Elements of Statistical Learning","author":"Hastie","year":"2009","edition":"2nd ed"},{"key":"2020110612444749100_ocx022-B16","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1136\/amiajnl-2012-000881","article-title":"Vaccine adverse event text mining system for extracting features from vaccine safety reports","volume":"19","author":"Botsis","year":"2012","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612444749100_ocx022-B17","first-page":"144","article-title":"Can natural language processing improve the efficiency of vaccine adverse event report review?","volume":"55","author":"Baer","year":"2015","journal-title":"Methods Inf Med."},{"key":"2020110612444749100_ocx022-B18","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1136\/amiajnl-2010-000022","article-title":"Text mining for the Vaccine Adverse Event Reporting System: medical text classification using informative feature selection","volume":"18","author":"Botsis","year":"2011","journal-title":"J Am Med Inform Assoc."},{"key":"2020110612444749100_ocx022-B19","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1007\/s40264-013-0064-4","article-title":"Application of information retrieval approaches to case classification in the vaccine adverse event reporting system","volume":"36","author":"Botsis","year":"2013","journal-title":"Drug Saf."},{"key":"2020110612444749100_ocx022-B20","doi-asserted-by":"crossref","first-page":"88","DOI":"10.4338\/ACI-2012-11-RA-0049","article-title":"The contribution of the vaccine adverse event text mining system to the classification of possible Guillain-Barr\u00e9 syndrome reports","volume":"4","author":"Botsis","year":"2013","journal-title":"Appl Clin Inform."},{"key":"2020110612444749100_ocx022-B21","doi-asserted-by":"crossref","first-page":"125ra31","DOI":"10.1126\/scitranslmed.3003377","article-title":"Data-driven prediction of drug effects and interactions","volume":"4","author":"Tatonetti","year":"2012","journal-title":"Sci Transl Med."},{"key":"2020110612444749100_ocx022-B22","doi-asserted-by":"crossref","first-page":"137","DOI":"10.2165\/00002018-200932020-00006","article-title":"Drug-versus-drug adverse event rate comparisons","volume":"32","author":"Hochberg","year":"2009","journal-title":"Drug-Safety."},{"key":"2020110612444749100_ocx022-B23","doi-asserted-by":"crossref","first-page":"S7","DOI":"10.1186\/1471-2105-11-S9-S7","article-title":"Mining multi-item drug adverse effect associations in spontaneous reporting systems","volume":"11","author":"Harpaz","year":"2010","journal-title":"BMC Bioinformatics."},{"key":"2020110612444749100_ocx022-B24","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1186\/1471-2105-15-17","article-title":"Large-scale combining signals from both biomedical literature and the FDA Adverse Event Reporting System (FAERS) to improve post-marketing drug safety signal detection","volume":"15","author":"Xu","year":"2014","journal-title":"BMC Bioinformatics."},{"key":"2020110612444749100_ocx022-B25","doi-asserted-by":"crossref","first-page":"796","DOI":"10.7150\/ijms.6048","article-title":"Data mining of the public version of the FDA adverse event reporting system","volume":"10","author":"Sakaeda","year":"2013","journal-title":"Int J Med Sci."},{"key":"2020110612444749100_ocx022-B26","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1002\/pds.1746","article-title":"Drug-induced torsades de pointes: data mining of the public version of the FDA Adverse Event Reporting System (AERS)","volume":"18","author":"Poluzzi","year":"2009","journal-title":"Pharmacoepidemiol Drug Saf."},{"key":"2020110612444749100_ocx022-B27","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1186\/2041-1480-5-36","article-title":"Standardizing adverse drug event reporting data","volume":"5","author":"Wang","year":"2014","journal-title":"J Biomed Semantics."},{"key":"2020110612444749100_ocx022-B28","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The Unified Medical Language System (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/5\/913\/34149301\/ocx022.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/24\/5\/913\/34149301\/ocx022.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,6]],"date-time":"2020-11-06T18:23:37Z","timestamp":1604687017000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/24\/5\/913\/3076824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,21]]},"references-count":28,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2017,3,21]]},"published-print":{"date-parts":[[2017,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocx022","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"type":"print","value":"1067-5027"},{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2017,9]]},"published":{"date-parts":[[2017,3,21]]}}}