{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T09:50:34Z","timestamp":1770285034955,"version":"3.49.0"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2015,11,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Objective Adverse drug events (ADEs) are undesired harmful effects resulting from use of a medication, and occur in 30% of hospitalized patients. The authors have developed a data-mining method for systematic, automated detection of ADEs from electronic medical records.<\/jats:p><jats:p>Materials and Methods This method uses the text from 9.5 million clinical notes, along with prior knowledge of drug usages and known ADEs, as inputs. These inputs are further processed into statistics used by a discriminative classifier which outputs the probability that a given drug\u2013disorder pair represents a valid ADE association. Putative ADEs identified by the classifier are further filtered for positive support in 2 independent, complementary data sources. The authors evaluate this method by assessing support for the predictions in other curated data sources, including a manually curated, time-indexed reference standard of label change events.<\/jats:p><jats:p>Results This method uses a classifier that achieves an area under the curve of 0.94 on a held out test set. The classifier is used on 2\u2009362\u2009950 possible drug\u2013disorder pairs comprised of 1602 unique drugs and 1475 unique disorders for which we had data, resulting in 240 high-confidence, well-supported drug-AE associations. Eighty-seven of them (36%) are supported in at least one of the resources that have information that was not available to the classifier.<\/jats:p><jats:p>Conclusion This method demonstrates the feasibility of systematic post-marketing surveillance for ADEs using electronic medical records, a key component of the learning healthcare system.<\/jats:p>","DOI":"10.1093\/jamia\/ocv102","type":"journal-article","created":{"date-parts":[[2015,11,10]],"date-time":"2015-11-10T07:37:40Z","timestamp":1447141060000},"page":"1196-1204","source":"Crossref","is-referenced-by-count":57,"title":["A method for systematic discovery of adverse drug events from clinical notes"],"prefix":"10.1093","volume":"22","author":[{"given":"Guan","family":"Wang","sequence":"first","affiliation":[{"name":"Stanford University, Center for Biomedical Informatics, Stanford, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenneth","family":"Jung","sequence":"additional","affiliation":[{"name":"Stanford University, Center for Biomedical Informatics, Stanford, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rainer","family":"Winnenburg","sequence":"additional","affiliation":[{"name":"Stanford University, Center for Biomedical Informatics, Stanford, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nigam H","family":"Shah","sequence":"additional","affiliation":[{"name":"Stanford University, Center for Biomedical Informatics, Stanford, California, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2015,7,31]]},"reference":[{"issue":"4","key":"2020110612595145300_ocv102-B1","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1001\/jama.1997.03540280039031","article-title":"Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality","volume":"277","author":"Classen","year":"1997","journal-title":"JAMA."},{"issue":"4","key":"2020110612595145300_ocv102-B2","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1377\/hlthaff.2011.0190","article-title":"\u2018Global trigger tool' shows that adverse events in hospitals may be ten times greater than previously measured","volume":"30","author":"Classen","year":"2011","journal-title":"Health Aff."},{"issue":"15","key":"2020110612595145300_ocv102-B3","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1001\/jama.279.15.1200","article-title":"Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies","volume":"279","author":"Lazarou","year":"1998","journal-title":"JAMA."},{"issue":"125","key":"2020110612595145300_ocv102-B4","doi-asserted-by":"crossref","first-page":"125ra31","DOI":"10.1126\/scitranslmed.3003377","article-title":"Data-driven prediction of drug effects and interactions","volume":"4","author":"Tatonetti","year":"2012","journal-title":"Sci Transl Med."},{"issue":"6","key":"2020110612595145300_ocv102-B5","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1038\/clpt.2013.47","article-title":"Pharmacovigilance using clinical notes","volume":"93","author":"Lependu","year":"2013","journal-title":"Clin Pharmacol Therapeutics."},{"issue":"10","key":"2020110612595145300_ocv102-B6","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1007\/s40264-014-0218-z","article-title":"Text mining for adverse drug events: the promise, challenges, and state of the art","volume":"37","author":"Harpaz","year":"2014","journal-title":"Drug Safety."},{"key":"2020110612595145300_ocv102-B7","first-page":"1","article-title":"Discovering novel adverse drug events using natural language processing and mining of the electronic health record","author":"Friedman","year":"2009","journal-title":"AMIE 2009: Proceedings of the 12th Conference on Artificial Intelligence In Medicine."},{"issue":"30","key":"2020110612595145300_ocv102-B8","doi-asserted-by":"crossref","first-page":"4401","DOI":"10.1002\/sim.5620","article-title":"Empirical assessment of methods for risk identification in healthcare data: results from the experiments of the Observational Medical Outcomes Partnership","volume":"31","author":"Ryan","year":"2012","journal-title":"Stat Med."},{"issue":"8","key":"2020110612595145300_ocv102-B9","doi-asserted-by":"crossref","first-page":"e1002614","DOI":"10.1371\/journal.pcbi.1002614","article-title":"Literature based drug interaction prediction with clinical assessment using electronic medical records: novel myopathy associated drug interactions","volume":"8","author":"Duke","year":"2012","journal-title":"PLoS Comput Biol."},{"issue":"57","key":"2020110612595145300_ocv102-B10","doi-asserted-by":"crossref","first-page":"57cm29","DOI":"10.1126\/scitranslmed.3001456","article-title":"Achieving a nationwide learning health system","volume":"2","author":"Friedman","year":"2010","journal-title":"Sci Transl Med."},{"issue":"6","key":"2020110612595145300_ocv102-B11","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1038\/clpt.2013.24","article-title":"Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system","volume":"93","author":"Harpaz","year":"2013","journal-title":"Clin Pharmacol Therapeutics."},{"key":"2020110612595145300_ocv102-B12","first-page":"e76","article-title":"Medication-wide association studies","volume":"2","author":"Ryan","year":"2013","journal-title":"CPT: Pharmacometrics Syst Pharmacol."},{"key":"2020110612595145300_ocv102-B13","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/1472-6947-10-10","article-title":"Assessing the accuracy of an inter-institutional automated patient-specific health problem list","volume":"10","author":"Poissant","year":"2010","journal-title":"BMC Med Informat Dec Mak."},{"issue":"3","key":"2020110612595145300_ocv102-B14","first-page":"328","article-title":"Active computerized pharmacovigilance using natural language processing, statistics, and electronic health records: a feasibility study","volume":"16","author":"Wang","year":"2009","journal-title":"JAMIA."},{"issue":"2","key":"2020110612595145300_ocv102-B15","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1002\/sim.5925","article-title":"Interpreting observational studies: why empirical calibration is needed to correct p-values","volume":"33","author":"Schuemie","year":"2014","journal-title":"Stats Med."},{"issue":"8","key":"2020110612595145300_ocv102-B16","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1007\/s40264-014-0204-5","article-title":"Improved statistical signal detection in pharmacovigilance by combining multiple strength-of-evidence aspects in vigiRank","volume":"37","author":"Caster","year":"2014","journal-title":"Drug Safety."},{"issue":"3","key":"2020110612595145300_ocv102-B17","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1038\/clpt.2013.107","article-title":"Logistic regression in signal detection: another piece added to the puzzle","volume":"94","author":"Caster","year":"2013","journal-title":"Clin Pharmacol Therapeutics."},{"issue":"3","key":"2020110612595145300_ocv102-B18","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1038\/clpt.2013.125","article-title":"Response to \u201cLogistic regression in signal detection: another piece added to the puzzle\"","volume":"94","author":"Harpaz","year":"2013","journal-title":"Clin Pharmacol Therapeutics."},{"issue":"114","key":"2020110612595145300_ocv102-B19","doi-asserted-by":"crossref","first-page":"114ra27","DOI":"10.1126\/scitranslmed.3002774","article-title":"Predicting adverse drug events using pharmacological network models","volume":"3","author":"Cami","year":"2011","journal-title":"Sci Transl Med."},{"key":"2020110612595145300_ocv102-B20","first-page":"47","article-title":"Using temporal patterns in medical records to discern adverse drug events from indications","volume":"2012","author":"Liu","year":"2012","journal-title":"AMIA Summits on Translational Science proceedings AMIA Summit on Translational Science."},{"key":"2020110612595145300_ocv102-B21","first-page":"391","article-title":"STRIDE\u2013An integrated standards-based translational research informatics platform","volume":"2009","author":"Lowe","year":"2009","journal-title":"AMIA Annual Symposium proceedings \/ AMIA Symposium AMIA Symposium."},{"issue":"2","key":"2020110612595145300_ocv102-B22","doi-asserted-by":"crossref","first-page":"e89324","DOI":"10.1371\/journal.pone.0089324","article-title":"Automated detection of off-label drug use","volume":"9","author":"Jung","year":"2014","journal-title":"PloS ONE."},{"key":"2020110612595145300_ocv102-B23","doi-asserted-by":"crossref","first-page":"S5","DOI":"10.1186\/2041-1480-3-S1-S5","article-title":"Annotation analysis for testing drug safety signals using unstructured clinical notes","volume":"3","author":"Lependu","year":"2012","journal-title":"J Biomed Semantics."},{"issue":"1","key":"2020110612595145300_ocv102-B24","first-page":"121","article-title":"Functional evaluation of out-of-the-box text-mining tools for data-mining tasks","volume":"22","author":"Jung","year":"2014","journal-title":"JAMIA."},{"key":"2020110612595145300_ocv102-B25","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1006\/jbin.2001.1029","article-title":"A simple algorithm for identifying negated findings and diseases in discharge summaries","volume":"34","author":"Chapman","year":"2001","journal-title":"J BiomedI Inform."},{"key":"2020110612595145300_ocv102-B26","first-page":"81","article-title":"ConText: an algorithm for identifying contextual features from clinical text","author":"Chapman","year":"2007","journal-title":"Proceedings of the Workshop on BioNLP."},{"issue":"4","key":"2020110612595145300_ocv102-B27","first-page":"441","article-title":"Normalized names for clinical drugs: RxNorm at 6 years","volume":"18","author":"Nelson","year":"2011","journal-title":"JAMIA."},{"issue":"1","key":"2020110612595145300_ocv102-B28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw."},{"key":"2020110612595145300_ocv102-B29","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1023\/A:1009715923555","article-title":"A tutorial on support vector machines for pattern recognition","volume":"2","author":"Burges","year":"1998","journal-title":"Data Min KnowlDisc."},{"issue":"1","key":"2020110612595145300_ocv102-B30","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn."},{"issue":"3","key":"2020110612595145300_ocv102-B31","first-page":"18","article-title":"Classification and regression by randomforest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News."},{"key":"2020110612595145300_ocv102-B32","author":"Friedman"},{"key":"2020110612595145300_ocv102-B33","author":"Meyer"},{"key":"2020110612595145300_ocv102-B34","author":"Liaw"},{"issue":"Pt 1","key":"2020110612595145300_ocv102-B35","first-page":"477","article-title":"VA National Drug File Reference Terminology: a cross-institutional content coverage study","volume":"107","author":"Brown","year":"2004","journal-title":"Stud Health Technol Inform."},{"key":"2020110612595145300_ocv102-B36"},{"issue":"3","key":"2020110612595145300_ocv102-B37","first-page":"446","article-title":"Design and validation of an automated method to detect known adverse drug reactions in MEDLINE: a contribution from the EU-ADR project","volume":"20","author":"Avillach","year":"2013","journal-title":"JAMIA."},{"key":"2020110612595145300_ocv102-B38","doi-asserted-by":"crossref","first-page":"140043","DOI":"10.1038\/sdata.2014.43","article-title":"A time-indexed reference standard of adverse drug reactions","volume":"1","author":"Harpaz","year":"2014","journal-title":"Scientific Data."},{"issue":"3","key":"2020110612595145300_ocv102-B39","first-page":"413","article-title":"Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions","volume":"20","author":"Harpaz","year":"2013","journal-title":"JAMIA."},{"issue":"1","key":"2020110612595145300_ocv102-B40","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s40264-014-0245-9","article-title":"Comment on: \u201cZoo or savannah? Choice of training ground for evidence-based pharmacovigilance\"","volume":"38","author":"Harpaz","year":"2015","journal-title":"Drug Safety."},{"issue":"9","key":"2020110612595145300_ocv102-B41","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1007\/s40264-014-0198-z","article-title":"Zoo or savannah? Choice of training ground for evidence-based pharmacovigilance","volume":"37","author":"Noren","year":"2014","journal-title":"Drug Safety."},{"issue":"1","key":"2020110612595145300_ocv102-B42","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s40264-014-0246-8","article-title":"Authors' reply to Harpaz et\u00a0al . comment on: \u201cZoo or savannah? Choice of training ground for evidence-based pharmacovigilance\"","volume":"38","author":"Noren","year":"2015","journal-title":"Drug Safety."},{"key":"2020110612595145300_ocv102-B43","doi-asserted-by":"crossref","first-page":"1339","DOI":"10.1145\/2487575.2488214","article-title":"Empirical bayes model to combine signals of adverse drug reactions","author":"Harpaz","year":"2013","journal-title":"Knowledge Discovery and Data Mining'13."}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/6\/1196\/34146523\/ocv102.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/22\/6\/1196\/34146523\/ocv102.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,25]],"date-time":"2022-05-25T19:15:34Z","timestamp":1653506134000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/22\/6\/1196\/2357899"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,7,31]]},"references-count":43,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2015,7,31]]},"published-print":{"date-parts":[[2015,11,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocv102","relation":{},"ISSN":["1527-974X","1067-5027"],"issn-type":[{"value":"1527-974X","type":"electronic"},{"value":"1067-5027","type":"print"}],"subject":[],"published-other":{"date-parts":[[2015,11]]},"published":{"date-parts":[[2015,7,31]]}}}