{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,17]],"date-time":"2026-08-17T23:24:13Z","timestamp":1787009053596,"version":"build-2736575974"},"reference-count":36,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2020,11,5]],"date-time":"2020-11-05T00:00:00Z","timestamp":1604534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"National Institute of Health"},{"name":"Dietary Supplements"},{"DOI":"10.13039\/100000049","name":"National Institute on Aging","doi-asserted-by":"publisher","award":["R01AT009457"],"award-info":[{"award-number":["R01AT009457"]}],"id":[{"id":"10.13039\/100000049","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Clinical and Translational Science Award program","award":["UL1TR002494"],"award-info":[{"award-number":["UL1TR002494"]}]},{"DOI":"10.13039\/100008460","name":"National Center for Complementary and Integrative Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008460","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Dietary Supplements"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>We sought to demonstrate the feasibility of utilizing deep learning models to extract safety signals related to the use of dietary supplements (DSs) in clinical text.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>Two tasks were performed in this study. For the named entity recognition (NER) task, Bi-LSTM-CRF (bidirectional long short-term memory conditional random field) and BERT (bidirectional encoder representations from transformers) models were trained and compared with CRF model as a baseline to recognize the named entities of DSs and events from clinical notes. In the relation extraction (RE) task, 2 deep learning models, including attention-based Bi-LSTM and convolutional neural network as well as a random forest model were trained to extract the relations between DSs and events, which were categorized into 3 classes: positive (ie, indication), negative (ie, adverse events), and not related. The best performed NER and RE models were further applied on clinical notes mentioning 88 DSs for discovering DSs adverse events and indications, which were compared with a DS knowledge base.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>For the NER task, deep learning models achieved a better performance than CRF, with F1 scores above 0.860. The attention-based Bi-LSTM model performed the best in the RE task, with an F1 score of 0.893. When comparing DS event pairs generated by the deep learning models with the knowledge base for DSs and event, we found both known and unknown pairs.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Deep learning models can detect adverse events and indication of DSs in clinical notes, which hold great potential for monitoring the safety of DS use.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa218","type":"journal-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T15:09:45Z","timestamp":1597936185000},"page":"569-577","source":"Crossref","is-referenced-by-count":23,"title":["Deep learning approaches for extracting adverse events and indications of dietary supplements from clinical text"],"prefix":"10.1093","volume":"28","author":[{"given":"Yadan","family":"Fan","sequence":"first","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sicheng","family":"Zhou","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yifan","family":"Li","sequence":"additional","affiliation":[{"name":"College of Pharmacy, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8258-3585","authenticated-orcid":false,"given":"Rui","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute for Health Informatics, University of Minnesota, Minneapolis, Minnesota, USA"},{"name":"College of Pharmacy, University of Minnesota, Minneapolis, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,11,5]]},"reference":[{"key":"2021030612311819300_ocaa218-B1"},{"issue":"3","key":"2021030612311819300_ocaa218-B2","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1002\/cpt.1113","article-title":"Challenges and opportunities for improving the safety assessment of botanical dietary supplements: a United States Pharmacopeia Perspective","volume":"104","author":"Oketch-Rabah","year":"2018","journal-title":"Clin Pharmacol Ther"},{"issue":"5","key":"2021030612311819300_ocaa218-B3","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1177\/1060028017744316","article-title":"Dietary supplement adverse event report data from the FDA center for food safety and applied nutrition adverse event reporting system (CAERS), 2004-2013","volume":"52","author":"Timbo","year":"2018","journal-title":"Ann Pharmacother"},{"issue":"6","key":"2021030612311819300_ocaa218-B4","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1038\/clpt.2013.24","article-title":"Performance of pharmacovigilance signal-detection algorithms for the FDA adverse event reporting system","volume":"93","author":"Harpaz","year":"2013","journal-title":"Clin Pharmacol Ther"},{"issue":"1","key":"2021030612311819300_ocaa218-B5","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/1472-6947-10-10","article-title":"Assessing the accuracy of an inter-institutional automated patient-specific health problem list","volume":"10","author":"Poissant","year":"2010","journal-title":"BMC Med Inform Decis Mak"},{"issue":"5","key":"2021030612311819300_ocaa218-B6","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1136\/jamia.2010.003947","article-title":"Extracting medication information from clinical text","volume":"17","author":"Uzuner","year":"2010","journal-title":"J Am Med Inform Assoc"},{"key":"2021030612311819300_ocaa218-B7"},{"issue":"1","key":"2021030612311819300_ocaa218-B8","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/s40264-018-0762-z","article-title":"Overview of the first natural language processing challenge for extracting medication, indication, and adverse drug events from electronic health record notes (MADE 1.0","volume":"42","author":"Jagannatha","year":"2019","journal-title":"Drug Saf"},{"key":"2021030612311819300_ocaa218-B9","author":"Grouin","year":"2009"},{"key":"2021030612311819300_ocaa218-B10","author":"Roberts","year":"2010"},{"key":"2021030612311819300_ocaa218-B11","author":"Patrick","year":"2009"},{"key":"2021030612311819300_ocaa218-B12","author":"Chalapathy","year":"2016"},{"issue":"14","key":"2021030612311819300_ocaa218-B13","doi-asserted-by":"crossref","first-page":"i37","DOI":"10.1093\/bioinformatics\/btx228","article-title":"Deep learning with word embeddings improves biomedical named entity recognition","volume":"33","author":"Habibi","year":"2017","journal-title":"Bioinformatics"},{"key":"2021030612311819300_ocaa218-B14","article-title":"A review of relation extraction","author":"Bach","year":"2007","journal-title":"Literature review for Language and Statistics II"},{"issue":"1","key":"2021030612311819300_ocaa218-B15","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3390\/nu10010041","article-title":"Dietary supplements: regulatory challenges and research resources","volume":"10","author":"Dwyer","year":"2018","journal-title":"Nutrients"},{"key":"2021030612311819300_ocaa218-B16","doi-asserted-by":"crossref","DOI":"10.1093\/jamia\/ocaa128","article-title":"Assessing the enrichment of dietary supplement coverage in the UMLS","author":"Vasilakes","year":"2020","journal-title":"J Am Med Inform Assoc"},{"issue":"2","key":"2021030612311819300_ocaa218-B17","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1093\/jamiaopen\/ooz007","article-title":"Using word embeddings to expand terminology of dietary supplements on clinical notes","volume":"2","author":"Fan","year":"2019","journal-title":"JAMIA Open"},{"key":"2021030612311819300_ocaa218-B18","first-page":"258","article-title":"Detecting signals of dietary supplement adverse events from the CFSAN Adverse Event Reporting System (CAERS)","volume":"2019","author":"Vasilakes","year":"2019","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"2021030612311819300_ocaa218-B19","first-page":"53","author":"Wang","year":"2018"},{"key":"2021030612311819300_ocaa218-B20","first-page":"614","article-title":"Mining adverse events of dietary supplements from product labels by topic modeling","volume":"245","author":"Wang","year":"2017","journal-title":"Stud Health Technol Inform"},{"key":"2021030612311819300_ocaa218-B21","first-page":"408","article-title":"Normalizing dietary supplement product names using the RxNorm model","volume":"264","author":"Vasilakes","year":"2019","journal-title":"Stud Health Technol Inform"},{"issue":"18","key":"2021030612311819300_ocaa218-B22","doi-asserted-by":"crossref","first-page":"E4304","DOI":"10.1073\/pnas.1803294115","article-title":"Deep learning improves prediction of drug\u2013drug and drug\u2013food interactions","volume":"115","author":"Ryu","year":"2018","journal-title":"Proc Natl Acad Sci U S A"},{"issue":"4","key":"2021030612311819300_ocaa218-B23","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1093\/jamia\/ocz216","article-title":"iDISK: the integrated DIetary Supplements Knowledge base","volume":"27","author":"Rizvi","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"2021030612311819300_ocaa218-B24"},{"key":"2021030612311819300_ocaa218-B25","author":"Lample","year":"2016"},{"key":"2021030612311819300_ocaa218-B26","author":"Ma","year":"2016"},{"key":"2021030612311819300_ocaa218-B27","author":"Devlin","year":"2018"},{"key":"2021030612311819300_ocaa218-B28"},{"key":"2021030612311819300_ocaa218-B29","author":"Clark","year":"2019"},{"key":"2021030612311819300_ocaa218-B30","author":"Alsentzer","year":"2019"},{"key":"2021030612311819300_ocaa218-B31","first-page":"39","author":"Nguyen","year":"2015"},{"key":"2021030612311819300_ocaa218-B32","first-page":"207","author":"Zhou","year":"2016"},{"issue":"6","key":"2021030612311819300_ocaa218-B33","doi-asserted-by":"crossref","first-page":"547","DOI":"10.1038\/clpt.2013.47","article-title":"Pharmacovigilance using clinical notes","volume":"93","author":"LePendu","year":"2013","journal-title":"Clin Pharmacol Ther"},{"issue":"2","key":"2021030612311819300_ocaa218-B34","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1038\/clpt.2012.54","article-title":"Detection of pharmacovigilance-related adverse events using electronic health records and automated methods","volume":"92","author":"Haerian","year":"2012","journal-title":"Clin Pharmacol Ther"},{"key":"2021030612311819300_ocaa218-B35","first-page":"57","article-title":"UArizona at the MADE1. 0 NLP Challenge","volume":"90","author":"Xu","year":"2018","journal-title":"Proc Mach Learn Res"},{"key":"2021030612311819300_ocaa218-B36","first-page":"167","author":"Yadav","year":"2018"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/3\/569\/36428649\/ocaa218.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/3\/569\/36428649\/ocaa218.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,6]],"date-time":"2021-03-06T07:32:18Z","timestamp":1615015938000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/3\/569\/5956340"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,5]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,11,5]]},"published-print":{"date-parts":[[2021,3,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa218","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,3,1]]},"published":{"date-parts":[[2020,11,5]]}}}